[jira] [Resolved] (NIFI-7062) Implement native multipart/form-data handling
[
https://issues.apache.org/jira/browse/NIFI-7062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann resolved NIFI-7062.
Resolution: Implemented
Thanks for raising the question [~esecules], it looks like multipart/form-data
support for InvokeHTTP was implemented in NIFI-7394. Item 3 described in this
issue would require additional work, but it seems like that would better
handled in a separate issue, if necessary. Closing this issue as implemented.
> Implement native multipart/form-data handling
> -
>
> Key: NIFI-7062
> URL: https://issues.apache.org/jira/browse/NIFI-7062
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Extensions
>Affects Versions: 1.10.0
>Reporter: Andy LoPresto
>Priority: Major
> Labels: http, multipart/form-data
>
> Multiple users have expressed difficultly configuring the {{InvokeHTTP}}
> processor to send {{multipart/form-data}} content to remote APIs.
> While it is possible using a combination of {{ReplaceText}} processor to
> build the content boundaries and a custom header via a dynamic property in
> {{InvokeHTTP}}, this is not an ideal experience.
> Rather, this could be implemented by:
> # a native boolean property on {{InvokeHTTP}} which selects
> "multipart/form-data" (default *false*)
> # a multiple value property which has options
> *application/x-www-form-urlencoded* (default), *multipart/form-data*, and
> *text/plain*
> # a new processor which handles incoming flowfiles and modifies the content
> and attributes to be prepared for this transmission -- this could be
> implemented for a single piece of content per flowfile or using the Record
> mechanism
> Notes:
> * Implementing the properties directly on the {{InvokeHTTP}} processor
> eliminates the need for an additional processor to prepare the data and does
> not prevent continuing operation of the same flowfile downstream in a linear
> flow. However, it does change the current scenario where "flowfile content is
> _exactly_ what is sent" because it would "invisibly" wrap the content in the
> boundary markers. This could be remediated with a custom content viewer or
> custom UI
> References:
> * https://stackoverflow.com/a/4526286/70465
> *
> https://dev.to/sidthesloth92/understanding-html-form-encoding-url-encoded-and-multipart-forms-3lpa
> * https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/POST
> * https://tools.ietf.org/html/rfc2388
> *
> https://community.cloudera.com/t5/Support-Questions/How-to-send-an-http-POST-multipart-form-data-request-with-a/td-p/240035
> *
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (NIFI-7062) Implement native multipart/form-data handling
[
https://issues.apache.org/jira/browse/NIFI-7062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17317538#comment-17317538
]
Eric Secules commented on NIFI-7062:
Is this ticket resolvable? Im looking at the docs for nifi 1.13 and I see
mention of multipart form data.
[https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.13.0/org.apache.nifi.processors.standard.InvokeHTTP/]
> Implement native multipart/form-data handling
> -
>
> Key: NIFI-7062
> URL: https://issues.apache.org/jira/browse/NIFI-7062
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Extensions
>Affects Versions: 1.10.0
>Reporter: Andy LoPresto
>Priority: Major
> Labels: http, multipart/form-data
>
> Multiple users have expressed difficultly configuring the {{InvokeHTTP}}
> processor to send {{multipart/form-data}} content to remote APIs.
> While it is possible using a combination of {{ReplaceText}} processor to
> build the content boundaries and a custom header via a dynamic property in
> {{InvokeHTTP}}, this is not an ideal experience.
> Rather, this could be implemented by:
> # a native boolean property on {{InvokeHTTP}} which selects
> "multipart/form-data" (default *false*)
> # a multiple value property which has options
> *application/x-www-form-urlencoded* (default), *multipart/form-data*, and
> *text/plain*
> # a new processor which handles incoming flowfiles and modifies the content
> and attributes to be prepared for this transmission -- this could be
> implemented for a single piece of content per flowfile or using the Record
> mechanism
> Notes:
> * Implementing the properties directly on the {{InvokeHTTP}} processor
> eliminates the need for an additional processor to prepare the data and does
> not prevent continuing operation of the same flowfile downstream in a linear
> flow. However, it does change the current scenario where "flowfile content is
> _exactly_ what is sent" because it would "invisibly" wrap the content in the
> boundary markers. This could be remediated with a custom content viewer or
> custom UI
> References:
> * https://stackoverflow.com/a/4526286/70465
> *
> https://dev.to/sidthesloth92/understanding-html-form-encoding-url-encoded-and-multipart-forms-3lpa
> * https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/POST
> * https://tools.ietf.org/html/rfc2388
> *
> https://community.cloudera.com/t5/Support-Questions/How-to-send-an-http-POST-multipart-form-data-request-with-a/td-p/240035
> *
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (NIFI-8344) Allow TailFile to continue tailing a file for some time after it has been rolled over
[ https://issues.apache.org/jira/browse/NIFI-8344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard resolved NIFI-8344. -- Resolution: Fixed > Allow TailFile to continue tailing a file for some time after it has been > rolled over > - > > Key: NIFI-8344 > URL: https://issues.apache.org/jira/browse/NIFI-8344 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 1.14.0 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > TailFile makes the assumption that once a file has been rolled over, it will > never be appended to. If the file's Last Modified timestamp changes, the > processor assumes that it's a new file and imports the entire contents of the > file again. > However, one practice that I've encountered is that users have a syslog > server that rotates periodically. To rotate, they rename the existing file, > and then restart the server. When that happens, the server will flush out any > data that it has buffered to the file that was just rolled over, and then > begin writing to the new file. > This results in the TailFile processor ingesting the entire file that has > been rolled over. Because we can't keep state about every file that is rolled > over, we should introduce a property that allows the user to indicate that > upon rollover they want to continue tailing that rolled over file until it is > no longer being written to, and then begin tailing the new file. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-8398) Maven 3.8.1 disables support for repositories using "http" protocol
[
https://issues.apache.org/jira/browse/NIFI-8398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17317527#comment-17317527
]
David Handermann commented on NIFI-8398:
MacOS builds on GitHub are now using Maven 3.8.1 and failing as a result, so
resolving this issue is critical.
> Maven 3.8.1 disables support for repositories using "http" protocol
>
>
> Key: NIFI-8398
> URL: https://issues.apache.org/jira/browse/NIFI-8398
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Paul Grey
>Priority: Critical
> Attachments: mvn.http.txt
>
>
> Maven 3.8.1 (released April 2021) has disabled support (by default) for
> "http" repositories. This causes an issue with tips of NiFi project when
> doing a clean build:
> {code:java}
> mvn clean install
> ...
> [INFO] nifi-stateless-system-test-suite ... SKIPPED
> [INFO]
>
> [INFO] BUILD FAILURE
> [INFO]
>
> [INFO] Total time: 08:18 min
> [INFO] Finished at: 2021-04-06T13:20:08-04:00
> [INFO]
>
> [ERROR] Failed to execute goal on project nifi-standard-processors: Could not
> resolve dependencies for project
> org.apache.nifi:nifi-standard-processors:jar:1.14.0-SNAPSHOT: Failed to
> collect dependencies at com.fluenda:ParCEFone:jar:1.2.6 ->
> com.martiansoftware:macnificent:jar:0.2.0: Failed to read artifact descriptor
> for com.martiansoftware:macnificent:jar:0.2.0: Could not transfer artifact
> com.martiansoftware:macnificent:pom:0.2.0 from/to maven-default-http-blocker
> (http://0.0.0.0/): Blocked mirror for repositories: [martiansoftware
> (http://mvn.martiansoftware.com, default, releases+snapshots)] -> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e
> switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR]
> [ERROR] For more information about the errors and possible solutions, please
> read the following articles:
> [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
> [ERROR]
> [ERROR] After correcting the problems, you can resume the build with the
> command
> [ERROR] mvn -rf :nifi-standard-processors
> pgrey@10457 nifi % {code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (NIFI-8398) Maven 3.8.1 disables support for repositories using "http" protocol
[
https://issues.apache.org/jira/browse/NIFI-8398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann updated NIFI-8398:
---
Priority: Critical (was: Minor)
> Maven 3.8.1 disables support for repositories using "http" protocol
>
>
> Key: NIFI-8398
> URL: https://issues.apache.org/jira/browse/NIFI-8398
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Paul Grey
>Priority: Critical
> Attachments: mvn.http.txt
>
>
> Maven 3.8.1 (released April 2021) has disabled support (by default) for
> "http" repositories. This causes an issue with tips of NiFi project when
> doing a clean build:
> {code:java}
> mvn clean install
> ...
> [INFO] nifi-stateless-system-test-suite ... SKIPPED
> [INFO]
>
> [INFO] BUILD FAILURE
> [INFO]
>
> [INFO] Total time: 08:18 min
> [INFO] Finished at: 2021-04-06T13:20:08-04:00
> [INFO]
>
> [ERROR] Failed to execute goal on project nifi-standard-processors: Could not
> resolve dependencies for project
> org.apache.nifi:nifi-standard-processors:jar:1.14.0-SNAPSHOT: Failed to
> collect dependencies at com.fluenda:ParCEFone:jar:1.2.6 ->
> com.martiansoftware:macnificent:jar:0.2.0: Failed to read artifact descriptor
> for com.martiansoftware:macnificent:jar:0.2.0: Could not transfer artifact
> com.martiansoftware:macnificent:pom:0.2.0 from/to maven-default-http-blocker
> (http://0.0.0.0/): Blocked mirror for repositories: [martiansoftware
> (http://mvn.martiansoftware.com, default, releases+snapshots)] -> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e
> switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR]
> [ERROR] For more information about the errors and possible solutions, please
> read the following articles:
> [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
> [ERROR]
> [ERROR] After correcting the problems, you can resume the build with the
> command
> [ERROR] mvn -rf :nifi-standard-processors
> pgrey@10457 nifi % {code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi] jfrazee commented on pull request #4576: NIFI-7886: FetchAzureBlobStorage, FetchS3Object, and FetchGCSObject processors should be able to fetch ranges
jfrazee commented on pull request #4576: URL: https://github.com/apache/nifi/pull/4576#issuecomment-816187172 @pkelly-nifi I tested `FetchGCSObject` using the [`LocalStorageHelper`](https://github.com/googleapis/java-storage-nio/blob/master/google-cloud-nio/src/main/java/com/google/cloud/storage/contrib/nio/testing/LocalStorageHelper.java) from Google's java-storage-nio and all the test cases except one pass. Right now, if you provide a range start that is larger than the content, it routes a zero-length string to `REL_SUCCESS`. This is not the same behavior for the other processors. AFAICT, this isn't a quirk of the `LocalStorageHelper`. [`BlobReadChannel`](https://github.com/googleapis/java-storage/blob/master/google-cloud-storage/src/main/java/com/google/cloud/storage/BlobReadChannel) doesn't seem to care if you seek past the size of the content. So question, do we allow different behavior since the cloud providers are different and this apparently isn't an issue for GCS, or do we enforce some consistency and check against the length in the blob metadata? I think it should be the latter. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (NIFI-8405) Add debug logging around request replication
[ https://issues.apache.org/jira/browse/NIFI-8405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann resolved NIFI-8405. Fix Version/s: 1.14.0 Resolution: Fixed > Add debug logging around request replication > > > Key: NIFI-8405 > URL: https://issues.apache.org/jira/browse/NIFI-8405 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 1.14.0 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > I'll occasionally have a cluster where some requests take a lot longer on one > node than others (a couple of seconds vs. tens of milliseconds). > We should add logging information that indicates timing information around > some of the critical components involved in that path. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-8405) Add debug logging around request replication
[ https://issues.apache.org/jira/browse/NIFI-8405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17317471#comment-17317471 ] ASF subversion and git services commented on NIFI-8405: --- Commit 14e6dc3dc6533fc6189a301cf720c0ee57650729 in nifi's branch refs/heads/main from Mark Payne [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=14e6dc3 ] NIFI-8405: Added debug logging around how long it takes to establish connections/query dns/read and write headers and body when replication requests; added additional timing around Ranger audits and authorizations and monitoring of long-running tasks because those run often and frequently show up in the logs at the same time as the long requests This closes #4983 Signed-off-by: David Handermann > Add debug logging around request replication > > > Key: NIFI-8405 > URL: https://issues.apache.org/jira/browse/NIFI-8405 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Time Spent: 1h > Remaining Estimate: 0h > > I'll occasionally have a cluster where some requests take a lot longer on one > node than others (a couple of seconds vs. tens of milliseconds). > We should add logging information that indicates timing information around > some of the critical components involved in that path. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] asfgit closed pull request #4983: NIFI-8405: Added debug logging around how long it takes to establish …
asfgit closed pull request #4983: URL: https://github.com/apache/nifi/pull/4983 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on pull request #4983: NIFI-8405: Added debug logging around how long it takes to establish …
exceptionfactory commented on pull request #4983: URL: https://github.com/apache/nifi/pull/4983#issuecomment-816167169 Thanks for the responses @markap14, the implementation seems reasonable given the goals described. The unit test update looks good. +1 Merging. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] markap14 commented on a change in pull request #4983: NIFI-8405: Added debug logging around how long it takes to establish …
markap14 commented on a change in pull request #4983:
URL: https://github.com/apache/nifi/pull/4983#discussion_r610014118
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/components/monitor/LongRunningTaskMonitor.java
##
@@ -73,7 +75,8 @@ public void run() {
}
}
-getLogger().info("Active threads: {}; Long running threads: {}",
activeThreadCount, longRunningThreadCount);
+final long nanos = System.nanoTime() - start;
+getLogger().info("Active threads: {}; Long running threads: {}; time
to check: {} nanos", activeThreadCount, longRunningThreadCount,
NumberFormat.getInstance().format(nanos));
Review comment:
D'oh, thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] markap14 commented on a change in pull request #4983: NIFI-8405: Added debug logging around how long it takes to establish …
markap14 commented on a change in pull request #4983:
URL: https://github.com/apache/nifi/pull/4983#discussion_r610012031
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-cluster/src/main/java/org/apache/nifi/cluster/coordination/http/replication/okhttp/OkHttpReplicationClient.java
##
@@ -317,6 +318,7 @@ private OkHttpClient createOkHttpClient(final
NiFiProperties properties) {
okHttpClientBuilder.followRedirects(true);
final int connectionPoolSize =
properties.getClusterNodeMaxConcurrentRequests();
okHttpClientBuilder.connectionPool(new
ConnectionPool(connectionPoolSize, 5, TimeUnit.MINUTES));
+okHttpClientBuilder.eventListener(new
RequestReplicationEventListener());
Review comment:
I took a look at the LoggingEventListener, but it logs each individual
event individually, and since we're always replicating to each node in the
cluster, that becomes really difficult to partition out and understand, either
programmatically or manually.
I did also consider using a mechanism to avoid the expensive of doing this
if logging is not enabled. But since this is done only when we replicate
requests across the cluster, that's not terribly frequently and so the overhead
is almost nil. We could, in the future, improve it, by checking
isDebugEnabled() but I didn't want to cloud the code at this point, given the
(little) expense of the operation and how infrequently it's performed. Plus, I
can envision improving this in the future to do something like always warn if
DNS lookup takes more than 1 second or something like that, even if debug
logging is not enabled.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Assigned] (NIFI-3580) Setting SSL ciphers used by NiFi
[ https://issues.apache.org/jira/browse/NIFI-3580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Grey reassigned NIFI-3580: --- Assignee: Paul Grey > Setting SSL ciphers used by NiFi > > > Key: NIFI-3580 > URL: https://issues.apache.org/jira/browse/NIFI-3580 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI > Environment: Linux >Reporter: Alexander M >Assignee: Paul Grey >Priority: Major > Labels: features, security > > Does NiFi have an ability to choose which OpenSSL ciphers suites use in web > config (web UI https port / cluster interconnect port)? > I know that Jetty have this ability from the box, but I can't find related > parameters in NiFi configuration and documentation. > Can you turn me to the right way if this possible to change default ciphers > suite set or open a feature request for this possibility. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi-minifi-cpp] arpadboda commented on a change in pull request #1042: MINIFICPP-1457 - Implement InputRequirements
arpadboda commented on a change in pull request #1042:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1042#discussion_r609997038
##
File path: extensions/standard-processors/processors/ExecuteProcess.h
##
@@ -51,7 +51,7 @@ namespace processors {
#ifndef WIN32
// ExecuteProcess Class
-class ExecuteProcess : public core::Processor {
+class ExecuteProcess : public core::Processor, public
core::annotation::input::Forbidden {
Review comment:
Yep, makes sense to differentiate the two, but that's definitely out of
scope of this change.
Leave it as "allowed" for now, later we can align the processors to NiFi
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on a change in pull request #4983: NIFI-8405: Added debug logging around how long it takes to establish …
exceptionfactory commented on a change in pull request #4983:
URL: https://github.com/apache/nifi/pull/4983#discussion_r609992388
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/components/monitor/LongRunningTaskMonitor.java
##
@@ -73,7 +75,8 @@ public void run() {
}
}
-getLogger().info("Active threads: {}; Long running threads: {}",
activeThreadCount, longRunningThreadCount);
+final long nanos = System.nanoTime() - start;
+getLogger().info("Active threads: {}; Long running threads: {}; time
to check: {} nanos", activeThreadCount, longRunningThreadCount,
NumberFormat.getInstance().format(nanos));
Review comment:
Unfortunately, changing this logging statement broke the existing unit
test, so some adjustment will be necessary.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on a change in pull request #4983: NIFI-8405: Added debug logging around how long it takes to establish …
exceptionfactory commented on a change in pull request #4983:
URL: https://github.com/apache/nifi/pull/4983#discussion_r609989120
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-cluster/src/main/java/org/apache/nifi/cluster/coordination/http/replication/okhttp/OkHttpReplicationClient.java
##
@@ -317,6 +318,7 @@ private OkHttpClient createOkHttpClient(final
NiFiProperties properties) {
okHttpClientBuilder.followRedirects(true);
final int connectionPoolSize =
properties.getClusterNodeMaxConcurrentRequests();
okHttpClientBuilder.connectionPool(new
ConnectionPool(connectionPoolSize, 5, TimeUnit.MINUTES));
+okHttpClientBuilder.eventListener(new
RequestReplicationEventListener());
Review comment:
Instead of introducing the custom RequestReplicationEventListener and
CallEventListener classes for logging, would the
`okhttp3.logging.LoggingEventListener` class accomplish the same goal?
Introducing the potential for logging events could have performance
implications, what do you think about adding check for
`logger.isDebugEnabled()` to control whether or not to add the Event Listener?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] markap14 opened a new pull request #4983: NIFI-8405: Added debug logging around how long it takes to establish …
markap14 opened a new pull request #4983:
URL: https://github.com/apache/nifi/pull/4983
…connections/query dns/read and write headers and body when replication
requests; added additional timing around Ranger audits and authorizations and
monitoring of long-running tasks because those run often and frequently show up
in the logs at the same time as the long requests
Thank you for submitting a contribution to Apache NiFi.
Please provide a short description of the PR here:
Description of PR
_Enables X functionality; fixes bug NIFI-._
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [ ] Does your PR title start with **NIFI-** where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [ ] Has your PR been rebased against the latest commit within the target
branch (typically `main`)?
- [ ] Is your initial contribution a single, squashed commit? _Additional
commits in response to PR reviewer feedback should be made on this branch and
pushed to allow change tracking. Do not `squash` or use `--force` when pushing
to allow for clean monitoring of changes._
### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via `mvn
-Pcontrib-check clean install` at the root `nifi` folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] Have you verified that the full build is successful on JDK 8?
- [ ] Have you verified that the full build is successful on JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main
`LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main
`NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI for
build issues and submit an update to your PR as soon as possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Assigned] (NIFI-8403) Implement Self-Signed Certificate Generation for HTTPS Configuration
[
https://issues.apache.org/jira/browse/NIFI-8403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joseph Gresock reassigned NIFI-8403:
Assignee: Joseph Gresock
> Implement Self-Signed Certificate Generation for HTTPS Configuration
>
>
> Key: NIFI-8403
> URL: https://issues.apache.org/jira/browse/NIFI-8403
> Project: Apache NiFi
> Issue Type: Sub-task
>Affects Versions: 1.14.0
>Reporter: David Handermann
>Assignee: Joseph Gresock
>Priority: Major
> Labels: https, pkcs12, security
>
> Enabling HTTPS through default configuration properties requires the presence
> of keystore and truststore files. For default standalone installations, this
> requires generating a self-signed certificate and private key for storage in
> a keystore. The certificate should be stored in a truststore and both files
> should be placed in a standard location within the NiFi home directory.
> The following requirements should be considered as part of the implementation:
> * Keystore and Truststore format should be PKCS12
> * Keystore and Truststore passwords should use secure random generation
> * The self-signed certificate must contain at least one DNS Subject
> Alternative Name
> The following implementation questions should be addressed as part of the
> implementation:
> * Should the certificate subject always use {{localhost}} or should other
> host addresses be evaluated and added as subject alternative names?
> * What is the default expiration for the generated certificate? Something
> short should be considered to encourage provisioning a certificate through
> other means
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (NIFI-8405) Add debug logging around request replication
Mark Payne created NIFI-8405: Summary: Add debug logging around request replication Key: NIFI-8405 URL: https://issues.apache.org/jira/browse/NIFI-8405 Project: Apache NiFi Issue Type: Improvement Components: Core Framework Reporter: Mark Payne Assignee: Mark Payne I'll occasionally have a cluster where some requests take a lot longer on one node than others (a couple of seconds vs. tens of milliseconds). We should add logging information that indicates timing information around some of the critical components involved in that path. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] thenatog commented on pull request #4979: NIFI-7468 Correct SSLSocketChannelSender.close() order of operations
thenatog commented on pull request #4979: URL: https://github.com/apache/nifi/pull/4979#issuecomment-815961455 Reviewing this one -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-8333) Issues with invokehttp/SSL Service after "Terminate Task" (e.g. due to long running transactions) and misleading error-message
[ https://issues.apache.org/jira/browse/NIFI-8333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-8333: --- Labels: https security (was: ) > Issues with invokehttp/SSL Service after "Terminate Task" (e.g. due to long > running transactions) and misleading error-message > -- > > Key: NIFI-8333 > URL: https://issues.apache.org/jira/browse/NIFI-8333 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.12.1 >Reporter: Andreas Klingenstein >Priority: Major > Labels: https, security > Attachments: 1_config_invokeHttp_settings.png, > 2_config_invokeHttp_scheduling.png, 3_config_invokeHttp_config_1.png, > 4_config_invokeHttp_config_2.png, 5_config_invokeHttp_config_3.png, > 6_sslcontextservice_settings.png, 7_sslcontextservice_properties.png, > 8_config_invokeHttp_Terminate_Task.png, SSLHandshakeException.txt > > > There seems to be an issue with InvokeHTTP 1.12.1 or > StandardRestrictedSSLContextService 1.12.1 or both. > We observed a lot of "SSLHandshakeException"s (...) "unable to find valid > certification path to requested target" in some situations. > At first we had a very close look into our keystore which is used in our > StandardRestrictedSSLContextService and made sure everything was fine. In the > end we figured out we had used an older, no longer valid oAuth token and the > webservice simlpy had rejected our request. > Then we got the "SSLHandshakeException"s out of the blue in situations of > very intense testing where we started and stopped tests runs over and over > again. The oAuth token was still valid and other processors of the same kind > and even exact copies of the now failing processor worked fine > We discovered the following steps to reproduce the issue in our environment > (Nifi 1.12.1): > - create input data for the InvokeHTTP > - make sure the contacted endpoint does not respond but have a high timeout > setting in InvokeHttp to be able to terminate the processor > - start InvokeHTTP > - let it run for some time > - stop the InvokeHTTP-processor > - "Terminate" should be available in the context menu if there are still > open requests waiting for an answer (or timeout) > - terminate it > - wait until all tasks are finally stopped > - start the InvokeHTTP again > "SSLHandshakeException" Errors should be visible now in the nifi-app.log on > all machines where the invokehttp-processor had to terminate some tasks. > They'll > stay until you restart those nifi instances > Our configuration > - Cluster: 3 server, one nifi instance each > - nifi in "http-mode" on port 8081 (nifi-properties) > more details in the screenshots > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (NIFI-1983) Add flag to disable PostHttp's two phase commit
[ https://issues.apache.org/jira/browse/NIFI-1983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann resolved NIFI-1983. Resolution: Won't Fix NIFI-6018 deprecated PostHTTP in favor of InvokeHTTP. > Add flag to disable PostHttp's two phase commit > --- > > Key: NIFI-1983 > URL: https://issues.apache.org/jira/browse/NIFI-1983 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Edgardo Vega >Priority: Critical > > By default PostHttp interprets the response, and based on a series of > conditions can intentionally issue a delete. This delete action is good some > environments there were timeouts that would occur quite frequently between > PostHTTP / ListenHTTP and this resulted in quite a lot of data duplication. > In environments where this is not an issue or where data duplication is okay > this causes problems with certain load balancers such as AWS ELB. > Having a flag to disable this feature would be useful in getting Nifi to work > better in AWS in general. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-5302) Support for OIDC access token based auth for API
[ https://issues.apache.org/jira/browse/NIFI-5302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-5302: --- Priority: Major (was: Critical) > Support for OIDC access token based auth for API > > > Key: NIFI-5302 > URL: https://issues.apache.org/jira/browse/NIFI-5302 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Federico Michele Facca >Priority: Major > Labels: security > > Currently OIDC is only supported via implicit grant flow. This is perfectly > fine for the UI, but it makes impossible to programmatically interact with > the APIs using OIDC. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-5302) Support for OIDC access token based auth for API
[ https://issues.apache.org/jira/browse/NIFI-5302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-5302: --- Labels: security (was: ) > Support for OIDC access token based auth for API > > > Key: NIFI-5302 > URL: https://issues.apache.org/jira/browse/NIFI-5302 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Federico Michele Facca >Priority: Critical > Labels: security > > Currently OIDC is only supported via implicit grant flow. This is perfectly > fine for the UI, but it makes impossible to programmatically interact with > the APIs using OIDC. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-8404) Support "Rollback on Failure" for PublishKafka(Record) processors
[ https://issues.apache.org/jira/browse/NIFI-8404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17317283#comment-17317283 ] ASF subversion and git services commented on NIFI-8404: --- Commit 78b69f10dcef71ea149bdeb8e97073b7175b8e7b in nifi's branch refs/heads/main from Mark Payne [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=78b69f1 ] NIFI-8404: Added capability to rollback on failure for PublishKafka(Record)_2_0 and _2_6 NIFI-8404: Added unit tests for new Failure Strategy property This closes #4982. Signed-off-by: Peter Turcsanyi > Support "Rollback on Failure" for PublishKafka(Record) processors > - > > Key: NIFI-8404 > URL: https://issues.apache.org/jira/browse/NIFI-8404 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > When the PublishKafka(Record) processors encounter a failure, they route the > FlowFiles to the 'failure' relationship. That's usually what we want. But in > some cases, we want to be able to rollback on failure. This goes hand-in-hand > with the recent feature that was added to allow static assignment of Kafka > Partitions to NiFi nodes. That feature ensures that even in a NiFi cluster, > we won't have data out-of-order (assuming proper configuration) by sending > data from the same partition to multiple NiFi nodes. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-8404) Support "Rollback on Failure" for PublishKafka(Record) processors
[ https://issues.apache.org/jira/browse/NIFI-8404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Turcsanyi updated NIFI-8404: -- Fix Version/s: 1.14.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Support "Rollback on Failure" for PublishKafka(Record) processors > - > > Key: NIFI-8404 > URL: https://issues.apache.org/jira/browse/NIFI-8404 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 1.14.0 > > Time Spent: 20m > Remaining Estimate: 0h > > When the PublishKafka(Record) processors encounter a failure, they route the > FlowFiles to the 'failure' relationship. That's usually what we want. But in > some cases, we want to be able to rollback on failure. This goes hand-in-hand > with the recent feature that was added to allow static assignment of Kafka > Partitions to NiFi nodes. That feature ensures that even in a NiFi cluster, > we won't have data out-of-order (assuming proper configuration) by sending > data from the same partition to multiple NiFi nodes. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-8404) Support "Rollback on Failure" for PublishKafka(Record) processors
[ https://issues.apache.org/jira/browse/NIFI-8404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17317284#comment-17317284 ] ASF subversion and git services commented on NIFI-8404: --- Commit 78b69f10dcef71ea149bdeb8e97073b7175b8e7b in nifi's branch refs/heads/main from Mark Payne [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=78b69f1 ] NIFI-8404: Added capability to rollback on failure for PublishKafka(Record)_2_0 and _2_6 NIFI-8404: Added unit tests for new Failure Strategy property This closes #4982. Signed-off-by: Peter Turcsanyi > Support "Rollback on Failure" for PublishKafka(Record) processors > - > > Key: NIFI-8404 > URL: https://issues.apache.org/jira/browse/NIFI-8404 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > When the PublishKafka(Record) processors encounter a failure, they route the > FlowFiles to the 'failure' relationship. That's usually what we want. But in > some cases, we want to be able to rollback on failure. This goes hand-in-hand > with the recent feature that was added to allow static assignment of Kafka > Partitions to NiFi nodes. That feature ensures that even in a NiFi cluster, > we won't have data out-of-order (assuming proper configuration) by sending > data from the same partition to multiple NiFi nodes. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] asfgit closed pull request #4982: NIFI-8404: Added capability to rollback on failure for PublishKafka(R…
asfgit closed pull request #4982: URL: https://github.com/apache/nifi/pull/4982 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Comment Edited] (NIFI-8398) Maven 3.8.1 disables support for repositories using "http" protocol
[
https://issues.apache.org/jira/browse/NIFI-8398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17317275#comment-17317275
]
Paul Grey edited comment on NIFI-8398 at 4/8/21, 3:33 PM:
--
Did some further research on the maven central part of the issue.
{code:java}
Downloading from maven-central-repo: http://repo1.maven.org/maven2/
{code}
Using the clue of the referenced repository id, I searched my pristine (except
for nifi) maven repository for POM references to project repositories with that
id:
{code:java}
maven-central-repo
http://repo1.maven.org/maven2
...
{code}
I found two:
[https://repo1.maven.org/maven2/at/favre/lib/bytes|https://repo1.maven.org/maven2/at/favre/lib/bytes/1.3.0/bytes-1.3.0.pom]
[/1.3.0/bytes-1.3.0.pom|https://repo1.maven.org/maven2/at/favre/lib/bytes/1.3.0/bytes-1.3.0.pom]
[https://repo1.maven.org/maven2/at/favre/lib/bcrypt-parent/0.9.0/bcrypt-parent-0.9.0.pom]
These are referenced by subproject 'maven-assembly', which is toward the end of
the build.
was (Author: pgrey):
Did some further research on the maven central part of the issue.
Downloading from maven-central-repo:
[http://repo1.maven.org/maven2/]
Using the clue of the referenced repository id, I searched my pristine (except
for nifi) maven repository for POM references to project repositories with that
id:
maven-central-repo
[http://repo1.maven.org/maven2]
...
I found two:
[https://repo1.maven.org/maven2/at/favre/lib/bytes|https://repo1.maven.org/maven2/at/favre/lib/bytes/1.3.0/bytes-1.3.0.pom]
[/1.3.0/bytes-1.3.0.pom|https://repo1.maven.org/maven2/at/favre/lib/bytes/1.3.0/bytes-1.3.0.pom]
[https://repo1.maven.org/maven2/at/favre/lib/bcrypt-parent/0.9.0/bcrypt-parent-0.9.0.pom]
These are referenced by subproject 'maven-assembly', which is toward the end of
the build.
> Maven 3.8.1 disables support for repositories using "http" protocol
>
>
> Key: NIFI-8398
> URL: https://issues.apache.org/jira/browse/NIFI-8398
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Paul Grey
>Priority: Minor
> Attachments: mvn.http.txt
>
>
> Maven 3.8.1 (released April 2021) has disabled support (by default) for
> "http" repositories. This causes an issue with tips of NiFi project when
> doing a clean build:
> {code:java}
> mvn clean install
> ...
> [INFO] nifi-stateless-system-test-suite ... SKIPPED
> [INFO]
>
> [INFO] BUILD FAILURE
> [INFO]
>
> [INFO] Total time: 08:18 min
> [INFO] Finished at: 2021-04-06T13:20:08-04:00
> [INFO]
>
> [ERROR] Failed to execute goal on project nifi-standard-processors: Could not
> resolve dependencies for project
> org.apache.nifi:nifi-standard-processors:jar:1.14.0-SNAPSHOT: Failed to
> collect dependencies at com.fluenda:ParCEFone:jar:1.2.6 ->
> com.martiansoftware:macnificent:jar:0.2.0: Failed to read artifact descriptor
> for com.martiansoftware:macnificent:jar:0.2.0: Could not transfer artifact
> com.martiansoftware:macnificent:pom:0.2.0 from/to maven-default-http-blocker
> (http://0.0.0.0/): Blocked mirror for repositories: [martiansoftware
> (http://mvn.martiansoftware.com, default, releases+snapshots)] -> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e
> switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR]
> [ERROR] For more information about the errors and possible solutions, please
> read the following articles:
> [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
> [ERROR]
> [ERROR] After correcting the problems, you can resume the build with the
> command
> [ERROR] mvn -rf :nifi-standard-processors
> pgrey@10457 nifi % {code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (NIFI-8398) Maven 3.8.1 disables support for repositories using "http" protocol
[
https://issues.apache.org/jira/browse/NIFI-8398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17317275#comment-17317275
]
Paul Grey commented on NIFI-8398:
-
Did some further research on the maven central part of the issue.
Downloading from maven-central-repo:
[http://repo1.maven.org/maven2/]
Using the clue of the referenced repository id, I searched my pristine (except
for nifi) maven repository for POM references to project repositories with that
id:
maven-central-repo
[http://repo1.maven.org/maven2]
...
I found two:
[https://repo1.maven.org/maven2/at/favre/lib/bytes|https://repo1.maven.org/maven2/at/favre/lib/bytes/1.3.0/bytes-1.3.0.pom]
[/1.3.0/bytes-1.3.0.pom|https://repo1.maven.org/maven2/at/favre/lib/bytes/1.3.0/bytes-1.3.0.pom]
[https://repo1.maven.org/maven2/at/favre/lib/bcrypt-parent/0.9.0/bcrypt-parent-0.9.0.pom]
These are referenced by subproject 'maven-assembly', which is toward the end of
the build.
> Maven 3.8.1 disables support for repositories using "http" protocol
>
>
> Key: NIFI-8398
> URL: https://issues.apache.org/jira/browse/NIFI-8398
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Paul Grey
>Priority: Minor
> Attachments: mvn.http.txt
>
>
> Maven 3.8.1 (released April 2021) has disabled support (by default) for
> "http" repositories. This causes an issue with tips of NiFi project when
> doing a clean build:
> {code:java}
> mvn clean install
> ...
> [INFO] nifi-stateless-system-test-suite ... SKIPPED
> [INFO]
>
> [INFO] BUILD FAILURE
> [INFO]
>
> [INFO] Total time: 08:18 min
> [INFO] Finished at: 2021-04-06T13:20:08-04:00
> [INFO]
>
> [ERROR] Failed to execute goal on project nifi-standard-processors: Could not
> resolve dependencies for project
> org.apache.nifi:nifi-standard-processors:jar:1.14.0-SNAPSHOT: Failed to
> collect dependencies at com.fluenda:ParCEFone:jar:1.2.6 ->
> com.martiansoftware:macnificent:jar:0.2.0: Failed to read artifact descriptor
> for com.martiansoftware:macnificent:jar:0.2.0: Could not transfer artifact
> com.martiansoftware:macnificent:pom:0.2.0 from/to maven-default-http-blocker
> (http://0.0.0.0/): Blocked mirror for repositories: [martiansoftware
> (http://mvn.martiansoftware.com, default, releases+snapshots)] -> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e
> switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR]
> [ERROR] For more information about the errors and possible solutions, please
> read the following articles:
> [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
> [ERROR]
> [ERROR] After correcting the problems, you can resume the build with the
> command
> [ERROR] mvn -rf :nifi-standard-processors
> pgrey@10457 nifi % {code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (NIFI-8404) Support "Rollback on Failure" for PublishKafka(Record) processors
[ https://issues.apache.org/jira/browse/NIFI-8404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-8404: - Status: Patch Available (was: Open) > Support "Rollback on Failure" for PublishKafka(Record) processors > - > > Key: NIFI-8404 > URL: https://issues.apache.org/jira/browse/NIFI-8404 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > When the PublishKafka(Record) processors encounter a failure, they route the > FlowFiles to the 'failure' relationship. That's usually what we want. But in > some cases, we want to be able to rollback on failure. This goes hand-in-hand > with the recent feature that was added to allow static assignment of Kafka > Partitions to NiFi nodes. That feature ensures that even in a NiFi cluster, > we won't have data out-of-order (assuming proper configuration) by sending > data from the same partition to multiple NiFi nodes. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] markap14 opened a new pull request #4982: NIFI-8404: Added capability to rollback on failure for PublishKafka(R…
markap14 opened a new pull request #4982:
URL: https://github.com/apache/nifi/pull/4982
…ecord)_2_0 and _2_6
Thank you for submitting a contribution to Apache NiFi.
Please provide a short description of the PR here:
Description of PR
_Enables X functionality; fixes bug NIFI-._
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [ ] Does your PR title start with **NIFI-** where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [ ] Has your PR been rebased against the latest commit within the target
branch (typically `main`)?
- [ ] Is your initial contribution a single, squashed commit? _Additional
commits in response to PR reviewer feedback should be made on this branch and
pushed to allow change tracking. Do not `squash` or use `--force` when pushing
to allow for clean monitoring of changes._
### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via `mvn
-Pcontrib-check clean install` at the root `nifi` folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] Have you verified that the full build is successful on JDK 8?
- [ ] Have you verified that the full build is successful on JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main
`LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main
`NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI for
build issues and submit an update to your PR as soon as possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Created] (NIFI-8404) Support "Rollback on Failure" for PublishKafka(Record) processors
Mark Payne created NIFI-8404: Summary: Support "Rollback on Failure" for PublishKafka(Record) processors Key: NIFI-8404 URL: https://issues.apache.org/jira/browse/NIFI-8404 Project: Apache NiFi Issue Type: Improvement Components: Extensions Reporter: Mark Payne Assignee: Mark Payne When the PublishKafka(Record) processors encounter a failure, they route the FlowFiles to the 'failure' relationship. That's usually what we want. But in some cases, we want to be able to rollback on failure. This goes hand-in-hand with the recent feature that was added to allow static assignment of Kafka Partitions to NiFi nodes. That feature ensures that even in a NiFi cluster, we won't have data out-of-order (assuming proper configuration) by sending data from the same partition to multiple NiFi nodes. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] jfrazee commented on pull request #4576: NIFI-7886: FetchAzureBlobStorage, FetchS3Object, and FetchGCSObject processors should be able to fetch ranges
jfrazee commented on pull request #4576: URL: https://github.com/apache/nifi/pull/4576#issuecomment-815867389 @pvillard31 @pkelly-nifi The ITs are a nice to have but we can also handle those separately. The testing on GCS is more important. I'll look into that again today. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] joewitt commented on pull request #4946: NIFI-8295: upgrade cassandra driver to 3.11.0
joewitt commented on pull request #4946: URL: https://github.com/apache/nifi/pull/4946#issuecomment-815864831 I dont see any changes to the LICENSE and NOTICE files. Does that mean that from 3.3 to 3.11 there have been no changes at all to the depenedencies of this lib? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (NIFI-7134) Enable JettyServer to automatically detect keystore changes and update
[ https://issues.apache.org/jira/browse/NIFI-7134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Gresock reassigned NIFI-7134: Assignee: Joseph Gresock > Enable JettyServer to automatically detect keystore changes and update > -- > > Key: NIFI-7134 > URL: https://issues.apache.org/jira/browse/NIFI-7134 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework, Security >Affects Versions: 1.11.1 >Reporter: patrick white >Assignee: Joseph Gresock >Priority: Minor > Labels: jetty, keystore, restart, security, tls > > TLS/keystore credential change currently requires a service restart to > update, [~alopresto] noted on 'users' that Jetty 9.3+ supports the ability to > dynamically update credentials, and provided reference [1]. > Request enabling NiFi JettyServer to support detection and reload of its > keystore when it changes, such as during credentials update or rotation, will > link this request to epic [2]. > [1] https://github.com/eclipse/jetty.project/issues/918 > [2] https://issues.apache.org/jira/browse/NIFI-5458 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (NIFI-8403) Implement Self-Signed Certificate Generation for HTTPS Configuration
David Handermann created NIFI-8403:
--
Summary: Implement Self-Signed Certificate Generation for HTTPS
Configuration
Key: NIFI-8403
URL: https://issues.apache.org/jira/browse/NIFI-8403
Project: Apache NiFi
Issue Type: Sub-task
Affects Versions: 1.14.0
Reporter: David Handermann
Enabling HTTPS through default configuration properties requires the presence
of keystore and truststore files. For default standalone installations, this
requires generating a self-signed certificate and private key for storage in a
keystore. The certificate should be stored in a truststore and both files
should be placed in a standard location within the NiFi home directory.
The following requirements should be considered as part of the implementation:
* Keystore and Truststore format should be PKCS12
* Keystore and Truststore passwords should use secure random generation
* The self-signed certificate must contain at least one DNS Subject
Alternative Name
The following implementation questions should be addressed as part of the
implementation:
* Should the certificate subject always use {{localhost}} or should other host
addresses be evaluated and added as subject alternative names?
* What is the default expiration for the generated certificate? Something
short should be considered to encourage provisioning a certificate through
other means
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi-minifi-cpp] martinzink commented on a change in pull request #1032: MINIFICPP-1504: Add Resource consumption data to heartbeats
martinzink commented on a change in pull request #1032:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1032#discussion_r609731373
##
File path: libminifi/src/utils/SystemCPUUsageTracker.cpp
##
@@ -0,0 +1,190 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "utils/SystemCPUUsageTracker.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace utils {
+#ifdef __linux__
+
+SystemCPUUsageTracker::SystemCPUUsageTracker() :
+total_user_(0), previous_total_user_(0),
+total_user_low_(0), previous_total_user_low_(0),
+total_sys_(0), previous_total_sys_(0),
+total_idle_(0), previous_total_idle_(0) {
+ queryCPUTimes();
+}
+
+double SystemCPUUsageTracker::getCPUUsageAndRestartCollection() {
+ queryCPUTimes();
+ if (isCurrentQuerySameAsPrevious() || isCurrentQuerySameAsPrevious()) {
Review comment:
Oops, you are right. Fixed it in
[d2832cf](https://github.com/apache/nifi-minifi-cpp/pull/1032/commits/d2832cfcb3afa837e30e7bc95f496f0754d7603f)
##
File path: libminifi/include/core/state/nodes/AgentInformation.h
##
@@ -425,77 +427,123 @@ class AgentStatus : public StateMonitorNode {
std::vector serialize() {
std::vector serialized;
+auto serializedRepositories = serializeRepositories();
+if (!serializedRepositories.empty()) {
+ serialized.push_back(serializedRepositories);
+}
+serialized.push_back(serializeUptime());
-SerializedResponseNode uptime;
-
-uptime.name = "uptime";
-if (nullptr != monitor_) {
- uptime.value = monitor_->getUptime();
-} else {
- uptime.value = "0";
+auto serializedComponents = serializeComponents();
+if (!serializedComponents.empty()) {
+ serialized.push_back(serializedComponents);
}
-if (!repositories_.empty()) {
- SerializedResponseNode repositories;
+serialized.push_back(serializeResourceConsumption());
- repositories.name = "repositories";
+return serialized;
+ }
+
+ protected:
+ SerializedResponseNode serializeRepositories() {
+SerializedResponseNode repositories;
- for (auto : repositories_) {
-SerializedResponseNode repoNode;
-repoNode.collapsible = false;
-repoNode.name = repo.first;
+repositories.name = "repositories";
-SerializedResponseNode queuesize;
-queuesize.name = "size";
-queuesize.value = repo.second->getRepoSize();
+for (auto : repositories_) {
+ SerializedResponseNode repoNode;
+ repoNode.collapsible = false;
+ repoNode.name = repo.first;
-SerializedResponseNode isRunning;
-isRunning.name = "running";
-isRunning.value = repo.second->isRunning();
+ SerializedResponseNode queuesize;
+ queuesize.name = "size";
+ queuesize.value = repo.second->getRepoSize();
-SerializedResponseNode isFull;
-isFull.name = "full";
-isFull.value = repo.second->isFull();
+ SerializedResponseNode isRunning;
+ isRunning.name = "running";
+ isRunning.value = repo.second->isRunning();
-repoNode.children.push_back(queuesize);
-repoNode.children.push_back(isRunning);
-repoNode.children.push_back(isFull);
-repositories.children.push_back(repoNode);
- }
- serialized.push_back(repositories);
+ SerializedResponseNode isFull;
+ isFull.name = "full";
+ isFull.value = repo.second->isFull();
+
+ repoNode.children.push_back(queuesize);
+ repoNode.children.push_back(isRunning);
+ repoNode.children.push_back(isFull);
+ repositories.children.push_back(repoNode);
}
+return repositories;
+ }
-serialized.push_back(uptime);
+ SerializedResponseNode serializeUptime() {
+SerializedResponseNode uptime;
+uptime.name = "uptime";
if (nullptr != monitor_) {
+ uptime.value = monitor_->getUptime();
+} else {
+ uptime.value = "0";
+}
+
+return uptime;
+ }
+
+ SerializedResponseNode serializeComponents() {
+SerializedResponseNode components_node(false);
+components_node.name = "components";
+if (monitor_ != nullptr) {
auto components =
[GitHub] [nifi-minifi-cpp] adam-markovics commented on a change in pull request #1042: MINIFICPP-1457 - Implement InputRequirements
adam-markovics commented on a change in pull request #1042:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1042#discussion_r609730581
##
File path: extensions/standard-processors/processors/ExecuteProcess.h
##
@@ -51,7 +51,7 @@ namespace processors {
#ifndef WIN32
// ExecuteProcess Class
-class ExecuteProcess : public core::Processor {
+class ExecuteProcess : public core::Processor, public
core::annotation::input::Forbidden {
Review comment:
ExecuteStreamCommand is the one in NiFi that accepts commands from an
incoming flowfile. We don't have that (yet?).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez opened a new pull request #1047: MINIFICPP-1503 Build Tensorflow extension in the CI
lordgamez opened a new pull request #1047: URL: https://github.com/apache/nifi-minifi-cpp/pull/1047 - Fix warnings in Tensorflow extension - Pull Tensorflow dependencies from docker image in CI - Build Tensorflow extension in ubuntu 20.04 job Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFICPP- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically main)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check GitHub Actions CI results for build issues and submit an update to your PR as soon as possible. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] pkelly-nifi commented on pull request #4576: NIFI-7886: FetchAzureBlobStorage, FetchS3Object, and FetchGCSObject processors should be able to fetch ranges
pkelly-nifi commented on pull request #4576: URL: https://github.com/apache/nifi/pull/4576#issuecomment-815841071 > @jfrazee - are there still requested changes on this pull request? I haven't had a chance yet to add any tests, so we're still waiting on that. If someone else wants to kick in some tests that'd be much appreciated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Comment Edited] (NIFI-1467) Provide Bcrypt salt generation code
[
https://issues.apache.org/jira/browse/NIFI-1467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17317179#comment-17317179
]
David Handermann edited comment on NIFI-1467 at 4/8/21, 1:07 PM:
-
Resolved with methods added to BcryptCipherProvider as part of work under
NIFI-7122 and NIFI-7268.
was (Author: exceptionfactory):
Resolved with methods added to BcryptCipherProvider as part of work under
NIFI-7122 and NIFI-7638.
> Provide Bcrypt salt generation code
> ---
>
> Key: NIFI-1467
> URL: https://issues.apache.org/jira/browse/NIFI-1467
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
>Affects Versions: 0.5.0
>Reporter: Andy LoPresto
>Assignee: Andy LoPresto
>Priority: Minor
> Labels: encryption, security
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> Currently, the {{ScryptCipherProvider}} can accept both formatted salt in the
> mcrypt format {{$s0$e0101$ABCDEFGHIJKLMNOPQRSTUV}} or raw salt {{0x01 23 45
> 67 89 AB CD EF FE DC BA 98 76 54 32 10}} format and combine that with the
> instance parameters {{N}}, {{r}}, and {{p}} to return a complete salt. At the
> same time, due to inconsistency in the {{Base64}} formatting,
> {{BcryptCipherProvider}} can only accept fully formatted salts, and cannot
> generate a complete salt from raw input.
> Use the custom {{Base64}} encoding as provided in {{BCrypt.java}} to resolve
> this issue.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (NIFI-1467) Provide Bcrypt salt generation code
[
https://issues.apache.org/jira/browse/NIFI-1467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann resolved NIFI-1467.
Resolution: Fixed
Resolved with methods added to BcryptCipherProvider as part of work under
NIFI-7122 and NIFI-7638.
> Provide Bcrypt salt generation code
> ---
>
> Key: NIFI-1467
> URL: https://issues.apache.org/jira/browse/NIFI-1467
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
>Affects Versions: 0.5.0
>Reporter: Andy LoPresto
>Assignee: Andy LoPresto
>Priority: Minor
> Labels: encryption, security
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> Currently, the {{ScryptCipherProvider}} can accept both formatted salt in the
> mcrypt format {{$s0$e0101$ABCDEFGHIJKLMNOPQRSTUV}} or raw salt {{0x01 23 45
> 67 89 AB CD EF FE DC BA 98 76 54 32 10}} format and combine that with the
> instance parameters {{N}}, {{r}}, and {{p}} to return a complete salt. At the
> same time, due to inconsistency in the {{Base64}} formatting,
> {{BcryptCipherProvider}} can only accept fully formatted salts, and cannot
> generate a complete salt from raw input.
> Use the custom {{Base64}} encoding as provided in {{BCrypt.java}} to resolve
> this issue.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (NIFI-8401) o.apache.nifi.controller.FlowController Failed to capture component stats for Stats History on Windows Systems
[ https://issues.apache.org/jira/browse/NIFI-8401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17317172#comment-17317172 ] Eric Detwiler commented on NIFI-8401: - Thanks Nadeem. > o.apache.nifi.controller.FlowController Failed to capture component stats for > Stats History on Windows Systems > -- > > Key: NIFI-8401 > URL: https://issues.apache.org/jira/browse/NIFI-8401 > Project: Apache NiFi > Issue Type: Bug > Components: Configuration >Affects Versions: 1.13.2 > Environment: Windows 10 > Windows Server 2019 >Reporter: Eric Detwiler >Priority: Major > > 1.13.2 on Windows systems does not capture stats for Stats History by > default. Error log below: > Also located issue mentioned in post from March 2021. > https://community.cloudera.com/t5/Support-Questions/ERROR-FlowController-Failed-to-capture-component-stats-for/td-p/312486 > Status History as a result does not function. > > 2021-04-02 07:56:00,013 INFO [pool-20-thread-1] > o.a.n.c.r.WriteAheadFlowFileRepository Initiating checkpoint of FlowFile > Repository > 2021-04-02 07:56:00,014 INFO [pool-20-thread-1] > o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile > Repository with 0 records in 0 milliseconds > 2021-04-02 07:56:17,311 ERROR [Timer-Driven Process Thread-7] > o.apache.nifi.controller.FlowController Failed to capture component stats for > Stats History > java.lang.NullPointerException: null > at > org.apache.nifi.diagnostics.SystemDiagnostics.getOpenFileHandles(SystemDiagnostics.java:224) > at > org.apache.nifi.controller.FlowController.getNodeStatusSnapshot(FlowController.java:3011) > at > org.apache.nifi.controller.FlowController.access$400(FlowController.java:230) > at org.apache.nifi.controller.FlowController$2.run(FlowController.java:697) > at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi-minifi-cpp] arpadboda commented on a change in pull request #1042: MINIFICPP-1457 - Implement InputRequirements
arpadboda commented on a change in pull request #1042: URL: https://github.com/apache/nifi-minifi-cpp/pull/1042#discussion_r609664576 ## File path: .github/workflows/ci.yml ## @@ -218,7 +218,7 @@ jobs: sudo ln -s /usr/lib/x86_64-linux-gnu/odbc/libsqlite3odbc.so /usr/lib/x86_64-linux-gnu/libsqlite3odbc.so echo "PATH=/usr/lib/ccache:$PATH" >> $GITHUB_ENV - id: build -run: sudo mount tmpfs -t tmpfs /tmp && ./bootstrap.sh -e -t && cd build && cmake -DUSE_SHARED_LIBS= -DENABLE_OPENWSMAN=ON -DENABLE_OPENCV=ON -DENABLE_MQTT=ON -DENABLE_GPS=ON -DENABLE_USB_CAMERA=ON -DENABLE_LIBRDKAFKA=ON -DENABLE_OPC=ON -DENABLE_SFTP=ON -DENABLE_MQTT=ON -DENABLE_COAP=ON -DENABLE_PYTHON=ON -DENABLE_SQL=ON -DENABLE_AWS=ON -DENABLE_AZURE=ON -DSTRICT_GSL_CHECKS=AUDIT -DFAIL_ON_WARNINGS=ON .. && cmake --build . --parallel 4 && make test ARGS="--timeout 300 -j8 --output-on-failure" Review comment: Okay, makes sense :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] adam-markovics commented on a change in pull request #1042: MINIFICPP-1457 - Implement InputRequirements
adam-markovics commented on a change in pull request #1042:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1042#discussion_r609663815
##
File path: extensions/standard-processors/processors/ExecuteProcess.h
##
@@ -51,7 +51,7 @@ namespace processors {
#ifndef WIN32
// ExecuteProcess Class
-class ExecuteProcess : public core::Processor {
+class ExecuteProcess : public core::Processor, public
core::annotation::input::Forbidden {
Review comment:
Input is also forbidden in NiFi, I will ask them what they think about
this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #1042: MINIFICPP-1457 - Implement InputRequirements
adamdebreceni commented on a change in pull request #1042:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1042#discussion_r609658525
##
File path: extensions/standard-processors/processors/ExecuteProcess.h
##
@@ -51,7 +51,7 @@ namespace processors {
#ifndef WIN32
// ExecuteProcess Class
-class ExecuteProcess : public core::Processor {
+class ExecuteProcess : public core::Processor, public
core::annotation::input::Forbidden {
Review comment:
it can be specified but it seems to me, that it won't be used, this
might be a bug
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] adam-markovics commented on a change in pull request #1042: MINIFICPP-1457 - Implement InputRequirements
adam-markovics commented on a change in pull request #1042: URL: https://github.com/apache/nifi-minifi-cpp/pull/1042#discussion_r609605401 ## File path: .github/workflows/ci.yml ## @@ -218,7 +218,7 @@ jobs: sudo ln -s /usr/lib/x86_64-linux-gnu/odbc/libsqlite3odbc.so /usr/lib/x86_64-linux-gnu/libsqlite3odbc.so echo "PATH=/usr/lib/ccache:$PATH" >> $GITHUB_ENV - id: build -run: sudo mount tmpfs -t tmpfs /tmp && ./bootstrap.sh -e -t && cd build && cmake -DUSE_SHARED_LIBS= -DENABLE_OPENWSMAN=ON -DENABLE_OPENCV=ON -DENABLE_MQTT=ON -DENABLE_GPS=ON -DENABLE_USB_CAMERA=ON -DENABLE_LIBRDKAFKA=ON -DENABLE_OPC=ON -DENABLE_SFTP=ON -DENABLE_MQTT=ON -DENABLE_COAP=ON -DENABLE_PYTHON=ON -DENABLE_SQL=ON -DENABLE_AWS=ON -DENABLE_AZURE=ON -DSTRICT_GSL_CHECKS=AUDIT -DFAIL_ON_WARNINGS=ON .. && cmake --build . --parallel 4 && make test ARGS="--timeout 300 -j8 --output-on-failure" Review comment: Because it was duplicated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] arpadboda commented on a change in pull request #1043: MINIFICPP-1359 Implement FollowRedirects property in InvokeHTTP
arpadboda commented on a change in pull request #1043:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1043#discussion_r609597141
##
File path: extensions/http-curl/processors/InvokeHTTP.cpp
##
@@ -75,7 +75,11 @@ core::Property InvokeHTTP::ReadTimeout(
core::Property InvokeHTTP::DateHeader(
core::PropertyBuilder::createProperty("Include Date
Header")->withDescription("Include an RFC-2616 Date header in the
request.")->isRequired(false)->withDefaultValue(true)->build());
-core::Property InvokeHTTP::FollowRedirects("Follow Redirects", "Follow HTTP
redirects issued by remote server.", "True");
+core::Property InvokeHTTP::FollowRedirects(
+ core::PropertyBuilder::createProperty("Follow Redirects")
+ ->withDescription("Follow HTTP redirects issued by remote server.")
+ ->withDefaultValue(true)
Review comment:
Hmm, good question.
My reasoning was not based on Curl's default, but what we did before (which
was based on Curl's default).
As minifi is still 0.xxx version, I tend to agree with you and introduce the
breaking change to align with NiFi in this aspect.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] arpadboda commented on a change in pull request #1042: MINIFICPP-1457 - Implement InputRequirements
arpadboda commented on a change in pull request #1042:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1042#discussion_r609592749
##
File path: extensions/standard-processors/processors/ExecuteProcess.h
##
@@ -51,7 +51,7 @@ namespace processors {
#ifndef WIN32
// ExecuteProcess Class
-class ExecuteProcess : public core::Processor {
+class ExecuteProcess : public core::Processor, public
core::annotation::input::Forbidden {
Review comment:
I think this should allow - the process to be executed can be speficied
in flowfile attributes and whatnot
##
File path: .github/workflows/ci.yml
##
@@ -218,7 +218,7 @@ jobs:
sudo ln -s /usr/lib/x86_64-linux-gnu/odbc/libsqlite3odbc.so
/usr/lib/x86_64-linux-gnu/libsqlite3odbc.so
echo "PATH=/usr/lib/ccache:$PATH" >> $GITHUB_ENV
- id: build
-run: sudo mount tmpfs -t tmpfs /tmp && ./bootstrap.sh -e -t && cd
build && cmake -DUSE_SHARED_LIBS= -DENABLE_OPENWSMAN=ON -DENABLE_OPENCV=ON
-DENABLE_MQTT=ON -DENABLE_GPS=ON -DENABLE_USB_CAMERA=ON -DENABLE_LIBRDKAFKA=ON
-DENABLE_OPC=ON -DENABLE_SFTP=ON -DENABLE_MQTT=ON -DENABLE_COAP=ON
-DENABLE_PYTHON=ON -DENABLE_SQL=ON -DENABLE_AWS=ON -DENABLE_AZURE=ON
-DSTRICT_GSL_CHECKS=AUDIT -DFAIL_ON_WARNINGS=ON .. && cmake --build .
--parallel 4 && make test ARGS="--timeout 300 -j8 --output-on-failure"
Review comment:
Why remove MQTT?
Even if we don't test it, at least the build doesn't get broken.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a change in pull request #1043: MINIFICPP-1359 Implement FollowRedirects property in InvokeHTTP
lordgamez commented on a change in pull request #1043:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1043#discussion_r609593406
##
File path: extensions/http-curl/processors/InvokeHTTP.cpp
##
@@ -75,7 +75,11 @@ core::Property InvokeHTTP::ReadTimeout(
core::Property InvokeHTTP::DateHeader(
core::PropertyBuilder::createProperty("Include Date
Header")->withDescription("Include an RFC-2616 Date header in the
request.")->isRequired(false)->withDefaultValue(true)->build());
-core::Property InvokeHTTP::FollowRedirects("Follow Redirects", "Follow HTTP
redirects issued by remote server.", "True");
+core::Property InvokeHTTP::FollowRedirects(
+ core::PropertyBuilder::createProperty("Follow Redirects")
+ ->withDescription("Follow HTTP redirects issued by remote server.")
+ ->withDefaultValue(true)
Review comment:
That's true, but I was checking the usage of this property in NiFi's
version of
[InvokeHTTP](https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.13.2/org.apache.nifi.processors.standard.InvokeHTTP/index.html)
and there the default value is True. Should we stil prefer the Curl's version
for the default value?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a change in pull request #1031: MINIFICPP-1476 Improve parallel S3 request handling in S3 processors
lordgamez commented on a change in pull request #1031:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1031#discussion_r609591007
##
File path: extensions/aws/processors/DeleteS3Object.h
##
@@ -65,6 +65,11 @@ class DeleteS3Object : public S3Processor {
explicit DeleteS3Object(const std::string& name, const
minifi::utils::Identifier& uuid, std::unique_ptr
s3_request_sender)
: S3Processor(name, uuid,
logging::LoggerFactory::getLogger(),
std::move(s3_request_sender)) {
}
+
+ minifi::utils::optional
buildDeleteS3RequestParams(
+const std::shared_ptr ,
+const std::shared_ptr _file,
+const CommonProperties _properties);
Review comment:
Updated in 690848de02baef8826231bf93c75c981bc7df768
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a change in pull request #1031: MINIFICPP-1476 Improve parallel S3 request handling in S3 processors
lordgamez commented on a change in pull request #1031:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1031#discussion_r609590787
##
File path: extensions/aws/processors/PutS3Object.cpp
##
@@ -206,7 +206,8 @@
minifi::utils::optional PutS3Object::buildP
const std::shared_ptr ,
const std::shared_ptr _file,
const CommonProperties _properties) {
- aws::s3::PutObjectRequestParameters params;
+ aws::s3::PutObjectRequestParameters params(common_properties.credentials,
client_config_);
+ setClientConfig(params.client_config, common_properties);
Review comment:
Thanks, for the suggestion, I like the idea, added it in
21f8e90a07c8d47da881fcae5abb4d74e5f3657a
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] arpadboda commented on a change in pull request #1043: MINIFICPP-1359 Implement FollowRedirects property in InvokeHTTP
arpadboda commented on a change in pull request #1043:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1043#discussion_r609571094
##
File path: extensions/http-curl/processors/InvokeHTTP.cpp
##
@@ -75,7 +75,11 @@ core::Property InvokeHTTP::ReadTimeout(
core::Property InvokeHTTP::DateHeader(
core::PropertyBuilder::createProperty("Include Date
Header")->withDescription("Include an RFC-2616 Date header in the
request.")->isRequired(false)->withDefaultValue(true)->build());
-core::Property InvokeHTTP::FollowRedirects("Follow Redirects", "Follow HTTP
redirects issued by remote server.", "True");
+core::Property InvokeHTTP::FollowRedirects(
+ core::PropertyBuilder::createProperty("Follow Redirects")
+ ->withDescription("Follow HTTP redirects issued by remote server.")
+ ->withDefaultValue(true)
Review comment:
According to Curl, this is false by default:
https://curl.se/libcurl/c/CURLOPT_FOLLOWLOCATION.html
So I think this is a breaking change, I would prefer to have false here by
default.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] arpadboda commented on a change in pull request #1032: MINIFICPP-1504: Add Resource consumption data to heartbeats
arpadboda commented on a change in pull request #1032:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1032#discussion_r609510112
##
File path: libminifi/include/core/state/nodes/AgentInformation.h
##
@@ -425,77 +427,123 @@ class AgentStatus : public StateMonitorNode {
std::vector serialize() {
std::vector serialized;
+auto serializedRepositories = serializeRepositories();
+if (!serializedRepositories.empty()) {
+ serialized.push_back(serializedRepositories);
+}
+serialized.push_back(serializeUptime());
-SerializedResponseNode uptime;
-
-uptime.name = "uptime";
-if (nullptr != monitor_) {
- uptime.value = monitor_->getUptime();
-} else {
- uptime.value = "0";
+auto serializedComponents = serializeComponents();
+if (!serializedComponents.empty()) {
+ serialized.push_back(serializedComponents);
}
-if (!repositories_.empty()) {
- SerializedResponseNode repositories;
+serialized.push_back(serializeResourceConsumption());
- repositories.name = "repositories";
+return serialized;
+ }
+
+ protected:
+ SerializedResponseNode serializeRepositories() {
+SerializedResponseNode repositories;
- for (auto : repositories_) {
-SerializedResponseNode repoNode;
-repoNode.collapsible = false;
-repoNode.name = repo.first;
+repositories.name = "repositories";
-SerializedResponseNode queuesize;
-queuesize.name = "size";
-queuesize.value = repo.second->getRepoSize();
+for (auto : repositories_) {
+ SerializedResponseNode repoNode;
+ repoNode.collapsible = false;
+ repoNode.name = repo.first;
-SerializedResponseNode isRunning;
-isRunning.name = "running";
-isRunning.value = repo.second->isRunning();
+ SerializedResponseNode queuesize;
+ queuesize.name = "size";
+ queuesize.value = repo.second->getRepoSize();
-SerializedResponseNode isFull;
-isFull.name = "full";
-isFull.value = repo.second->isFull();
+ SerializedResponseNode isRunning;
+ isRunning.name = "running";
+ isRunning.value = repo.second->isRunning();
-repoNode.children.push_back(queuesize);
-repoNode.children.push_back(isRunning);
-repoNode.children.push_back(isFull);
-repositories.children.push_back(repoNode);
- }
- serialized.push_back(repositories);
+ SerializedResponseNode isFull;
+ isFull.name = "full";
+ isFull.value = repo.second->isFull();
+
+ repoNode.children.push_back(queuesize);
+ repoNode.children.push_back(isRunning);
+ repoNode.children.push_back(isFull);
+ repositories.children.push_back(repoNode);
}
+return repositories;
+ }
-serialized.push_back(uptime);
+ SerializedResponseNode serializeUptime() {
+SerializedResponseNode uptime;
+uptime.name = "uptime";
if (nullptr != monitor_) {
+ uptime.value = monitor_->getUptime();
+} else {
+ uptime.value = "0";
+}
+
+return uptime;
+ }
+
+ SerializedResponseNode serializeComponents() {
+SerializedResponseNode components_node(false);
+components_node.name = "components";
+if (monitor_ != nullptr) {
auto components = monitor_->getAllComponents();
- SerializedResponseNode componentsNode(false);
- componentsNode.name = "components";
for (auto component : components) {
-SerializedResponseNode componentNode(false);
-componentNode.name = component->getComponentName();
+SerializedResponseNode component_node(false);
+component_node.name = component->getComponentName();
-SerializedResponseNode uuidNode;
-uuidNode.name = "uuid";
-uuidNode.value =
std::string{component->getComponentUUID().to_string()};
+SerializedResponseNode uuid_node;
+uuid_node.name = "uuid";
+uuid_node.value =
std::string{component->getComponentUUID().to_string()};
-SerializedResponseNode componentStatusNode;
-componentStatusNode.name = "running";
-componentStatusNode.value = component->isRunning();
+SerializedResponseNode component_status_node;
+component_status_node.name = "running";
+component_status_node.value = component->isRunning();
-componentNode.children.push_back(componentStatusNode);
-componentNode.children.push_back(uuidNode);
-componentsNode.children.push_back(componentNode);
+component_node.children.push_back(component_status_node);
+component_node.children.push_back(uuid_node);
+components_node.children.push_back(component_node);
}
- serialized.push_back(componentsNode);
}
+return components_node;
+ }
+
+ SerializedResponseNode serializeAgentMemoryUsage() {
Review comment:
This func could be const
##
File path:
[jira] [Commented] (NIFI-4390) Add a keyboard shortcut for Connection related dialogs
[ https://issues.apache.org/jira/browse/NIFI-4390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17317015#comment-17317015 ] Laurent Edel commented on NIFI-4390: digging up the topic, from a developper perspective it would be a huge improvment to be able to use keyboard shortcuts. > Add a keyboard shortcut for Connection related dialogs > -- > > Key: NIFI-4390 > URL: https://issues.apache.org/jira/browse/NIFI-4390 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Reporter: Yuri >Priority: Minor > Labels: dialogs, shortcuts, ui, ux > Attachments: nifi_dialogs_v1.ods > > > Current dialogs don't allow to bound a keyboard shortcut to an action. This > hinders the UX, since there are many dialogs involved in the most common > interactions. > For instance, adding a new connection with a single relationship still > requires a click at the confirm button. Instead, it should be possible to > confirm the dialog simply by hitting the Enter key. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] pvillard31 commented on pull request #4576: NIFI-7886: FetchAzureBlobStorage, FetchS3Object, and FetchGCSObject processors should be able to fetch ranges
pvillard31 commented on pull request #4576: URL: https://github.com/apache/nifi/pull/4576#issuecomment-815597928 @jfrazee - are there still requested changes on this pull request? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-8402) refresh after runOnce execution
Laurent Edel created NIFI-8402: -- Summary: refresh after runOnce execution Key: NIFI-8402 URL: https://issues.apache.org/jira/browse/NIFI-8402 Project: Apache NiFi Issue Type: Improvement Components: Core Framework Reporter: Laurent Edel NIFI-8188 brings the functionality of generating a single FF which is fantastic from a development or demo perspective. While doing that, the human interface between the chair and the keyboard is probably waiting to see that FF, so it need a right click/refresh on the canvas. Is it possible to chain a refresh just after a runOnce? No strong opinion about if that should be a fixed delay or after the FF has really been emitted -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (MINIFICPP-987) Error handling in GetFile has multiple issues
[
https://issues.apache.org/jira/browse/MINIFICPP-987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ferenc Gerlits reassigned MINIFICPP-987:
Assignee: Ferenc Gerlits (was: Arpad Boda)
> Error handling in GetFile has multiple issues
> -
>
> Key: MINIFICPP-987
> URL: https://issues.apache.org/jira/browse/MINIFICPP-987
> Project: Apache NiFi MiNiFi C++
> Issue Type: Bug
>Affects Versions: 0.6.0
>Reporter: Arpad Boda
>Assignee: Ferenc Gerlits
>Priority: Major
>
> In case GetFile is set to delete the transferred files, the following check
> is made:
> {code:java}
> access(fullName.c_str(), W_OK) != 0{code}
> This doesn't really help as the containing folder should be checked on Unix
> file systems. Not to mention ACL... On Windows, there is no reliable way to
> predict the result of the delete operation.
> In session::import, we try to delete the file, but the return value of the
> function is ignored:
> {code:java}
> if (!keepSource)
> std::remove(source.c_str());{code}
> Which means that in case of failure at delete, the file is going to be
> transmitted endlessly in every ontrigger call.
> Although adding error handling here (throwing an exception) wouldn't really
> help as exceptions are handled per execution, not per file. Which means any
> error (a file couldn't be read for eg.) in the given directory would prevent
> the processor to transmit the rest.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (MINIFICPP-1538) Investigate per process group column families
[ https://issues.apache.org/jira/browse/MINIFICPP-1538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Debreceni updated MINIFICPP-1538: -- Description: Currently all operations in FlowFileRepository go through a single column family (the default). We should investigate the performance benefit, if any, of using different column families for different process groups (special care must be taken for RPGs, the processors of which logically belong to their parent groups). (was: Currently all operations in FlowFileRepository go through a single column family (the default). We should investigate the performance benefit, if any, of using different column families for different process groups (special care must be taken for RPGs).) > Investigate per process group column families > - > > Key: MINIFICPP-1538 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1538 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Adam Debreceni >Assignee: Adam Debreceni >Priority: Major > > Currently all operations in FlowFileRepository go through a single column > family (the default). We should investigate the performance benefit, if any, > of using different column families for different process groups (special care > must be taken for RPGs, the processors of which logically belong to their > parent groups). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (MINIFICPP-1538) Investigate per process group column families
Adam Debreceni created MINIFICPP-1538: - Summary: Investigate per process group column families Key: MINIFICPP-1538 URL: https://issues.apache.org/jira/browse/MINIFICPP-1538 Project: Apache NiFi MiNiFi C++ Issue Type: New Feature Reporter: Adam Debreceni Assignee: Adam Debreceni Currently all operations in FlowFileRepository go through a single column family (the default). We should investigate the performance benefit, if any, of using different column families for different process groups (special care must be taken for RPGs). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (NIFI-8401) o.apache.nifi.controller.FlowController Failed to capture component stats for Stats History on Windows Systems
[ https://issues.apache.org/jira/browse/NIFI-8401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Turcsanyi resolved NIFI-8401. --- Resolution: Duplicate > o.apache.nifi.controller.FlowController Failed to capture component stats for > Stats History on Windows Systems > -- > > Key: NIFI-8401 > URL: https://issues.apache.org/jira/browse/NIFI-8401 > Project: Apache NiFi > Issue Type: Bug > Components: Configuration >Affects Versions: 1.13.2 > Environment: Windows 10 > Windows Server 2019 >Reporter: Eric Detwiler >Priority: Major > > 1.13.2 on Windows systems does not capture stats for Stats History by > default. Error log below: > Also located issue mentioned in post from March 2021. > https://community.cloudera.com/t5/Support-Questions/ERROR-FlowController-Failed-to-capture-component-stats-for/td-p/312486 > Status History as a result does not function. > > 2021-04-02 07:56:00,013 INFO [pool-20-thread-1] > o.a.n.c.r.WriteAheadFlowFileRepository Initiating checkpoint of FlowFile > Repository > 2021-04-02 07:56:00,014 INFO [pool-20-thread-1] > o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile > Repository with 0 records in 0 milliseconds > 2021-04-02 07:56:17,311 ERROR [Timer-Driven Process Thread-7] > o.apache.nifi.controller.FlowController Failed to capture component stats for > Stats History > java.lang.NullPointerException: null > at > org.apache.nifi.diagnostics.SystemDiagnostics.getOpenFileHandles(SystemDiagnostics.java:224) > at > org.apache.nifi.controller.FlowController.getNodeStatusSnapshot(FlowController.java:3011) > at > org.apache.nifi.controller.FlowController.access$400(FlowController.java:230) > at org.apache.nifi.controller.FlowController$2.run(FlowController.java:697) > at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-8400) SystemDiagnostics throws NPE on Windows
[ https://issues.apache.org/jira/browse/NIFI-8400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Turcsanyi updated NIFI-8400: -- Resolution: Fixed Status: Resolved (was: Patch Available) > SystemDiagnostics throws NPE on Windows > --- > > Key: NIFI-8400 > URL: https://issues.apache.org/jira/browse/NIFI-8400 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Major > Fix For: 1.14.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > SystemDiagnostics includes some Long member variables such as openFileHandles > that are not populated on Windows, so they remain null and when > getOpenFileHandles() is called, the null is cast to a long which throws an > NPE. > The member variables should be long not Long, thereby getting a default value > of zero and avoiding an NPE when the values are not populated. If Long is > used elsewhere, a null check should be added to avoid possible NPEs when > calling the setter methods. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-8400) SystemDiagnostics throws NPE on Windows
[ https://issues.apache.org/jira/browse/NIFI-8400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17316944#comment-17316944 ] ASF subversion and git services commented on NIFI-8400: --- Commit 33ec8c8427eb2cebf52f5f91e81785bc285e6544 in nifi's branch refs/heads/main from Matt Burgess [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=33ec8c8 ] NIFI-8400: Use longs in SystemDiagnostics, add null checks This closes #4980. Signed-off-by: Peter Turcsanyi > SystemDiagnostics throws NPE on Windows > --- > > Key: NIFI-8400 > URL: https://issues.apache.org/jira/browse/NIFI-8400 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Major > Fix For: 1.14.0 > > Time Spent: 20m > Remaining Estimate: 0h > > SystemDiagnostics includes some Long member variables such as openFileHandles > that are not populated on Windows, so they remain null and when > getOpenFileHandles() is called, the null is cast to a long which throws an > NPE. > The member variables should be long not Long, thereby getting a default value > of zero and avoiding an NPE when the values are not populated. If Long is > used elsewhere, a null check should be added to avoid possible NPEs when > calling the setter methods. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] asfgit closed pull request #4980: NIFI-8400: Use longs in SystemDiagnostics, add null checks
asfgit closed pull request #4980: URL: https://github.com/apache/nifi/pull/4980 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] turcsanyip commented on pull request #4980: NIFI-8400: Use longs in SystemDiagnostics, add null checks
turcsanyip commented on pull request #4980: URL: https://github.com/apache/nifi/pull/4980#issuecomment-815548774 @mattyb149 Thanks for the fix! @naddym Thanks for the review! +1 LGTM Merging to main. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #1042: MINIFICPP-1457 - Implement InputRequirements
adamdebreceni commented on a change in pull request #1042:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1042#discussion_r609397913
##
File path: libminifi/src/core/Processor.cpp
##
@@ -386,6 +379,39 @@ std::shared_ptr
Processor::pickIncomingConnection() {
return getNextIncomingConnectionImpl(rel_guard);
}
+using annotation::input::EInputRequirement;
+
+void Processor::validateAnnotations() const {
Review comment:
some unit tests could 'secure' this method
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #1042: MINIFICPP-1457 - Implement InputRequirements
adamdebreceni commented on a change in pull request #1042: URL: https://github.com/apache/nifi-minifi-cpp/pull/1042#discussion_r609396538 ## File path: libminifi/src/ThreadedSchedulingAgent.cpp ## @@ -88,6 +88,7 @@ void ThreadedSchedulingAgent::schedule(std::shared_ptr processo auto sessionFactory = std::make_shared(processContext); + processor->validateAnnotations(); Review comment: given our direction towards all kinds of flow-verification, this could be moved into either a `FlowController::verifyFlow(ProcessGroup&)` or `ProcessGroup::verify` (the former might be better as then we could access the services and repositories all at once) which could then be expanded as needed (calling it from `FlowController::load`) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] adam-markovics commented on a change in pull request #1042: MINIFICPP-1457 - Implement InputRequirements
adam-markovics commented on a change in pull request #1042:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1042#discussion_r609378101
##
File path: libminifi/src/core/Processor.cpp
##
@@ -386,6 +379,39 @@ std::shared_ptr
Processor::pickIncomingConnection() {
return getNextIncomingConnectionImpl(rel_guard);
}
+using annotation::input::EInputRequirement;
+
+void Processor::validateAnnotations() const {
+ switch (getInputRequirement()) {
+case EInputRequirement::INPUT_ALLOWED:
+ return;
+case EInputRequirement::INPUT_REQUIRED: {
+ if (!hasIncomingConnections()) {
+throw Exception(PROCESS_SCHEDULE_EXCEPTION, "INPUT_REQUIRED was
specified for the processor, but no inputs were found");
+ }
+ break;
+}
+case EInputRequirement::INPUT_FORBIDDEN: {
+ if (hasIncomingConnections()) {
+throw Exception(PROCESS_SCHEDULE_EXCEPTION, "INPUT_FORBIDDEN was
specified for the processor, but at least one input was found");
+ }
+ break;
+}
+ }
+}
+
+EInputRequirement Processor::getInputRequirement() const {
+ // default input requirement
+ EInputRequirement myInputRequirement = EInputRequirement::INPUT_ALLOWED;
+ // get own input annotation if InputRequirementAnnotationBase is a base class
Review comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #1042: MINIFICPP-1457 - Implement InputRequirements
adamdebreceni commented on a change in pull request #1042:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1042#discussion_r609356101
##
File path: libminifi/src/core/Processor.cpp
##
@@ -386,6 +379,39 @@ std::shared_ptr
Processor::pickIncomingConnection() {
return getNextIncomingConnectionImpl(rel_guard);
}
+using annotation::input::EInputRequirement;
+
+void Processor::validateAnnotations() const {
+ switch (getInputRequirement()) {
+case EInputRequirement::INPUT_ALLOWED:
+ return;
+case EInputRequirement::INPUT_REQUIRED: {
+ if (!hasIncomingConnections()) {
+throw Exception(PROCESS_SCHEDULE_EXCEPTION, "INPUT_REQUIRED was
specified for the processor, but no inputs were found");
+ }
+ break;
+}
+case EInputRequirement::INPUT_FORBIDDEN: {
+ if (hasIncomingConnections()) {
+throw Exception(PROCESS_SCHEDULE_EXCEPTION, "INPUT_FORBIDDEN was
specified for the processor, but at least one input was found");
+ }
+ break;
+}
+ }
+}
+
+EInputRequirement Processor::getInputRequirement() const {
+ // default input requirement
+ EInputRequirement myInputRequirement = EInputRequirement::INPUT_ALLOWED;
+ // get own input annotation if InputRequirementAnnotationBase is a base class
Review comment:
this comment got out of sync with the removal of
`InputRequirementAnnotationBase`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] timeabarna commented on pull request #4824: NIFI-8176 adding ackChecked flag to QuerySplunkIndexingStatus.onTrigg…
timeabarna commented on pull request #4824: URL: https://github.com/apache/nifi/pull/4824#issuecomment-815477807 @tpalfy Thanks Tamas, PR has been updated with the Property changes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
