[GitHub] [nifi] mcgilman opened a new pull request #3846: NIFI-6811: Allow enterGroup to complete prior to loading CS table
mcgilman opened a new pull request #3846: NIFI-6811: Allow enterGroup to complete prior to loading CS table URL: https://github.com/apache/nifi/pull/3846 NIFI-6811: - Allowing enterGroup to complete prior to loading the controller service table. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (NIFI-6811) UI - Go to controller service issue
Matt Gilman created NIFI-6811: - Summary: UI - Go to controller service issue Key: NIFI-6811 URL: https://issues.apache.org/jira/browse/NIFI-6811 Project: Apache NiFi Issue Type: Bug Components: Core UI Reporter: Matt Gilman Assignee: Matt Gilman When going to a controller service in an ancestor group from the controller service table in a descendant group, we need to wait for enterGroup to complete before attempting to load the table. This is because the table will ask for the parameter context which is loaded conditionally based on the currently loaded group. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] mcgilman commented on issue #3841: NIFI-6807: When a Controller Service's state is transitioned to ENABL…
mcgilman commented on issue #3841: NIFI-6807: When a Controller Service's state is transitioned to ENABL… URL: https://github.com/apache/nifi/pull/3841#issuecomment-546164438 Will review... This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi-registry] apiri opened a new pull request #246: NIFIREG-341 Make use of ASF shortened URL to point to community Slack…
apiri opened a new pull request #246: NIFIREG-341 Make use of ASF shortened URL to point to community Slack… URL: https://github.com/apache/nifi-registry/pull/246 … invite link. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Moved] (NIFIREG-341) Update README to make use of ASF url for community slack channel
[ https://issues.apache.org/jira/browse/NIFIREG-341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aldrin Piri moved NIFI-6810 to NIFIREG-341: --- Key: NIFIREG-341 (was: NIFI-6810) Workflow: jira (was: patch-available, re-open possible) Project: NiFi Registry (was: Apache NiFi) > Update README to make use of ASF url for community slack channel > > > Key: NIFIREG-341 > URL: https://issues.apache.org/jira/browse/NIFIREG-341 > Project: NiFi Registry > Issue Type: Improvement >Reporter: Aldrin Piri >Assignee: Aldrin Piri >Priority: Trivial > > We can use the shortened URL https://s.apache.org/nifi-community-slack to > point to the community slack invite link should it need to be refreshed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (NIFI-6810) Update README to make use of ASF url for community slack channel
Aldrin Piri created NIFI-6810: - Summary: Update README to make use of ASF url for community slack channel Key: NIFI-6810 URL: https://issues.apache.org/jira/browse/NIFI-6810 Project: Apache NiFi Issue Type: Improvement Reporter: Aldrin Piri Assignee: Aldrin Piri We can use the shortened URL https://s.apache.org/nifi-community-slack to point to the community slack invite link should it need to be refreshed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-6809) Update README to make use of ASF url for community slack channel
[ https://issues.apache.org/jira/browse/NIFI-6809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aldrin Piri updated NIFI-6809: -- Priority: Trivial (was: Major) > Update README to make use of ASF url for community slack channel > > > Key: NIFI-6809 > URL: https://issues.apache.org/jira/browse/NIFI-6809 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Aldrin Piri >Assignee: Aldrin Piri >Priority: Trivial > Time Spent: 10m > Remaining Estimate: 0h > > We can use the shortened URL https://s.apache.org/nifi-community-slack to > point to the community slack invite link should it need to be refreshed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-6809) Update README to make use of ASF url for community slack channel
[ https://issues.apache.org/jira/browse/NIFI-6809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aldrin Piri updated NIFI-6809: -- Status: Patch Available (was: Open) > Update README to make use of ASF url for community slack channel > > > Key: NIFI-6809 > URL: https://issues.apache.org/jira/browse/NIFI-6809 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Aldrin Piri >Assignee: Aldrin Piri >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > We can use the shortened URL https://s.apache.org/nifi-community-slack to > point to the community slack invite link should it need to be refreshed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] apiri opened a new pull request #3845: NIFI-6809 Make use of ASF shortened URL to point to community SLack i…
apiri opened a new pull request #3845: NIFI-6809 Make use of ASF shortened URL to point to community SLack i… URL: https://github.com/apache/nifi/pull/3845 …nvite link. Thank you for submitting a contribution to Apache NiFi. Please provide a short description of the PR here: Description of PR _Enables X functionality; fixes bug NIFI-._ In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with **NIFI-** where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically `master`)? - [ ] Is your initial contribution a single, squashed commit? _Additional commits in response to PR reviewer feedback should be made on this branch and pushed to allow change tracking. Do not `squash` or use `--force` when pushing to allow for clean monitoring of changes._ ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via `mvn -Pcontrib-check clean install` at the root `nifi` folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] Have you verified that the full build is successful on both JDK 8 and JDK 11? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE` file, including the main `LICENSE` file under `nifi-assembly`? - [ ] If applicable, have you updated the `NOTICE` file, including the main `NOTICE` file found under `nifi-assembly`? - [ ] If adding new Properties, have you added `.displayName` in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (NIFI-6809) Update README to make use of ASF url for community slack channel
Aldrin Piri created NIFI-6809: - Summary: Update README to make use of ASF url for community slack channel Key: NIFI-6809 URL: https://issues.apache.org/jira/browse/NIFI-6809 Project: Apache NiFi Issue Type: Improvement Reporter: Aldrin Piri Assignee: Aldrin Piri We can use the shortened URL https://s.apache.org/nifi-community-slack to point to the community slack invite link should it need to be refreshed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] alopresto commented on a change in pull request #3844: NIFI-1474 Added validation for p in scrypt
alopresto commented on a change in pull request #3844: NIFI-1474 Added validation for p in scrypt URL: https://github.com/apache/nifi/pull/3844#discussion_r338842726 ## File path: nifi-commons/nifi-security-utils/src/main/java/org/apache/nifi/security/util/crypto/ScryptCipherProvider.java ## @@ -78,6 +79,17 @@ public ScryptCipherProvider(int n, int r, int p) { if (p < DEFAULT_P) { logger.warn("The provided parallelization factor {} is below the recommended minimum {}", p, DEFAULT_P); } + +if (!isPValid(r, p)) { +logger.warn("Based on the provided block size {}, the provided parallelization factor {} is out of bounds", r, p); +throw new IllegalArgumentException("Invalid p value exceeds p boundary"); +} +} + +public static boolean isPValid(int r, int p) { +// Calculate p boundary +double pBoundary = ((Math.pow(2, 32)) - 1) * (32.0 / (r * 128)); Review comment: I think we may need to validate that `r` is a positive integer as well, because submitting a negative number or 0 here will cause problems. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] alopresto commented on issue #3844: NIFI-1474 Added validation for p in scrypt
alopresto commented on issue #3844: NIFI-1474 Added validation for p in scrypt URL: https://github.com/apache/nifi/pull/3844#issuecomment-546150069 Reviewing... This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] mtien-apache opened a new pull request #3844: NIFI-1474 Added validation for p in scrypt
mtien-apache opened a new pull request #3844: NIFI-1474 Added validation for p in scrypt URL: https://github.com/apache/nifi/pull/3844 Thank you for submitting a contribution to Apache NiFi. Please provide a short description of the PR here: Description of PR Added validation, used in constructor, and added unit tests. Failing unit test `TestListFile#testFilterAge` but this is not related to my changes. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with **NIFI-** where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically `master`)? - [ ] Is your initial contribution a single, squashed commit? _Additional commits in response to PR reviewer feedback should be made on this branch and pushed to allow change tracking. Do not `squash` or use `--force` when pushing to allow for clean monitoring of changes._ ### For code changes: - [x] Have you ensured that the full suite of tests is executed via `mvn -Pcontrib-check clean install` at the root `nifi` folder? - [x] Have you written or updated unit tests to verify your changes? - [ ] Have you verified that the full build is successful on both JDK 8 and JDK 11? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE` file, including the main `LICENSE` file under `nifi-assembly`? - [ ] If applicable, have you updated the `NOTICE` file, including the main `NOTICE` file found under `nifi-assembly`? - [ ] If adding new Properties, have you added `.displayName` in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [x] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (NIFI-6806) IllegalStateExceptionMapper, etc. should always log full stack trace
[ https://issues.apache.org/jira/browse/NIFI-6806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16959299#comment-16959299 ] ASF subversion and git services commented on NIFI-6806: --- Commit 8079d2531e561ad5dbbfe9e8dc4b818d694d57a6 in nifi's branch refs/heads/master from Mark Payne [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=8079d25 ] NIFI-6805, NIFI-6806: Updated toString() of StandardControllerServiceNode so that it includes the component's type and UUID, updated the more generic Exception Mappers to include stack traces regardless of the log level that is enabled This closes #3840 Signed-off-by: Mike Thomsen > IllegalStateExceptionMapper, etc. should always log full stack trace > > > Key: NIFI-6806 > URL: https://issues.apache.org/jira/browse/NIFI-6806 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 1.11.0 > > > The IllegalStateExceptionMapper is used when any IllegalStateException makes > its way all the way back to the web tier. This happens, for instance, if a > Controller Service cannot be enabled. When this happens, the Exception Mapper > only logs the stack trace if DEBUG is enabled. This is an old pattern that > was followed during early releases to prevent such verbose logging, but this > Exception Mapper needs to be updated to always log the stack traces. > Otherwise, we have no way of knowing the underlying cause of the failure. > IllegalArgumentExceptionMapper and a few others also appear to follow this > same pattern and need to be updated. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-6805) When ControllerServiceNode is logged, it no longer includes the UUID of the Controller Service
[ https://issues.apache.org/jira/browse/NIFI-6805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16959298#comment-16959298 ] ASF subversion and git services commented on NIFI-6805: --- Commit 8079d2531e561ad5dbbfe9e8dc4b818d694d57a6 in nifi's branch refs/heads/master from Mark Payne [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=8079d25 ] NIFI-6805, NIFI-6806: Updated toString() of StandardControllerServiceNode so that it includes the component's type and UUID, updated the more generic Exception Mappers to include stack traces regardless of the log level that is enabled This closes #3840 Signed-off-by: Mike Thomsen > When ControllerServiceNode is logged, it no longer includes the UUID of the > Controller Service > -- > > Key: NIFI-6805 > URL: https://issues.apache.org/jira/browse/NIFI-6805 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.10.0 >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Critical > Fix For: 1.11.0 > > Time Spent: 10m > Remaining Estimate: 0h > > The `toString()` method of StandardControllerServiceNode was changed in > NIFI-6201. The new `toString()` method no longer contains the CS's UUID. This > means that when log messages are generated about a Controller Service, > there's no easy way to track it back to the Controller Service that is > affected. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] asfgit closed pull request #3840: NIFI-6805, NIFI-6806: Updated toString() of StandardControllerService…
asfgit closed pull request #3840: NIFI-6805, NIFI-6806: Updated toString() of StandardControllerService… URL: https://github.com/apache/nifi/pull/3840 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Assigned] (NIFI-6394) Ability to export contents of a List Queue
[ https://issues.apache.org/jira/browse/NIFI-6394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Taft reassigned NIFI-6394: --- Assignee: Adam Taft > Ability to export contents of a List Queue > -- > > Key: NIFI-6394 > URL: https://issues.apache.org/jira/browse/NIFI-6394 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Reporter: Tim Chermak >Assignee: Adam Taft >Priority: Minor > > When in the List Queue window, we would like to see a new feature enabling > the ability to export the detail contents of the List Queue to XLS or CSV, > etc. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (NIFI-6742) H2 test.mv.db left after build
[ https://issues.apache.org/jira/browse/NIFI-6742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Taft reassigned NIFI-6742: --- Assignee: Adam Taft > H2 test.mv.db left after build > -- > > Key: NIFI-6742 > URL: https://issues.apache.org/jira/browse/NIFI-6742 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.9.2 >Reporter: Adam Taft >Assignee: Adam Taft >Priority: Trivial > > DBCPServiceTest.java leaves a test file called `test.mv.db` in the home > directory when building NiFi. This is an H2 database file which should > ideally be cleaned up after building. > Recommend using a JUnit TemporaryFolder or a maven target directory which > would enable more automatic cleanup of the temporary file. > There are two references in the unit test to this H2 database file at lines: > 360, 408 > [https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-services/nifi-dbcp-service-bundle/nifi-dbcp-service/src/test/java/org/apache/nifi/dbcp/DBCPServiceTest.java] > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-6054) phoenix DBCP connection pool
[ https://issues.apache.org/jira/browse/NIFI-6054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16959210#comment-16959210 ] Karthik Narayanan commented on NIFI-6054: - create a new pull request, no new additional nars needed. [https://github.com/apache/nifi/pull/3843] > phoenix DBCP connection pool > > > Key: NIFI-6054 > URL: https://issues.apache.org/jira/browse/NIFI-6054 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Karthik Narayanan >Assignee: Karthik Narayanan >Priority: Major > > I have seen a lot of users having trouble connecting to phoenix and query > hbase. This Jira will create a DBCP connection pool controller service for > phoenix. This will be a lot easier than creating shaded jars with > configuration xmls embedded in them. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] rkarthik29 opened a new pull request #3843: phoenix dbcp service changes for Nifi-6054
rkarthik29 opened a new pull request #3843: phoenix dbcp service changes for Nifi-6054 URL: https://github.com/apache/nifi/pull/3843 Thank you for submitting a contribution to Apache NiFi. Please provide a short description of the PR here: Description of PR _Enables X functionality; fixes bug NIFI-._ In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? Yes - [ ] Does your PR title start with **NIFI-** where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. Yes - [ ] Has your PR been rebased against the latest commit within the target branch (typically `master`)?Yes - [ ] Is your initial contribution a single, squashed commit? _Additional commits in response to PR reviewer feedback should be made on this branch and pushed to allow change tracking. Do not `squash` or use `--force` when pushing to allow for clean monitoring of changes._ ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via `mvn -Pcontrib-check clean install` at the root `nifi` folder? Yes - [ ] Have you written or updated unit tests to verify your changes? Yes - [ ] Have you verified that the full build is successful on both JDK 8 and JDK 11? Yes - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? Yes - [ ] If applicable, have you updated the `LICENSE` file, including the main `LICENSE` file under `nifi-assembly`? Yes - [ ] If applicable, have you updated the `NOTICE` file, including the main `NOTICE` file found under `nifi-assembly`? - [ ] If adding new Properties, have you added `.displayName` in addition to .name (programmatic access) for each of the new properties? Yes ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] mcgilman opened a new pull request #3842: NIFI-6687: Fixing Ranger unit tests
mcgilman opened a new pull request #3842: NIFI-6687: Fixing Ranger unit tests URL: https://github.com/apache/nifi/pull/3842 NIFI-6687: - Using correct NiFi Ranger Service name in unit tests. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Reopened] (NIFI-6687) Update Ranger dependencies
[ https://issues.apache.org/jira/browse/NIFI-6687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman reopened NIFI-6687: --- > Update Ranger dependencies > -- > > Key: NIFI-6687 > URL: https://issues.apache.org/jira/browse/NIFI-6687 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Major > Fix For: 1.11.0 > > Time Spent: 40m > Remaining Estimate: 0h > > Upgrade the Ranger authorizer to leverage version 2.x. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-6687) Update Ranger dependencies
[ https://issues.apache.org/jira/browse/NIFI-6687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16959203#comment-16959203 ] Matt Gilman commented on NIFI-6687: --- Reopening to address broken unit tests > Update Ranger dependencies > -- > > Key: NIFI-6687 > URL: https://issues.apache.org/jira/browse/NIFI-6687 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Major > Fix For: 1.11.0 > > Time Spent: 40m > Remaining Estimate: 0h > > Upgrade the Ranger authorizer to leverage version 2.x. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] SandishKumarHN commented on issue #3748: NIFI-6680: Nifi PutKudu processor - Convert record field names to low…
SandishKumarHN commented on issue #3748: NIFI-6680: Nifi PutKudu processor - Convert record field names to low… URL: https://github.com/apache/nifi/pull/3748#issuecomment-546081593 > @SandishKumarHN, Would you be able to merge these changes, or should I tag someone else? @kjmccarthy after build pass. I think @tpalfy should be able to do it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] SandishKumarHN commented on issue #3611: NIFI-6009 ScanKudu Processor
SandishKumarHN commented on issue #3611: NIFI-6009 ScanKudu Processor URL: https://github.com/apache/nifi/pull/3611#issuecomment-546081236 @tpalfy @pvillard31 can we please merge this ? and this https://github.com/apache/nifi/pull/. we are waiting on these two to place at customer place. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] rkarthik29 closed pull request #3336: Nifi 6054
rkarthik29 closed pull request #3336: Nifi 6054 URL: https://github.com/apache/nifi/pull/3336 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] kjmccarthy commented on issue #3748: NIFI-6680: Nifi PutKudu processor - Convert record field names to low…
kjmccarthy commented on issue #3748: NIFI-6680: Nifi PutKudu processor - Convert record field names to low… URL: https://github.com/apache/nifi/pull/3748#issuecomment-546055143 @SandishKumarHN, Would you be able to merge these changes, or should I tag someone else? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] kjmccarthy opened a new pull request #3748: NIFI-6680: Nifi PutKudu processor - Convert record field names to low…
kjmccarthy opened a new pull request #3748: NIFI-6680: Nifi PutKudu processor - Convert record field names to low… URL: https://github.com/apache/nifi/pull/3748 Thank you for submitting a contribution to Apache NiFi. Please provide a short description of the PR here: Description of PR Adds additional attribute that specifies whether or not to convert columns in the Kudu schema to lowercase when retrieving the column's index In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with **NIFI-** where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically `master`)? - [x] Is your initial contribution a single, squashed commit? _Additional commits in response to PR reviewer feedback should be made on this branch and pushed to allow change tracking. Do not `squash` or use `--force` when pushing to allow for clean monitoring of changes._ ### For code changes: - [x] Have you ensured that the full suite of tests is executed via `mvn -Pcontrib-check clean install` at the root `nifi` folder? - [x] Have you written or updated unit tests to verify your changes? - [x] Have you verified that the full build is successful on both JDK 8 and JDK 11? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE` file, including the main `LICENSE` file under `nifi-assembly`? - [ ] If applicable, have you updated the `NOTICE` file, including the main `NOTICE` file found under `nifi-assembly`? - [ ] If adding new Properties, have you added `.displayName` in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] kjmccarthy closed pull request #3748: NIFI-6680: Nifi PutKudu processor - Convert record field names to low…
kjmccarthy closed pull request #3748: NIFI-6680: Nifi PutKudu processor - Convert record field names to low… URL: https://github.com/apache/nifi/pull/3748 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] rkarthik29 commented on issue #3306: nifi oracle cdc changes using xstream
rkarthik29 commented on issue #3306: nifi oracle cdc changes using xstream URL: https://github.com/apache/nifi/pull/3306#issuecomment-546035850 @joewitt all conflicts and build issues resolved. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi-registry] kevdoran commented on issue #245: NIFIREG-252: adding mavendocker build profile
kevdoran commented on issue #245: NIFIREG-252: adding mavendocker build profile URL: https://github.com/apache/nifi-registry/pull/245#issuecomment-546028884 +1 for relocating it and making it a maven project, as well as for doing that as a separate jira/pr. I will re-review this PR as-is. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (NIFI-6799) Add RecordSink implementation for Prometheus scraping
[ https://issues.apache.org/jira/browse/NIFI-6799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16958986#comment-16958986 ] ASF subversion and git services commented on NIFI-6799: --- Commit 8771c35f4a01c38ab94ab80b96997ee5f82af094 in nifi's branch refs/heads/master from Matt Burgess [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=8771c35 ] NIFI-6799: Add PrometheusRecordSink and reset() method to RecordSinkService (#3839) This closes #3839 > Add RecordSink implementation for Prometheus scraping > - > > Key: NIFI-6799 > URL: https://issues.apache.org/jira/browse/NIFI-6799 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Major > Time Spent: 0.5h > Remaining Estimate: 0h > > With the advent of NIFI-6780, a RecordSinkService controller service > interface is available and is used by components like QueryNiFiReportingTask > to decouple the collection of data from its destination or transmission > method. Initial implementations in NIFI-6780 included a Site-to-Site record > sink as well as a DatabaseRecordSink for writing to RDBMS targets. > This Jira proposes to add a RecordSinkService implementation to enable > Prometheus scraping. The PrometheusReportingTask exists (via NIFI-5435) to > allow scraping of many available metrics, but a PrometheusRecordSink would > allow scraping of these plus derived metrics (via SQL queries executed by > QueryNiFiReportingTask) and analytical metrics such as connection status > predictions (Time-to-Backpressure, e.g.). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] YolandaMDavis merged pull request #3839: NIFI-6799: Add PrometheusRecordSink and reset() method to RecordSinkService
YolandaMDavis merged pull request #3839: NIFI-6799: Add PrometheusRecordSink and reset() method to RecordSinkService URL: https://github.com/apache/nifi/pull/3839 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (NIFI-6807) When enabling Controller Service and referencing components, some references may fail to enable
[ https://issues.apache.org/jira/browse/NIFI-6807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-6807: - Fix Version/s: 1.11.0 Status: Patch Available (was: Open) > When enabling Controller Service and referencing components, some references > may fail to enable > --- > > Key: NIFI-6807 > URL: https://issues.apache.org/jira/browse/NIFI-6807 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Critical > Fix For: 1.11.0 > > Time Spent: 10m > Remaining Estimate: 0h > > When a user chooses to enable a Controller Service and its referencing > components, there exists a situation where we can sometimes see the > referencing Controller Service fail to enable. This is particularly > problematic in a clustered environment, as it can result in nodes being > disconnected from the cluster because the service didn't enable. > To replicate, create a JSONRecordSetWriter service and an AvroSchemaRegistry > service. Configure the record writer to depend on the schema registry (and > all other necessary configuration to make the services valid). Now, enable > AvroSchemaRegistry and all referencing components. The JSONRecordSetWriter > will sometimes fail to enable because it is not yet valid. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-6806) IllegalStateExceptionMapper, etc. should always log full stack trace
[ https://issues.apache.org/jira/browse/NIFI-6806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-6806: - Fix Version/s: 1.11.0 Status: Patch Available (was: Open) > IllegalStateExceptionMapper, etc. should always log full stack trace > > > Key: NIFI-6806 > URL: https://issues.apache.org/jira/browse/NIFI-6806 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 1.11.0 > > > The IllegalStateExceptionMapper is used when any IllegalStateException makes > its way all the way back to the web tier. This happens, for instance, if a > Controller Service cannot be enabled. When this happens, the Exception Mapper > only logs the stack trace if DEBUG is enabled. This is an old pattern that > was followed during early releases to prevent such verbose logging, but this > Exception Mapper needs to be updated to always log the stack traces. > Otherwise, we have no way of knowing the underlying cause of the failure. > IllegalArgumentExceptionMapper and a few others also appear to follow this > same pattern and need to be updated. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-6805) When ControllerServiceNode is logged, it no longer includes the UUID of the Controller Service
[ https://issues.apache.org/jira/browse/NIFI-6805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-6805: - Fix Version/s: 1.11.0 Status: Patch Available (was: Open) > When ControllerServiceNode is logged, it no longer includes the UUID of the > Controller Service > -- > > Key: NIFI-6805 > URL: https://issues.apache.org/jira/browse/NIFI-6805 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.10.0 >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Critical > Fix For: 1.11.0 > > Time Spent: 10m > Remaining Estimate: 0h > > The `toString()` method of StandardControllerServiceNode was changed in > NIFI-6201. The new `toString()` method no longer contains the CS's UUID. This > means that when log messages are generated about a Controller Service, > there's no easy way to track it back to the Controller Service that is > affected. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] markap14 opened a new pull request #3841: NIFI-6807: When a Controller Service's state is transitioned to ENABL…
markap14 opened a new pull request #3841: NIFI-6807: When a Controller Service's state is transitioned to ENABL… URL: https://github.com/apache/nifi/pull/3841 …ING, complete the Future successfully, even if the Controller Service is not valid. The Controller Service will remain in the ENABLING state until it is made valid, at which point it will ENABLE (unless explicitly disabled first). This allows us to Enable a Controller Service and its referencing components, even if the referencing component is still invalid due to it not yet recognizing the the referenced service has been enabled. Thank you for submitting a contribution to Apache NiFi. Please provide a short description of the PR here: Description of PR _Enables X functionality; fixes bug NIFI-._ In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with **NIFI-** where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically `master`)? - [ ] Is your initial contribution a single, squashed commit? _Additional commits in response to PR reviewer feedback should be made on this branch and pushed to allow change tracking. Do not `squash` or use `--force` when pushing to allow for clean monitoring of changes._ ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via `mvn -Pcontrib-check clean install` at the root `nifi` folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] Have you verified that the full build is successful on both JDK 8 and JDK 11? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE` file, including the main `LICENSE` file under `nifi-assembly`? - [ ] If applicable, have you updated the `NOTICE` file, including the main `NOTICE` file found under `nifi-assembly`? - [ ] If adding new Properties, have you added `.displayName` in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] markap14 opened a new pull request #3840: NIFI-6805, NIFI-6806: Updated toString() of StandardControllerService…
markap14 opened a new pull request #3840: NIFI-6805, NIFI-6806: Updated toString() of StandardControllerService… URL: https://github.com/apache/nifi/pull/3840 …Node so that it includes the component's type and UUID, updated the more generic Exception Mappers to include stack traces regardless of the log level that is enabled Thank you for submitting a contribution to Apache NiFi. Please provide a short description of the PR here: Description of PR _Enables X functionality; fixes bug NIFI-._ In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with **NIFI-** where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically `master`)? - [ ] Is your initial contribution a single, squashed commit? _Additional commits in response to PR reviewer feedback should be made on this branch and pushed to allow change tracking. Do not `squash` or use `--force` when pushing to allow for clean monitoring of changes._ ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via `mvn -Pcontrib-check clean install` at the root `nifi` folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] Have you verified that the full build is successful on both JDK 8 and JDK 11? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE` file, including the main `LICENSE` file under `nifi-assembly`? - [ ] If applicable, have you updated the `NOTICE` file, including the main `NOTICE` file found under `nifi-assembly`? - [ ] If adding new Properties, have you added `.displayName` in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi-registry] ekovacs commented on issue #245: NIFIREG-252: adding mavendocker build profile
ekovacs commented on issue #245: NIFIREG-252: adding mavendocker build profile URL: https://github.com/apache/nifi-registry/pull/245#issuecomment-545966491 > Yeah, good points. I would support moving nifi-registry-core/nifi-registry-docker to a top level module that is responsible for both dockerhub and dockermaven. Do you think we should do that now or save that improvement for later? currently `nifi-registry/nifi-registry-core/nifi-registry-docker/dockerhub` is not a maven project (no pom.xml in its folder) this is another difference between the NiFi dockerhub and the NiFi Registry counter part. if we were to move it out from under the `nifi-registry-core` hierarchy, i'd also go for turning it into a maven project. If it is OK with you, i'd rather file a separate Jira / PR for that move & mavenization, to keep this PR small & single-work-item-focused. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi-minifi-cpp] asdaraujo commented on issue #666: MINIFICPP-1070 - Fixed handling of PKCS12 certificates
asdaraujo commented on issue #666: MINIFICPP-1070 - Fixed handling of PKCS12 certificates URL: https://github.com/apache/nifi-minifi-cpp/pull/666#issuecomment-545965975 Thanks, @arpadboda, and thanks for the linter tip. I'm sending an update with the linter errors fixed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (NIFI-6808) Use KeytabCredentialsService in HWX Schema Registry Service
Bryan Bende created NIFI-6808: - Summary: Use KeytabCredentialsService in HWX Schema Registry Service Key: NIFI-6808 URL: https://issues.apache.org/jira/browse/NIFI-6808 Project: Apache NiFi Issue Type: Improvement Reporter: Bryan Bende Assignee: Bryan Bende Previously the schema registry client did not provide a way to pass in JAAS config, so it relied on the JAAS system property. They recently merged an improvement to the client: [https://github.com/hortonworks/registry/pull/611] We should now be able to use a KeytabCredentialService to obtain the principal and keytab and pass in the appropriate configuration to the client. We'll have to wait until a new release of their client jar before we can merge this change. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi-registry] kevdoran commented on issue #245: NIFIREG-252: adding mavendocker build profile
kevdoran commented on issue #245: NIFIREG-252: adding mavendocker build profile URL: https://github.com/apache/nifi-registry/pull/245#issuecomment-545951655 Yeah, good points. I would support moving nifi-registry-core/nifi-registry-docker to a top level module that is responsible for both dockerhub and dockermaven. Do you think we should do that now or save that improvement for later? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] YolandaMDavis commented on issue #3839: NIFI-6799: Add PrometheusRecordSink and reset() method to RecordSinkService
YolandaMDavis commented on issue #3839: NIFI-6799: Add PrometheusRecordSink and reset() method to RecordSinkService URL: https://github.com/apache/nifi/pull/3839#issuecomment-545946067 @mattyb149 thanks for the update. I've run some additional tests just to confirm Prometheus can scrape both endpoints and those looked good as well. +1. Will merge upon successful Travis build. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] turcsanyip commented on a change in pull request #3774: NIFI-6729 - Created AbstractSingleAttributeBasedControllerServiceLook…
turcsanyip commented on a change in pull request #3774: NIFI-6729 - Created AbstractSingleAttributeBasedControllerServiceLook… URL: https://github.com/apache/nifi/pull/3774#discussion_r337638638 ## File path: nifi-nar-bundles/nifi-extension-utils/nifi-service-utils/src/main/java/org/apache/nifi/service/lookup/AbstractSingleAttributeBasedControllerServiceLookup.java ## @@ -0,0 +1,162 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions andf + * limitations under the License. + */ +package org.apache.nifi.service.lookup; + +import org.apache.nifi.annotation.lifecycle.OnDisabled; +import org.apache.nifi.annotation.lifecycle.OnEnabled; +import org.apache.nifi.components.PropertyDescriptor; +import org.apache.nifi.components.ValidationContext; +import org.apache.nifi.components.ValidationResult; +import org.apache.nifi.controller.AbstractControllerService; +import org.apache.nifi.controller.ConfigurationContext; +import org.apache.nifi.controller.ControllerService; +import org.apache.nifi.processor.exception.ProcessException; +import org.apache.nifi.processor.util.StandardValidators; + +import java.util.ArrayList; +import java.util.Collection; +import java.util.Collections; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import java.util.Optional; + +/** + * A lookup ControllerService that can choose one (from probably multiple) ControllerServices of the given type {@link S}. + * + * Selection is based on a single {@linkplain String} lookup key. + * + * Lookup key is provided as a value in an attribute map (usually coming form a flowfile) + * with a predefined key (see {@link #lookupAttribute()}). + * + * @param The type of service to be looked up + */ +public abstract class AbstractSingleAttributeBasedControllerServiceLookup extends AbstractControllerService { +protected volatile Map serviceMap; + +/** + * @return the Class that represents the type of service that will be returned by {@link #lookupService(Map)} + */ +public abstract Class getServiceType(); + +/** + * @return the name of attribute (usually from a flowfile) the value of which serves as the lookup key + * for the desired service (of type {@link S}) + */ +protected abstract String lookupAttribute(); + +/** + * Returns a ControllerService (of type {@link S}) based on the provided attributes map (usually by retrieving + * a lookup attribute from it via {@link #lookupAttribute()} and use it to identify the appropriate service). + * @param attributes Map containing the lookup attribute based on which ControllerService is chosen + * @return the chosen ControllerService + */ +public S lookupService(Map attributes) { +if (attributes == null) { +throw new ProcessException("Attributes map is null"); +} else if (!attributes.containsKey(lookupAttribute())) { +throw new ProcessException("Attributes must contain an attribute name '" + lookupAttribute() + "'"); +} + +Object lookupKey = Optional.of(lookupAttribute()) +.map(attributes::get) +.orElseThrow(() -> new ProcessException(lookupAttribute() + " cannot be null or blank")); + +S service = serviceMap.get(lookupKey); + +if (service == null) { +throw new ProcessException("No " + getServiceName() + " found for " + lookupAttribute()); +} + +return service; +} + +@OnEnabled +public void onEnabled(final ConfigurationContext context) { +Map serviceMap = new HashMap<>(); + +context.getProperties().keySet().stream() +.filter(PropertyDescriptor::isDynamic) +.forEach(propertyDescriptor -> { +S service = context.getProperty(propertyDescriptor).asControllerService(getServiceType()); + +serviceMap.put(propertyDescriptor.getName(), service); +}); + +this.serviceMap = Collections.unmodifiableMap(serviceMap); +} + +@OnDisabled +public void onDisabled() { +serviceMap = null; +} + +@Override +protected PropertyDescriptor getSupportedDynamicPropertyDescriptor(final String propertyDescriptorName) { +
[GitHub] [nifi] mattyb149 commented on issue #3839: NIFI-6799: Add PrometheusRecordSink and reset() method to RecordSinkService
mattyb149 commented on issue #3839: NIFI-6799: Add PrometheusRecordSink and reset() method to RecordSinkService URL: https://github.com/apache/nifi/pull/3839#issuecomment-545914180 I force-pushed my update so I could rebase and add the Jira case to the commit message, but I added the unit test you suggested. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (NIFI-6801) Connection analytics models should be distinct for connections
[ https://issues.apache.org/jira/browse/NIFI-6801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-6801: --- Resolution: Fixed Status: Resolved (was: Patch Available) > Connection analytics models should be distinct for connections > -- > > Key: NIFI-6801 > URL: https://issues.apache.org/jira/browse/NIFI-6801 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.10.0 >Reporter: Yolanda M. Davis >Assignee: Yolanda M. Davis >Priority: Major > Fix For: 1.11.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > Connections should have Status Analytics Model instances that are unique for > each connection such that calculations for the model can be referred to or > updated when new observations are available. Currently the model appears to > be shared across connection objects which leads to erroneous results, > especially when a connection does not have enough observations to update the > model (since the older values of a previous connection may still be present). > This behavior can be seen when debugging is enabled for analytics and noting > differences between a Timer Driven Thread execution of predictions vs the > NiFi Web Server thread predictions for multiple connections (a flow with one > connection would not exhibit this issue). The Time Driven thread may not > retrieve enough connection status observations to refresh the model, and when > that occurs, it may still contain calculations from a previous model and use > that for predictions. Observers may see behavior where web server and timer > driven predictions only have similar or matching predictions for one > connection but others may be mismatched, such as in the below (where > connection has similar prediction between threads but , and do > not): > {code:java} > 2019-10-22 08:01:34,539 INFO [NiFi Web Server-23] > o.a.nifi.reporting.StandardEventAccess : predicted time to COUNT > backpressure = 00:44:13 / 2653416 > 2019-10-22 08:01:34,539 INFO [NiFi Web Server-23] > o.a.nifi.reporting.StandardEventAccess : predicted time to BYTES > backpressure = 01:09:13 / 4153511 > 2019-10-22 08:01:34,540 INFO [NiFi Web Server-23] > o.a.nifi.reporting.StandardEventAccess : predicted time to COUNT > backpressure = 00:32:31 / 1951609 > 2019-10-22 08:01:34,540 INFO [NiFi Web Server-23] > o.a.nifi.reporting.StandardEventAccess : predicted time to BYTES > backpressure = 00:32:28 / 1948504 > 2019-10-22 08:01:34,541 INFO [NiFi Web Server-23] > o.a.nifi.reporting.StandardEventAccess : predicted time to COUNT > backpressure = 07:29:14 / 26954837 > 2019-10-22 08:01:34,541 INFO [NiFi Web Server-23] > o.a.nifi.reporting.StandardEventAccess : predicted time to BYTES > backpressure = 05:24:56 / 19496954 > 2019-10-22 08:02:02,767 INFO [Timer-Driven Process Thread-2] > o.a.nifi.reporting.StandardEventAccess : predicted time to COUNT > backpressure = 04:08:36 / 14916150 > 2019-10-22 08:02:02,767 INFO [Timer-Driven Process Thread-2] > o.a.nifi.reporting.StandardEventAccess : predicted time to BYTES > backpressure = 04:17:49 / 15469243 > 2019-10-22 08:02:02,769 INFO [Timer-Driven Process Thread-2] > o.a.nifi.reporting.StandardEventAccess : predicted time to COUNT > backpressure = 01:13:42 / 4422829 > 2019-10-22 08:02:02,769 INFO [Timer-Driven Process Thread-2] > o.a.nifi.reporting.StandardEventAccess : predicted time to BYTES > backpressure = 00:57:15 / 3435109 > 2019-10-22 08:02:02,770 INFO [Timer-Driven Process Thread-2] > o.a.nifi.reporting.StandardEventAccess : predicted time to COUNT > backpressure = 07:28:46 / 26926608 > 2019-10-22 08:02:02,770 INFO [Timer-Driven Process Thread-2] > o.a.nifi.reporting.StandardEventAccess : predicted time to BYTES > backpressure = 05:24:28 / 19468725 > {code} > A workaround for this is to increase the nifi.analytics.query.interval > property or decrease the nifi.components.status.snapshot.frequency property > to ensure obtaining enough observations for a refresh. However an appropriate > fix is to ensure model instances are unique for each connection and to ensure > that, by default, all threads will obtain enough observations for predictions. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] asfgit closed pull request #3838: NIFI-6801 - fix to ensure unique model instance is created for each c…
asfgit closed pull request #3838: NIFI-6801 - fix to ensure unique model instance is created for each c… URL: https://github.com/apache/nifi/pull/3838 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] mattyb149 commented on issue #3838: NIFI-6801 - fix to ensure unique model instance is created for each c…
mattyb149 commented on issue #3838: NIFI-6801 - fix to ensure unique model instance is created for each c… URL: https://github.com/apache/nifi/pull/3838#issuecomment-545903378 +1 LGTM, verified the same predictions are available between threads (i.e. the models are unique per connection). Thanks for the fix! Merging to master This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (NIFI-6801) Connection analytics models should be distinct for connections
[ https://issues.apache.org/jira/browse/NIFI-6801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16958868#comment-16958868 ] ASF subversion and git services commented on NIFI-6801: --- Commit 147365285cbe385a85e58371dd9570dc01683936 in nifi's branch refs/heads/master from Yolanda M. Davis [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=1473652 ] NIFI-6801 - fix to ensure unique model instance is created for each connection. Also increased default query interval. Signed-off-by: Matthew Burgess This closes #3838 > Connection analytics models should be distinct for connections > -- > > Key: NIFI-6801 > URL: https://issues.apache.org/jira/browse/NIFI-6801 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.10.0 >Reporter: Yolanda M. Davis >Assignee: Yolanda M. Davis >Priority: Major > Fix For: 1.11.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Connections should have Status Analytics Model instances that are unique for > each connection such that calculations for the model can be referred to or > updated when new observations are available. Currently the model appears to > be shared across connection objects which leads to erroneous results, > especially when a connection does not have enough observations to update the > model (since the older values of a previous connection may still be present). > This behavior can be seen when debugging is enabled for analytics and noting > differences between a Timer Driven Thread execution of predictions vs the > NiFi Web Server thread predictions for multiple connections (a flow with one > connection would not exhibit this issue). The Time Driven thread may not > retrieve enough connection status observations to refresh the model, and when > that occurs, it may still contain calculations from a previous model and use > that for predictions. Observers may see behavior where web server and timer > driven predictions only have similar or matching predictions for one > connection but others may be mismatched, such as in the below (where > connection has similar prediction between threads but , and do > not): > {code:java} > 2019-10-22 08:01:34,539 INFO [NiFi Web Server-23] > o.a.nifi.reporting.StandardEventAccess : predicted time to COUNT > backpressure = 00:44:13 / 2653416 > 2019-10-22 08:01:34,539 INFO [NiFi Web Server-23] > o.a.nifi.reporting.StandardEventAccess : predicted time to BYTES > backpressure = 01:09:13 / 4153511 > 2019-10-22 08:01:34,540 INFO [NiFi Web Server-23] > o.a.nifi.reporting.StandardEventAccess : predicted time to COUNT > backpressure = 00:32:31 / 1951609 > 2019-10-22 08:01:34,540 INFO [NiFi Web Server-23] > o.a.nifi.reporting.StandardEventAccess : predicted time to BYTES > backpressure = 00:32:28 / 1948504 > 2019-10-22 08:01:34,541 INFO [NiFi Web Server-23] > o.a.nifi.reporting.StandardEventAccess : predicted time to COUNT > backpressure = 07:29:14 / 26954837 > 2019-10-22 08:01:34,541 INFO [NiFi Web Server-23] > o.a.nifi.reporting.StandardEventAccess : predicted time to BYTES > backpressure = 05:24:56 / 19496954 > 2019-10-22 08:02:02,767 INFO [Timer-Driven Process Thread-2] > o.a.nifi.reporting.StandardEventAccess : predicted time to COUNT > backpressure = 04:08:36 / 14916150 > 2019-10-22 08:02:02,767 INFO [Timer-Driven Process Thread-2] > o.a.nifi.reporting.StandardEventAccess : predicted time to BYTES > backpressure = 04:17:49 / 15469243 > 2019-10-22 08:02:02,769 INFO [Timer-Driven Process Thread-2] > o.a.nifi.reporting.StandardEventAccess : predicted time to COUNT > backpressure = 01:13:42 / 4422829 > 2019-10-22 08:02:02,769 INFO [Timer-Driven Process Thread-2] > o.a.nifi.reporting.StandardEventAccess : predicted time to BYTES > backpressure = 00:57:15 / 3435109 > 2019-10-22 08:02:02,770 INFO [Timer-Driven Process Thread-2] > o.a.nifi.reporting.StandardEventAccess : predicted time to COUNT > backpressure = 07:28:46 / 26926608 > 2019-10-22 08:02:02,770 INFO [Timer-Driven Process Thread-2] > o.a.nifi.reporting.StandardEventAccess : predicted time to BYTES > backpressure = 05:24:28 / 19468725 > {code} > A workaround for this is to increase the nifi.analytics.query.interval > property or decrease the nifi.components.status.snapshot.frequency property > to ensure obtaining enough observations for a refresh. However an appropriate > fix is to ensure model instances are unique for each connection and to ensure > that, by default, all threads will obtain enough observations for predictions. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi-registry] ekovacs commented on a change in pull request #245: NIFIREG-252: adding mavendocker build profile
ekovacs commented on a change in pull request #245: NIFIREG-252: adding mavendocker build profile URL: https://github.com/apache/nifi-registry/pull/245#discussion_r338448518 ## File path: nifi-registry-docker-maven/dockermaven/pom.xml ## @@ -0,0 +1,101 @@ + + +http://maven.apache.org/POM/4.0.0; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd;> +4.0.0 + + +nifi-registry-docker-maven +org.apache.nifi.registry +1.0.0-SNAPSHOT + +1.0.0-SNAPSHOT + +dockermaven + + + +docker + + + +com.spotify +dockerfile-maven-plugin +1.3.5 + + +default + +build + + + +1000 +1000 + ${project.version} + target/nifi-registry-${project.version}-bin.zip + + apache/nifi-registry +${project.version}-dockermaven + + + + + + +maven-antrun-plugin +1.8 + + +copy-for-docker +process-sources + + + + + + + + + + +run + + + + + + +exec-maven-plugin +org.codehaus.mojo + + +Docker integration tests +integration-test + +exec + + + + ${project.version}-dockermaven +${project.version} + + ${project.basedir}/integration-test.sh + + + + Review comment: updated PR with _skipping_ on `-DskipTests` now this will be printed, if `skipTests` is set: ``` [INFO] --- exec-maven-plugin:1.3.2:exec (Docker integration tests) @ dockermaven --- [INFO] skipping execute as per configuraion ``` also, please note that this test would only run, when `integration-test` or later maven lifecycle phase (eg.: `verify`) is specified. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi-minifi-cpp] arpadboda commented on issue #666: MINIFICPP-1070 - Fixed handling of PKCS12 certificates
arpadboda commented on issue #666: MINIFICPP-1070 - Fixed handling of PKCS12 certificates URL: https://github.com/apache/nifi-minifi-cpp/pull/666#issuecomment-545883300 Looks good, will merge soon, thanks! @asdaraujo : next time please take a look at linter errors! (You can run linter by executing "make linter") ``` /Users/travis/build/apache/nifi-minifi-cpp/libminifi/src//controllers/SSLContextService.cpp:101: An else should appear on the same line as the preceding } [whitespace/newline] [4] /Users/travis/build/apache/nifi-minifi-cpp/libminifi/src//controllers/SSLContextService.cpp:101: If an else has a brace on one side, it should have it on both [readability/braces] [5] /Users/travis/build/apache/nifi-minifi-cpp/libminifi/src//controllers/SSLContextService.cpp:31: Unexpected \r (^M) found; better to use only \n [whitespace/newline] [1] /Users/travis/build/apache/nifi-minifi-cpp/libminifi/src//controllers/SSLContextService.cpp:73: Unexpected \r (^M) found; better to use only \n [whitespace/newline] [1] /Users/travis/build/apache/nifi-minifi-cpp/libminifi/src//controllers/SSLContextService.cpp:87: Unexpected \r (^M) found; better to use only \n [whitespace/newline] [1] /Users/travis/build/apache/nifi-minifi-cpp/libminifi/src//controllers/SSLContextService.cpp:88: Unexpected \r (^M) found; better to use only \n [whitespace/newline] [1] /Users/travis/build/apache/nifi-minifi-cpp/libminifi/src//controllers/SSLContextService.cpp:89: Unexpected \r (^M) found; better to use only \n [whitespace/newline] [1] /Users/travis/build/apache/nifi-minifi-cpp/libminifi/src//controllers/SSLContextService.cpp:90: Unexpected \r (^M) found; better to use only \n [whitespace/newline] [1] /Users/travis/build/apache/nifi-minifi-cpp/libminifi/src//controllers/SSLContextService.cpp:91: Unexpected \r (^M) found; better to use only \n [whitespace/newline] [1] /Users/travis/build/apache/nifi-minifi-cpp/libminifi/src//controllers/SSLContextService.cpp:92: Unexpected \r (^M) found; better to use only \n [whitespace/newline] [1] /Users/travis/build/apache/nifi-minifi-cpp/libminifi/src//controllers/SSLContextService.cpp:93: Unexpected \r (^M) found; better to use only \n [whitespace/newline] [1] /Users/travis/build/apache/nifi-minifi-cpp/libminifi/src//controllers/SSLContextService.cpp:94: Unexpected \r (^M) found; better to use only \n [whitespace/newline] [1] ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi-minifi-cpp] arpadboda edited a comment on issue #666: MINIFICPP-1070 - Fixed handling of PKCS12 certificates
arpadboda edited a comment on issue #666: MINIFICPP-1070 - Fixed handling of PKCS12 certificates URL: https://github.com/apache/nifi-minifi-cpp/pull/666#issuecomment-545883300 Looks good, will merge soon, thanks! @asdaraujo : next time please take a look at linter errors! (You can run linter by executing "make linter"). Will fix the current ones when merging. ``` /Users/travis/build/apache/nifi-minifi-cpp/libminifi/src//controllers/SSLContextService.cpp:101: An else should appear on the same line as the preceding } [whitespace/newline] [4] /Users/travis/build/apache/nifi-minifi-cpp/libminifi/src//controllers/SSLContextService.cpp:101: If an else has a brace on one side, it should have it on both [readability/braces] [5] /Users/travis/build/apache/nifi-minifi-cpp/libminifi/src//controllers/SSLContextService.cpp:31: Unexpected \r (^M) found; better to use only \n [whitespace/newline] [1] /Users/travis/build/apache/nifi-minifi-cpp/libminifi/src//controllers/SSLContextService.cpp:73: Unexpected \r (^M) found; better to use only \n [whitespace/newline] [1] /Users/travis/build/apache/nifi-minifi-cpp/libminifi/src//controllers/SSLContextService.cpp:87: Unexpected \r (^M) found; better to use only \n [whitespace/newline] [1] /Users/travis/build/apache/nifi-minifi-cpp/libminifi/src//controllers/SSLContextService.cpp:88: Unexpected \r (^M) found; better to use only \n [whitespace/newline] [1] /Users/travis/build/apache/nifi-minifi-cpp/libminifi/src//controllers/SSLContextService.cpp:89: Unexpected \r (^M) found; better to use only \n [whitespace/newline] [1] /Users/travis/build/apache/nifi-minifi-cpp/libminifi/src//controllers/SSLContextService.cpp:90: Unexpected \r (^M) found; better to use only \n [whitespace/newline] [1] /Users/travis/build/apache/nifi-minifi-cpp/libminifi/src//controllers/SSLContextService.cpp:91: Unexpected \r (^M) found; better to use only \n [whitespace/newline] [1] /Users/travis/build/apache/nifi-minifi-cpp/libminifi/src//controllers/SSLContextService.cpp:92: Unexpected \r (^M) found; better to use only \n [whitespace/newline] [1] /Users/travis/build/apache/nifi-minifi-cpp/libminifi/src//controllers/SSLContextService.cpp:93: Unexpected \r (^M) found; better to use only \n [whitespace/newline] [1] /Users/travis/build/apache/nifi-minifi-cpp/libminifi/src//controllers/SSLContextService.cpp:94: Unexpected \r (^M) found; better to use only \n [whitespace/newline] [1] ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi-registry] ekovacs commented on issue #245: NIFIREG-252: adding mavendocker build profile
ekovacs commented on issue #245: NIFIREG-252: adding mavendocker build profile URL: https://github.com/apache/nifi-registry/pull/245#issuecomment-545815844 > Thanks for the contribution @ekovacs! > > Overall this looks good to me. > > I had one inline suggestion about disabling a plugin execution under certain conditions, but that is more of a nice-to-have than a deal-breaker. > > At some point I would like to try to combine this with the `nifi-registry-docker` module, especially to de-duplicate items such as the shell scripts that get added to the image, but we can save that for a future improvement to the project. > > Ran the build and the output is working for me. This will help with testing, esp. integration tests with nifi. > > Thanks! at first, i put `dockermaven` project as a child of nifi-registry-docker (which is nifi-registry-core's child), but then i was unable to make sure that `nifi-registry-assembly` would execute prior to `dockermaven` (because `nifi-registry-core` must be built prior to `nifi-registry-assembly`), so i put it under the root directly, and specified `nifi-registry-docker-maven` later then `nifi-registry-assembly` in the root `pom.xml`'s module list, so this way i have the built artifacts ready when starting the docker build. i could have moved the existing `nifi-registry-docker` to the root to have docker things in one place, but i didn't want to break any external infra which may expect `nifi-registry-docker` at the currrent location This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi-registry] ekovacs commented on a change in pull request #245: NIFIREG-252: adding mavendocker build profile
ekovacs commented on a change in pull request #245: NIFIREG-252: adding mavendocker build profile URL: https://github.com/apache/nifi-registry/pull/245#discussion_r338448518 ## File path: nifi-registry-docker-maven/dockermaven/pom.xml ## @@ -0,0 +1,101 @@ + + +http://maven.apache.org/POM/4.0.0; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd;> +4.0.0 + + +nifi-registry-docker-maven +org.apache.nifi.registry +1.0.0-SNAPSHOT + +1.0.0-SNAPSHOT + +dockermaven + + + +docker + + + +com.spotify +dockerfile-maven-plugin +1.3.5 + + +default + +build + + + +1000 +1000 + ${project.version} + target/nifi-registry-${project.version}-bin.zip + + apache/nifi-registry +${project.version}-dockermaven + + + + + + +maven-antrun-plugin +1.8 + + +copy-for-docker +process-sources + + + + + + + + + + +run + + + + + + +exec-maven-plugin +org.codehaus.mojo + + +Docker integration tests +integration-test + +exec + + + + ${project.version}-dockermaven +${project.version} + + ${project.basedir}/integration-test.sh + + + + Review comment: updated PR with _skipping_ on `-DskipTests` now this will be printed, if `skipTests` is set: ``` [INFO] --- exec-maven-plugin:1.3.2:exec (Docker integration tests) @ dockermaven --- [INFO] skipping execute as per configuraion ``` also, please note that the tests will be only run, when `integration-test` or later maven lifecycle phase (eg.: `verify`) is specified. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Reopened] (NIFI-6792) Evaluate Variable value containing expression language
[ https://issues.apache.org/jira/browse/NIFI-6792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Mamidyala reopened NIFI-6792: --- > Evaluate Variable value containing expression language > -- > > Key: NIFI-6792 > URL: https://issues.apache.org/jira/browse/NIFI-6792 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework, Documentation Website >Affects Versions: 1.10.0 >Reporter: Manoj Mamidyala >Assignee: Manoj Mamidyala >Priority: Minor > Labels: Development, expression-language, feature > Original Estimate: 48h > Time Spent: 10m > Remaining Estimate: 47h 50m > > Evaluate Variable value containing expression language > I loaded custom properties into nifi. > The registry variable called query has value "SELECT * FROM table WHERE id = > ${id_no}" where the id_no is coming from the flowfile attributes. There was > no functionality in nifi to evaluate the expression language inside the value. > So, need to add functionality such that we can evaluate EL inside the > variables. > example: ${query:stringEl} > where function `stringEl` will evaluate the EL value inside the query variable -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (NIFI-6792) Evaluate Variable value containing expression language
[ https://issues.apache.org/jira/browse/NIFI-6792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Mamidyala resolved NIFI-6792. --- Resolution: Fixed added a function in Expression Language to Evaluate Variable value containing expression Language > Evaluate Variable value containing expression language > -- > > Key: NIFI-6792 > URL: https://issues.apache.org/jira/browse/NIFI-6792 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework, Documentation Website >Affects Versions: 1.10.0 >Reporter: Manoj Mamidyala >Assignee: Manoj Mamidyala >Priority: Minor > Labels: Development, expression-language, feature > Original Estimate: 48h > Time Spent: 10m > Remaining Estimate: 47h 50m > > Evaluate Variable value containing expression language > I loaded custom properties into nifi. > The registry variable called query has value "SELECT * FROM table WHERE id = > ${id_no}" where the id_no is coming from the flowfile attributes. There was > no functionality in nifi to evaluate the expression language inside the value. > So, need to add functionality such that we can evaluate EL inside the > variables. > example: ${query:stringEl} > where function `stringEl` will evaluate the EL value inside the query variable -- This message was sent by Atlassian Jira (v8.3.4#803005)