[jira] [Comment Edited] (NIFI-8522) NiFi can duplicate controller services when generating templates
[ https://issues.apache.org/jira/browse/NIFI-8522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17343008#comment-17343008 ] Hsin-Ying Lee edited comment on NIFI-8522 at 5/12/21, 4:45 AM: --- Hi [~msclarke] I tried to reproduce the error in 1.11.4 and the least main branch but I didn't get the exception in step 6 do you get any error logs in nifi-app.log? was (Author: s9514171): Hi [~msclarke] I tried to reproduce the error but I didn't get the exception in step 6 do you get any error logs in nifi-app.log? > NiFi can duplicate controller services when generating templates > > > Key: NIFI-8522 > URL: https://issues.apache.org/jira/browse/NIFI-8522 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.9.0, 1.10.0, 1.11.4, 1.12.1 >Reporter: Matthew Clarke >Priority: Major > > NiFi throws "The specified observer identifier already exists" exception when > trying to instantiate a template to the canvas. Parsing the template xml > reveals multiple controller services with same UUID. > Steps to reproduce: > 1. Create "PG-level1" > 2. within "PG-level1" create two new sub process groups "PG-level2-A" and > "PG-level2-B". > 3. Create a controller service(s) in "PG-level2-A" like AvroReader and > CSVRecordSetWriter. Add a processor like ConvertRecord and configure it to > reference these two controller services. > 4. Right click on the ConvertRecord processor and copy it. Exit > "PG-level2-A" and enter "PG-level2-B" and paste that copy processor. The > processor will show both configured controller services as "Incompatible > Controller Service Configured" and invalid bulletin will reflect the > properties reference controller services that do not exist. > 5. go up to "PG-level1" level and generate a template. Template is created > successfully. > 6. Try to instantiate this template to the canvas and it will only partially > instantiate and throw the "observer identifier already exist exception. > Inspection of the template shows that the controller services scoped to > PG-level2-A were added into the template again at the parent "PG-level1" > scope with same UUID. > The expected behavior is that the controller services do not get duplicated > and the processor in the instantiate template either has no configured > controller services or still reflects that it references controller > service(s) that do not exist. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-8522) NiFi can duplicate controller services when generating templates
[ https://issues.apache.org/jira/browse/NIFI-8522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17343008#comment-17343008 ] Hsin-Ying Lee commented on NIFI-8522: - Hi [~msclarke] I tried to reproduce the error but I didn't get the exception in step 6 do you get any error logs in nifi-app.log? > NiFi can duplicate controller services when generating templates > > > Key: NIFI-8522 > URL: https://issues.apache.org/jira/browse/NIFI-8522 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.9.0, 1.10.0, 1.11.4, 1.12.1 >Reporter: Matthew Clarke >Priority: Major > > NiFi throws "The specified observer identifier already exists" exception when > trying to instantiate a template to the canvas. Parsing the template xml > reveals multiple controller services with same UUID. > Steps to reproduce: > 1. Create "PG-level1" > 2. within "PG-level1" create two new sub process groups "PG-level2-A" and > "PG-level2-B". > 3. Create a controller service(s) in "PG-level2-A" like AvroReader and > CSVRecordSetWriter. Add a processor like ConvertRecord and configure it to > reference these two controller services. > 4. Right click on the ConvertRecord processor and copy it. Exit > "PG-level2-A" and enter "PG-level2-B" and paste that copy processor. The > processor will show both configured controller services as "Incompatible > Controller Service Configured" and invalid bulletin will reflect the > properties reference controller services that do not exist. > 5. go up to "PG-level1" level and generate a template. Template is created > successfully. > 6. Try to instantiate this template to the canvas and it will only partially > instantiate and throw the "observer identifier already exist exception. > Inspection of the template shows that the controller services scoped to > PG-level2-A were added into the template again at the parent "PG-level1" > scope with same UUID. > The expected behavior is that the controller services do not get duplicated > and the processor in the instantiate template either has no configured > controller services or still reflects that it references controller > service(s) that do not exist. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-8533) Replace jython-shaded with jython-slim because jython-shaded project is obsolete
[ https://issues.apache.org/jira/browse/NIFI-8533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lindongdong updated NIFI-8533: -- Description: As jython-shaded is deparecated, so I suggest to replace jython-shaded with jython-slim [https://github.com/scijava/jython-shaded] !image-2021-05-12-09-30-36-729.png|width=589,height=311! was: As jython-shaded is deparecated, so [https://github.com/scijava/jython-shaded] !image-2021-05-12-09-30-36-729.png|width=589,height=311! > Replace jython-shaded with jython-slim because jython-shaded project is > obsolete > > > Key: NIFI-8533 > URL: https://issues.apache.org/jira/browse/NIFI-8533 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 1.11.4, 1.12.1, 1.13.2 >Reporter: lindongdong >Priority: Major > Attachments: image-2021-05-12-09-30-36-729.png > > > As jython-shaded is deparecated, so I suggest to replace jython-shaded with > jython-slim > [https://github.com/scijava/jython-shaded] > !image-2021-05-12-09-30-36-729.png|width=589,height=311! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-8533) Replace jython-shaded with jython-slim because jython-shaded project is obsolete
[ https://issues.apache.org/jira/browse/NIFI-8533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lindongdong updated NIFI-8533: -- Description: As jython-shaded is deparecated, so [https://github.com/scijava/jython-shaded] !image-2021-05-12-09-30-36-729.png|width=589,height=311! was: [https://github.com/scijava/jython-shaded] !image-2021-05-12-09-30-36-729.png|width=589,height=311! > Replace jython-shaded with jython-slim because jython-shaded project is > obsolete > > > Key: NIFI-8533 > URL: https://issues.apache.org/jira/browse/NIFI-8533 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 1.11.4, 1.12.1, 1.13.2 >Reporter: lindongdong >Priority: Major > Attachments: image-2021-05-12-09-30-36-729.png > > > As jython-shaded is deparecated, so > [https://github.com/scijava/jython-shaded] > !image-2021-05-12-09-30-36-729.png|width=589,height=311! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-8533) Replace jython-shaded with jython-slim because jython-shaded project is obsolete
[ https://issues.apache.org/jira/browse/NIFI-8533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lindongdong updated NIFI-8533: -- Description: [https://github.com/scijava/jython-shaded] !image-2021-05-12-09-30-36-729.png|width=589,height=311! was: [https://github.com/scijava/jython-shaded] !image-2021-05-12-09-30-36-729.png! > Replace jython-shaded with jython-slim because jython-shaded project is > obsolete > > > Key: NIFI-8533 > URL: https://issues.apache.org/jira/browse/NIFI-8533 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 1.11.4, 1.12.1, 1.13.2 >Reporter: lindongdong >Priority: Major > Attachments: image-2021-05-12-09-30-36-729.png > > > [https://github.com/scijava/jython-shaded] > > !image-2021-05-12-09-30-36-729.png|width=589,height=311! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (NIFI-8533) Replace jython-shaded with jython-slim because jython-shaded project is obsolete
lindongdong created NIFI-8533: - Summary: Replace jython-shaded with jython-slim because jython-shaded project is obsolete Key: NIFI-8533 URL: https://issues.apache.org/jira/browse/NIFI-8533 Project: Apache NiFi Issue Type: Improvement Components: Core Framework Affects Versions: 1.13.2, 1.12.1, 1.11.4 Reporter: lindongdong Attachments: image-2021-05-12-09-30-36-729.png [https://github.com/scijava/jython-shaded] !image-2021-05-12-09-30-36-729.png! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] github-actions[bot] commented on pull request #2269: NIFI-4400 - Advanced UI with code editor for...
github-actions[bot] commented on pull request #2269: URL: https://github.com/apache/nifi/pull/2269#issuecomment-839308782 We're marking this PR as stale due to lack of updates in the past few months. If after another couple of weeks the stale label has not been removed this PR will be closed. This stale marker and eventual auto close does not indicate a judgement of the PR just lack of reviewer bandwidth and helps us keep the PR queue more manageable. If you would like this PR re-opened you can do so and a committer can remove the stale tag. Or you can open a new PR. Try to help review other PRs to increase PR review bandwidth which in turn helps yours. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] github-actions[bot] commented on pull request #2157: NIFI-4390 - Add a keyboard shortcut for Connection...
github-actions[bot] commented on pull request #2157: URL: https://github.com/apache/nifi/pull/2157#issuecomment-839308915 We're marking this PR as stale due to lack of updates in the past few months. If after another couple of weeks the stale label has not been removed this PR will be closed. This stale marker and eventual auto close does not indicate a judgement of the PR just lack of reviewer bandwidth and helps us keep the PR queue more manageable. If you would like this PR re-opened you can do so and a committer can remove the stale tag. Or you can open a new PR. Try to help review other PRs to increase PR review bandwidth which in turn helps yours. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] github-actions[bot] commented on pull request #2286: NIFI-4630 Added Satori RTM processor bundle (ConsumeSatoriRTM, PublishSatoriRTM)
github-actions[bot] commented on pull request #2286: URL: https://github.com/apache/nifi/pull/2286#issuecomment-839308751 We're marking this PR as stale due to lack of updates in the past few months. If after another couple of weeks the stale label has not been removed this PR will be closed. This stale marker and eventual auto close does not indicate a judgement of the PR just lack of reviewer bandwidth and helps us keep the PR queue more manageable. If you would like this PR re-opened you can do so and a committer can remove the stale tag. Or you can open a new PR. Try to help review other PRs to increase PR review bandwidth which in turn helps yours. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] github-actions[bot] commented on pull request #2066: NIFI-4256 - Add support for all AWS S3 Encryption Options
github-actions[bot] commented on pull request #2066: URL: https://github.com/apache/nifi/pull/2066#issuecomment-839309087 We're marking this PR as stale due to lack of updates in the past few months. If after another couple of weeks the stale label has not been removed this PR will be closed. This stale marker and eventual auto close does not indicate a judgement of the PR just lack of reviewer bandwidth and helps us keep the PR queue more manageable. If you would like this PR re-opened you can do so and a committer can remove the stale tag. Or you can open a new PR. Try to help review other PRs to increase PR review bandwidth which in turn helps yours. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] github-actions[bot] commented on pull request #2425: Emit failures array
github-actions[bot] commented on pull request #2425: URL: https://github.com/apache/nifi/pull/2425#issuecomment-839308536 We're marking this PR as stale due to lack of updates in the past few months. If after another couple of weeks the stale label has not been removed this PR will be closed. This stale marker and eventual auto close does not indicate a judgement of the PR just lack of reviewer bandwidth and helps us keep the PR queue more manageable. If you would like this PR re-opened you can do so and a committer can remove the stale tag. Or you can open a new PR. Try to help review other PRs to increase PR review bandwidth which in turn helps yours. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] github-actions[bot] commented on pull request #2241: NIFI-4463 New MQTT Consume logic
github-actions[bot] commented on pull request #2241: URL: https://github.com/apache/nifi/pull/2241#issuecomment-839308813 We're marking this PR as stale due to lack of updates in the past few months. If after another couple of weeks the stale label has not been removed this PR will be closed. This stale marker and eventual auto close does not indicate a judgement of the PR just lack of reviewer bandwidth and helps us keep the PR queue more manageable. If you would like this PR re-opened you can do so and a committer can remove the stale tag. Or you can open a new PR. Try to help review other PRs to increase PR review bandwidth which in turn helps yours. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] github-actions[bot] commented on pull request #2105: NIFI-4169: Enhance PutWebSocket error handling
github-actions[bot] commented on pull request #2105: URL: https://github.com/apache/nifi/pull/2105#issuecomment-839309053 We're marking this PR as stale due to lack of updates in the past few months. If after another couple of weeks the stale label has not been removed this PR will be closed. This stale marker and eventual auto close does not indicate a judgement of the PR just lack of reviewer bandwidth and helps us keep the PR queue more manageable. If you would like this PR re-opened you can do so and a committer can remove the stale tag. Or you can open a new PR. Try to help review other PRs to increase PR review bandwidth which in turn helps yours. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] github-actions[bot] commented on pull request #2132: NIFI-4359 Based on field node type, whether value node or not, fetching the value of field.
github-actions[bot] commented on pull request #2132: URL: https://github.com/apache/nifi/pull/2132#issuecomment-839309006 We're marking this PR as stale due to lack of updates in the past few months. If after another couple of weeks the stale label has not been removed this PR will be closed. This stale marker and eventual auto close does not indicate a judgement of the PR just lack of reviewer bandwidth and helps us keep the PR queue more manageable. If you would like this PR re-opened you can do so and a committer can remove the stale tag. Or you can open a new PR. Try to help review other PRs to increase PR review bandwidth which in turn helps yours. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] github-actions[bot] commented on pull request #1016: NIFI-2724 New JMX Processor
github-actions[bot] commented on pull request #1016: URL: https://github.com/apache/nifi/pull/1016#issuecomment-839309204 We're marking this PR as stale due to lack of updates in the past few months. If after another couple of weeks the stale label has not been removed this PR will be closed. This stale marker and eventual auto close does not indicate a judgement of the PR just lack of reviewer bandwidth and helps us keep the PR queue more manageable. If you would like this PR re-opened you can do so and a committer can remove the stale tag. Or you can open a new PR. Try to help review other PRs to increase PR review bandwidth which in turn helps yours. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] github-actions[bot] commented on pull request #2138: NIFI-4371 - add support for query timeout in Hive processors
github-actions[bot] commented on pull request #2138: URL: https://github.com/apache/nifi/pull/2138#issuecomment-839308974 We're marking this PR as stale due to lack of updates in the past few months. If after another couple of weeks the stale label has not been removed this PR will be closed. This stale marker and eventual auto close does not indicate a judgement of the PR just lack of reviewer bandwidth and helps us keep the PR queue more manageable. If you would like this PR re-opened you can do so and a committer can remove the stale tag. Or you can open a new PR. Try to help review other PRs to increase PR review bandwidth which in turn helps yours. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] github-actions[bot] commented on pull request #1403: NIFI-3293 - Expose counters to reporting tasks, and send counters dat…
github-actions[bot] commented on pull request #1403: URL: https://github.com/apache/nifi/pull/1403#issuecomment-839309168 We're marking this PR as stale due to lack of updates in the past few months. If after another couple of weeks the stale label has not been removed this PR will be closed. This stale marker and eventual auto close does not indicate a judgement of the PR just lack of reviewer bandwidth and helps us keep the PR queue more manageable. If you would like this PR re-opened you can do so and a committer can remove the stale tag. Or you can open a new PR. Try to help review other PRs to increase PR review bandwidth which in turn helps yours. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] github-actions[bot] commented on pull request #2154: NIFI-2663: Add WebSocket support for MQTT processors
github-actions[bot] commented on pull request #2154: URL: https://github.com/apache/nifi/pull/2154#issuecomment-839308941 We're marking this PR as stale due to lack of updates in the past few months. If after another couple of weeks the stale label has not been removed this PR will be closed. This stale marker and eventual auto close does not indicate a judgement of the PR just lack of reviewer bandwidth and helps us keep the PR queue more manageable. If you would like this PR re-opened you can do so and a committer can remove the stale tag. Or you can open a new PR. Try to help review other PRs to increase PR review bandwidth which in turn helps yours. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] github-actions[bot] commented on pull request #1541: NIFI-329 - Introduce IRC Client Services and ConsumeIRC processor
github-actions[bot] commented on pull request #1541: URL: https://github.com/apache/nifi/pull/1541#issuecomment-839309136 We're marking this PR as stale due to lack of updates in the past few months. If after another couple of weeks the stale label has not been removed this PR will be closed. This stale marker and eventual auto close does not indicate a judgement of the PR just lack of reviewer bandwidth and helps us keep the PR queue more manageable. If you would like this PR re-opened you can do so and a committer can remove the stale tag. Or you can open a new PR. Try to help review other PRs to increase PR review bandwidth which in turn helps yours. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] github-actions[bot] commented on pull request #2158: NIFI-4360 Adding support for ADLS Processors. Feature includes List, …
github-actions[bot] commented on pull request #2158: URL: https://github.com/apache/nifi/pull/2158#issuecomment-839308879 We're marking this PR as stale due to lack of updates in the past few months. If after another couple of weeks the stale label has not been removed this PR will be closed. This stale marker and eventual auto close does not indicate a judgement of the PR just lack of reviewer bandwidth and helps us keep the PR queue more manageable. If you would like this PR re-opened you can do so and a committer can remove the stale tag. Or you can open a new PR. Try to help review other PRs to increase PR review bandwidth which in turn helps yours. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] github-actions[bot] commented on pull request #2031: NIFI-617 - Message destination option in MonitorActivity
github-actions[bot] commented on pull request #2031: URL: https://github.com/apache/nifi/pull/2031#issuecomment-839309107 We're marking this PR as stale due to lack of updates in the past few months. If after another couple of weeks the stale label has not been removed this PR will be closed. This stale marker and eventual auto close does not indicate a judgement of the PR just lack of reviewer bandwidth and helps us keep the PR queue more manageable. If you would like this PR re-opened you can do so and a committer can remove the stale tag. Or you can open a new PR. Try to help review other PRs to increase PR review bandwidth which in turn helps yours. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] github-actions[bot] commented on pull request #2222: NIFI-3926 - Edit Template information
github-actions[bot] commented on pull request #: URL: https://github.com/apache/nifi/pull/#issuecomment-839308849 We're marking this PR as stale due to lack of updates in the past few months. If after another couple of weeks the stale label has not been removed this PR will be closed. This stale marker and eventual auto close does not indicate a judgement of the PR just lack of reviewer bandwidth and helps us keep the PR queue more manageable. If you would like this PR re-opened you can do so and a committer can remove the stale tag. Or you can open a new PR. Try to help review other PRs to increase PR review bandwidth which in turn helps yours. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] github-actions[bot] closed pull request #4430: NIFI-7677 - Upgrade Azure Storage API to latest (v12) for enabling identity authentication
github-actions[bot] closed pull request #4430: URL: https://github.com/apache/nifi/pull/4430 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] github-actions[bot] commented on pull request #2336: NIFI-4688 - PutParquet should have RemoteOwner and RemoteGroup LE(Language Expression) turn to TRUE
github-actions[bot] commented on pull request #2336: URL: https://github.com/apache/nifi/pull/2336#issuecomment-839308687 We're marking this PR as stale due to lack of updates in the past few months. If after another couple of weeks the stale label has not been removed this PR will be closed. This stale marker and eventual auto close does not indicate a judgement of the PR just lack of reviewer bandwidth and helps us keep the PR queue more manageable. If you would like this PR re-opened you can do so and a committer can remove the stale tag. Or you can open a new PR. Try to help review other PRs to increase PR review bandwidth which in turn helps yours. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] github-actions[bot] commented on pull request #2502: NIFI-4165: Added RemoveFlowFilesWithMissingContent.java and associate…
github-actions[bot] commented on pull request #2502: URL: https://github.com/apache/nifi/pull/2502#issuecomment-839308453 We're marking this PR as stale due to lack of updates in the past few months. If after another couple of weeks the stale label has not been removed this PR will be closed. This stale marker and eventual auto close does not indicate a judgement of the PR just lack of reviewer bandwidth and helps us keep the PR queue more manageable. If you would like this PR re-opened you can do so and a committer can remove the stale tag. Or you can open a new PR. Try to help review other PRs to increase PR review bandwidth which in turn helps yours. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] github-actions[bot] commented on pull request #2411: NIFI-4789 Extract grok multi pattern support
github-actions[bot] commented on pull request #2411: URL: https://github.com/apache/nifi/pull/2411#issuecomment-839308585 We're marking this PR as stale due to lack of updates in the past few months. If after another couple of weeks the stale label has not been removed this PR will be closed. This stale marker and eventual auto close does not indicate a judgement of the PR just lack of reviewer bandwidth and helps us keep the PR queue more manageable. If you would like this PR re-opened you can do so and a committer can remove the stale tag. Or you can open a new PR. Try to help review other PRs to increase PR review bandwidth which in turn helps yours. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] github-actions[bot] closed pull request #4440: NIFI-528 add support to specify timeout in ExecuteProcess
github-actions[bot] closed pull request #4440: URL: https://github.com/apache/nifi/pull/4440 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] github-actions[bot] closed pull request #4453: NIFI-7710 - Flow XML validation warnings for DataSize
github-actions[bot] closed pull request #4453: URL: https://github.com/apache/nifi/pull/4453 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] github-actions[bot] commented on pull request #2375: autoclosesocket
github-actions[bot] commented on pull request #2375: URL: https://github.com/apache/nifi/pull/2375#issuecomment-839308657 We're marking this PR as stale due to lack of updates in the past few months. If after another couple of weeks the stale label has not been removed this PR will be closed. This stale marker and eventual auto close does not indicate a judgement of the PR just lack of reviewer bandwidth and helps us keep the PR queue more manageable. If you would like this PR re-opened you can do so and a committer can remove the stale tag. Or you can open a new PR. Try to help review other PRs to increase PR review bandwidth which in turn helps yours. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] github-actions[bot] commented on pull request #2287: NIFI-4625 - Added External Version to the PutElastic5 Processor. Added Testing around the version usage.
github-actions[bot] commented on pull request #2287: URL: https://github.com/apache/nifi/pull/2287#issuecomment-839308717 We're marking this PR as stale due to lack of updates in the past few months. If after another couple of weeks the stale label has not been removed this PR will be closed. This stale marker and eventual auto close does not indicate a judgement of the PR just lack of reviewer bandwidth and helps us keep the PR queue more manageable. If you would like this PR re-opened you can do so and a committer can remove the stale tag. Or you can open a new PR. Try to help review other PRs to increase PR review bandwidth which in turn helps yours. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] github-actions[bot] commented on pull request #2408: localization using the JSTL standard fmt tag for multilingualization and nf._.msg() function in resource.js
github-actions[bot] commented on pull request #2408: URL: https://github.com/apache/nifi/pull/2408#issuecomment-839308627 We're marking this PR as stale due to lack of updates in the past few months. If after another couple of weeks the stale label has not been removed this PR will be closed. This stale marker and eventual auto close does not indicate a judgement of the PR just lack of reviewer bandwidth and helps us keep the PR queue more manageable. If you would like this PR re-opened you can do so and a committer can remove the stale tag. Or you can open a new PR. Try to help review other PRs to increase PR review bandwidth which in turn helps yours. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] github-actions[bot] commented on pull request #2467: NIFI-4862 Add Copy original attributtes to SelectHiveQL processer
github-actions[bot] commented on pull request #2467: URL: https://github.com/apache/nifi/pull/2467#issuecomment-839308487 We're marking this PR as stale due to lack of updates in the past few months. If after another couple of weeks the stale label has not been removed this PR will be closed. This stale marker and eventual auto close does not indicate a judgement of the PR just lack of reviewer bandwidth and helps us keep the PR queue more manageable. If you would like this PR re-opened you can do so and a committer can remove the stale tag. Or you can open a new PR. Try to help review other PRs to increase PR review bandwidth which in turn helps yours. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] Lehel44 commented on a change in pull request #5028: NIFI-8325: Complete SNMP refactor: SNMP GET and SET processors rework, unit tests added
Lehel44 commented on a change in pull request #5028:
URL: https://github.com/apache/nifi/pull/5028#discussion_r630576988
##
File path:
nifi-nar-bundles/nifi-snmp-bundle/nifi-snmp-processors/src/main/resources/docs/org.apache.nifi.snmp.processors.GetSNMP/additionalDetails.html
##
@@ -44,27 +44,5 @@ SNMP Properties
In case of a single SNMP Get request, the following is the list of
available standard SNMP properties which may come with the PDU:
("snmp$errorIndex", "snmp$errorStatus", "snmp$errorStatusText",
"snmp$nonRepeaters", "snmp$requestID", "snmp$type")
-Configuration Details
Review comment:
All the information can already be found here in the property
descriptions. I think it's unnecessary to maintain the same list here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] Lehel44 commented on a change in pull request #5028: NIFI-8325: Complete SNMP refactor: SNMP GET and SET processors rework, unit tests added
Lehel44 commented on a change in pull request #5028:
URL: https://github.com/apache/nifi/pull/5028#discussion_r630574932
##
File path:
nifi-nar-bundles/nifi-snmp-bundle/nifi-snmp-processors/src/main/java/org/apache/nifi/snmp/factory/AbstractSNMPFactory.java
##
@@ -0,0 +1,91 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.snmp.factory;
+
+import org.apache.nifi.snmp.configuration.SNMPConfiguration;
+import org.apache.nifi.snmp.exception.CreateSNMPClientException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.snmp4j.CommunityTarget;
+import org.snmp4j.Snmp;
+import org.snmp4j.Target;
+import org.snmp4j.UserTarget;
+import org.snmp4j.security.SecurityLevel;
+import org.snmp4j.smi.OctetString;
+import org.snmp4j.smi.UdpAddress;
+import org.snmp4j.transport.DefaultUdpTransportMapping;
+
+import java.io.IOException;
+import java.util.Optional;
+
+public abstract class AbstractSNMPFactory {
+
+private static final Logger logger =
LoggerFactory.getLogger(AbstractSNMPFactory.class);
+
+protected AbstractSNMPFactory() {
+// hide implicit constructor
+}
+
+protected static Snmp createSimpleSnmpClient() {
Review comment:
I removed the simple from the signature. @turcsanyip recommended using
create instead of get because its created when called.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] Lehel44 commented on a change in pull request #5028: NIFI-8325: Complete SNMP refactor: SNMP GET and SET processors rework, unit tests added
Lehel44 commented on a change in pull request #5028:
URL: https://github.com/apache/nifi/pull/5028#discussion_r630573717
##
File path:
nifi-nar-bundles/nifi-snmp-bundle/nifi-snmp-processors/src/main/java/org/apache/nifi/snmp/dto/SNMPResponse.java
##
@@ -0,0 +1,66 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.snmp.dto;
+
+import org.apache.nifi.snmp.utils.SNMPUtils;
+import org.snmp4j.PDU;
+import org.snmp4j.Target;
+
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+public class SNMPResponse {
+
+private final Map attributes;
+private final List variableBindings;
+private final int errorStatus;
+private final String errorStatusText;
+private final String targetAddress;
+
+public SNMPResponse(final PDU responsePdu, final Target target) {
+attributes = SNMPUtils.getPduAttributeMap(responsePdu);
Review comment:
I extract information on the fly when SNMP Get is being called here:
[line](https://github.com/apache/nifi/blob/e121973906f72899354011fdeeb48538fba99a94/nifi-nar-bundles/nifi-snmp-bundle/nifi-snmp-processors/src/main/java/org/apache/nifi/snmp/operations/StandardSNMPRequestHandler.java#L72)
We extract the information only once, why do you think we should store it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (NIFI-8470) PutDatabaseRecord truncates strings to 1 character when inserting into PostgreSQL CHAR(n) field
[ https://issues.apache.org/jira/browse/NIFI-8470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17342887#comment-17342887 ] ASF subversion and git services commented on NIFI-8470: --- Commit fdda3582e3f60da74d82713e8677a1560d04a008 in nifi's branch refs/heads/main from Sandro Tolaini [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=fdda358 ] NIFI-8470: Fix DataTypeUtils for CHAR() sequences NIFI-8470: Added unit test Signed-off-by: Matthew Burgess This closes #5027 > PutDatabaseRecord truncates strings to 1 character when inserting into > PostgreSQL CHAR(n) field > --- > > Key: NIFI-8470 > URL: https://issues.apache.org/jira/browse/NIFI-8470 > Project: Apache NiFi > Issue Type: Bug > Environment: Linux CentOS 7.9, PostgreSQL 13, OpenJDK 1.8.0 >Reporter: Sandro Tolaini >Priority: Major > Fix For: 1.14.0 > > Time Spent: 2h > Remaining Estimate: 0h > > When inserting records into a CHAR(5) column in a PostgreSQL table, the value > is truncated to first character -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-8470) PutDatabaseRecord truncates strings to 1 character when inserting into PostgreSQL CHAR(n) field
[ https://issues.apache.org/jira/browse/NIFI-8470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-8470: --- Fix Version/s: 1.14.0 Resolution: Fixed Status: Resolved (was: Patch Available) > PutDatabaseRecord truncates strings to 1 character when inserting into > PostgreSQL CHAR(n) field > --- > > Key: NIFI-8470 > URL: https://issues.apache.org/jira/browse/NIFI-8470 > Project: Apache NiFi > Issue Type: Bug > Environment: Linux CentOS 7.9, PostgreSQL 13, OpenJDK 1.8.0 >Reporter: Sandro Tolaini >Priority: Major > Fix For: 1.14.0 > > Time Spent: 2h > Remaining Estimate: 0h > > When inserting records into a CHAR(5) column in a PostgreSQL table, the value > is truncated to first character -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] mattyb149 closed pull request #5027: NIFI-8470: PutDatabaseRecord truncates strings to 1 character for CHAR(n) field
mattyb149 closed pull request #5027: URL: https://github.com/apache/nifi/pull/5027 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-8470) PutDatabaseRecord truncates strings to 1 character when inserting into PostgreSQL CHAR(n) field
[ https://issues.apache.org/jira/browse/NIFI-8470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17342886#comment-17342886 ] ASF subversion and git services commented on NIFI-8470: --- Commit fdda3582e3f60da74d82713e8677a1560d04a008 in nifi's branch refs/heads/main from Sandro Tolaini [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=fdda358 ] NIFI-8470: Fix DataTypeUtils for CHAR() sequences NIFI-8470: Added unit test Signed-off-by: Matthew Burgess This closes #5027 > PutDatabaseRecord truncates strings to 1 character when inserting into > PostgreSQL CHAR(n) field > --- > > Key: NIFI-8470 > URL: https://issues.apache.org/jira/browse/NIFI-8470 > Project: Apache NiFi > Issue Type: Bug > Environment: Linux CentOS 7.9, PostgreSQL 13, OpenJDK 1.8.0 >Reporter: Sandro Tolaini >Priority: Major > Time Spent: 2h > Remaining Estimate: 0h > > When inserting records into a CHAR(5) column in a PostgreSQL table, the value > is truncated to first character -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] mattyb149 commented on pull request #5027: NIFI-8470: PutDatabaseRecord truncates strings to 1 character for CHAR(n) field
mattyb149 commented on pull request #5027: URL: https://github.com/apache/nifi/pull/5027#issuecomment-839201965 +1 LGTM, tried on Oracle, PostgreSQL and MySQL, all looks good. Thanks for the fix! Merging to main -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-8330) JythonScriptEngineConfigurator needs to recompile on init()
[ https://issues.apache.org/jira/browse/NIFI-8330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-8330: - Resolution: Fixed Status: Resolved (was: Patch Available) > JythonScriptEngineConfigurator needs to recompile on init() > --- > > Key: NIFI-8330 > URL: https://issues.apache.org/jira/browse/NIFI-8330 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Major > Fix For: 1.14.0 > > Time Spent: 1h > Remaining Estimate: 0h > > NIFI-8080 introduced a kind of regression in the sense that an invalid Jython > script can be supplied to InvokeScriptedProcessor (or related scripting > components) and it will successfully "compile". However it will generate an > error every time eval() is called, and will not be recompiled if the script > is changed because a CompiledScript object exists. The workaround/approach is > that the script must be corrected and then NiFi must be restarted. > The JythonScriptEngineConfigurator should recompile the script whenever > init() is called, whether there is an existing CompiledScript object or not. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-8330) JythonScriptEngineConfigurator needs to recompile on init()
[ https://issues.apache.org/jira/browse/NIFI-8330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17342865#comment-17342865 ] ASF subversion and git services commented on NIFI-8330: --- Commit a4d435c9952f306f748fa004fc839eba318f6187 in nifi's branch refs/heads/main from Matt Burgess [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=a4d435c ] NIFI-8330: Fix compilation of Jython scripts for scripting components - Add onPropertyModified to ExecuteScript to recompile on change This closes #4903. Signed-off-by: Mark Payne > JythonScriptEngineConfigurator needs to recompile on init() > --- > > Key: NIFI-8330 > URL: https://issues.apache.org/jira/browse/NIFI-8330 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Major > Fix For: 1.14.0 > > Time Spent: 1h > Remaining Estimate: 0h > > NIFI-8080 introduced a kind of regression in the sense that an invalid Jython > script can be supplied to InvokeScriptedProcessor (or related scripting > components) and it will successfully "compile". However it will generate an > error every time eval() is called, and will not be recompiled if the script > is changed because a CompiledScript object exists. The workaround/approach is > that the script must be corrected and then NiFi must be restarted. > The JythonScriptEngineConfigurator should recompile the script whenever > init() is called, whether there is an existing CompiledScript object or not. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] markap14 commented on pull request #4903: NIFI-8330: Fix compilation of Jython scripts for scripting components
markap14 commented on pull request #4903: URL: https://github.com/apache/nifi/pull/4903#issuecomment-839157299 Thanks @mattyb149, looking much better now! +1 merged to main! Thanks for knocking that out, definitely an important fix! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] markap14 closed pull request #4903: NIFI-8330: Fix compilation of Jython scripts for scripting components
markap14 closed pull request #4903: URL: https://github.com/apache/nifi/pull/4903 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] markap14 commented on a change in pull request #4780: NIFI-8126 Include Total Queued Duration in ConnectionStatus metrics
markap14 commented on a change in pull request #4780:
URL: https://github.com/apache/nifi/pull/4780#discussion_r630464003
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/queue/SwappablePriorityQueue.java
##
@@ -917,10 +954,81 @@ public SwapSummary recoverSwappedFlowFiles() {
logger.info("Recovered {} swap files for {} in {} millis",
swapLocations.size() - failures, this, millis);
}
-return new StandardSwapSummary(new QueueSize(swapFlowFileCount,
swapByteCount), maxId, resourceClaims);
+// minSwappedQueueDate and totalSwappedQueueDate within this
particular StandardSwapSummary are not ultimately used by the FlowController.
However,
+// it can't hurt to set them here accurately in case they ever are.
+return new StandardSwapSummary(new QueueSize(swapFlowFileCount,
swapByteCount), maxId, resourceClaims, minSwappedQueueDate,
totalSwappedQueueDate);
}
+public long getMinLastQueueDate() {
+readLock.lock();
+try {
+// We want the oldest timestamp, which will be the min
+boolean seen = false;
+Long min = null;
+for (FlowFileRecord flowFileRecord : activeQueue) {
+Long lastQueueDate = flowFileRecord.getLastQueueDate();
+if (lastQueueDate != null) {
+if (!seen || lastQueueDate.compareTo(min) < 0) {
+seen = true;
+min = lastQueueDate;
+}
+}
+}
+
+for (FlowFileRecord flowFileRecord : swapQueue) {
+Long lastQueueDate = flowFileRecord.getLastQueueDate();
+if (lastQueueDate != null) {
+if (!seen || lastQueueDate.compareTo(min) < 0) {
+seen = true;
+min = lastQueueDate;
+}
+}
+}
+
+for(Long minSwapQueueDate: minQueueDateInSwapLocation.values()) {
+if(!seen || minSwapQueueDate.compareTo(min) < 0) {
+seen = true;
+min = minSwapQueueDate;
+}
+}
+return seen ? min : 0L;
+} finally {
+readLock.unlock("Get Min Last Queue Date");
+}
+}
+
+public long getTotalQueuedDuration(long fromTimestamp) {
+readLock.lock();
+try {
+long sum = 0L;
+for (FlowFileRecord flowFileRecord : activeQueue) {
+if ((flowFileRecord.getLastQueueDate() != null)) {
Review comment:
getLastQueueDate() is guaranteed non-null here.
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/queue/SwappablePriorityQueue.java
##
@@ -917,10 +954,81 @@ public SwapSummary recoverSwappedFlowFiles() {
logger.info("Recovered {} swap files for {} in {} millis",
swapLocations.size() - failures, this, millis);
}
-return new StandardSwapSummary(new QueueSize(swapFlowFileCount,
swapByteCount), maxId, resourceClaims);
+// minSwappedQueueDate and totalSwappedQueueDate within this
particular StandardSwapSummary are not ultimately used by the FlowController.
However,
+// it can't hurt to set them here accurately in case they ever are.
+return new StandardSwapSummary(new QueueSize(swapFlowFileCount,
swapByteCount), maxId, resourceClaims, minSwappedQueueDate,
totalSwappedQueueDate);
}
+public long getMinLastQueueDate() {
+readLock.lock();
+try {
+// We want the oldest timestamp, which will be the min
+boolean seen = false;
Review comment:
I find the logic here a bit confusing. I think it can be simplified
quite a bit by using:
```
long min = 0;
for (FlowFileRecord flowFileRecord : activeQueue) {
long lastQueueDate = flowFileRecord.getLastQueueDate();
if (min == 0 || lastQueueDate < min) {
min = lastQueueDate;
}
}
```
And given that the next part of the code repeats the same logic, probably
makes sense to extract out a method that takes an Iterable and
returns the minimum. Then it could be as simple as:
```
long min = getOldestQueueDate(activeQueue);
min = Math.min(min, getOldestQueueDate(swapQueue);
for (Long minSwapQueueDate : minQueueDateInSwapLocation.values()) {
min = Math.min(min, minSwapQueueDate);
}
return min;
```
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/queue/clustered/SocketLoadBalancedFlowFileQueue.java
##
@@ -513,6 +525,30 @@ public QueueSize size() {
return totalSize.get();
}
+@Override
+public long
[GitHub] [nifi] tolaini commented on pull request #5027: NIFI-8470: PutDatabaseRecord truncates strings to 1 character for CHAR(n) field
tolaini commented on pull request #5027: URL: https://github.com/apache/nifi/pull/5027#issuecomment-839122863 @mattyb149 now the commit should be ok, please let me know if you can proceed with review/merge. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] ChrisSamo632 commented on pull request #4822: NIFI-2892 Implement AWS Kinesis Stream Get Processor
ChrisSamo632 commented on pull request #4822: URL: https://github.com/apache/nifi/pull/4822#issuecomment-839077520 @turcsanyip I've implemented a check in `onTrigger` to ensure the `WorkerState` never reaches `SHUT_DOWN` without the `ConsumeKinesisStream` processor being Stopped in NiFi - this is the only way I can see of trying to check whether the Worker has failed to initialise (it would remain at `CREATED` or `INITIALIZING` state until finally shutting down due to a configuration error). Unit test added to show the behaviour (happily the unit tests don't connect to a Kinesis/DynamoDB instance so the Worker always fails eventually, now we throw a `ProcessException` to note that in the NiFi UI and allow a Bulletin to be generated). Newer KCL versions allow for better handling of this scenario, but we're limited to when we can do in KCL 1.13.3 (see NIFI-8531 for upgrading to KCL 2.x and a note about improving this scenario). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-registry] cavemandaveman opened a new pull request #322: NIFIREG-453 Add ability to set DatabaseFlowPersistenceProvider in Docker
cavemandaveman opened a new pull request #322:
URL: https://github.com/apache/nifi-registry/pull/322
Description of PR
Enables DatabaseFlowPersistenceProvider to be set when using Docker.
### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [x] Does your PR title start with **NIFIREG-** where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [x] Has your PR been rebased against the latest commit within the target
branch (typically `main`)?
- [x] Is your initial contribution a single, squashed commit? _Additional
commits in response to PR reviewer feedback should be made on this branch and
pushed to allow change tracking. Do not `squash` or use `--force` when pushing
to allow for clean monitoring of changes._
### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via `mvn
-Pcontrib-check clean install` at the root `nifi-registry` folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] Have you verified that the full build is successful on JDK 8?
- [ ] Have you verified that the full build is successful on JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main
`LICENSE` file under `nifi-registry-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main
`NOTICE` file found under `nifi-registry-assembly`?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI for
build issues and submit an update to your PR as soon as possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Created] (NIFIREG-453) Add ability to set DatabaseFlowPersistenceProvider in Docker
cavemandaveman created NIFIREG-453: -- Summary: Add ability to set DatabaseFlowPersistenceProvider in Docker Key: NIFIREG-453 URL: https://issues.apache.org/jira/browse/NIFIREG-453 Project: NiFi Registry Issue Type: New Feature Reporter: cavemandaveman Currently the only flow providers that are able to be set in Docker are FileSystemFlowPersistenceProvider and GitFlowPersistenceProvider. DatabaseFlowPersistenceProvider is not currently an option. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (NIFI-8532) ConsumeKinesisStream tuning/performance testing
Chris Sampson created NIFI-8532:
---
Summary: ConsumeKinesisStream tuning/performance testing
Key: NIFI-8532
URL: https://issues.apache.org/jira/browse/NIFI-8532
Project: Apache NiFi
Issue Type: Improvement
Reporter: Chris Sampson
{{ConsumeKinesisStream}} was added in NIFI-2982 with some seemingly sensible
default property settings that are passed to the Kinesis Client Library.
Some performance testing/experimentation should be conducted for different NiFi
setups (e.g. single instance, remote instance, cluster) to determine whether
other settings should be used by default.
For example (from comments in PR
[#4822|https://github.com/apache/nifi/pull/4822]):
{quote}
[~turcsanyip] did some kind of performance tests with 1 Mio messages in 12
shards and running NiFi on [his] local machine (no cluster). It took 64-74
seconds with default settings (no additional dynamic properties).
The interesting thing that [he] could not see any difference when I set
Checkpoint Interval to 0 sec (that is checkpointing "synchronously" after each
bunch of messages received in IRecordProcessor.processRecords() callback). It
seems there is no significant overhead of checkpointing more frequently (and it
has the advantage of having fewer duplicated messages in case of restart).
It can be investigated further in a follow-up ticket with more sophisticated
performance tests (nifi cluster, non-local machine, tuning KCL properties, etc)
and the default can be adjusted if that is reasonable.
{quote}
(this is *that* ticket)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi] ChrisSamo632 commented on a change in pull request #4822: NIFI-2892 Implement AWS Kinesis Stream Get Processor
ChrisSamo632 commented on a change in pull request #4822:
URL: https://github.com/apache/nifi/pull/4822#discussion_r630459168
##
File path:
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/kinesis/stream/ConsumeKinesisStream.java
##
@@ -0,0 +1,704 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.aws.kinesis.stream;
+
+import
com.amazonaws.services.kinesis.clientlibrary.interfaces.v2.IRecordProcessorFactory;
+import
com.amazonaws.services.kinesis.clientlibrary.lib.worker.InitialPositionInStream;
+import
com.amazonaws.services.kinesis.clientlibrary.lib.worker.KinesisClientLibConfiguration;
+import com.amazonaws.services.kinesis.clientlibrary.lib.worker.Worker;
+import com.amazonaws.services.kinesis.metrics.impl.NullMetricsFactory;
+import org.apache.commons.beanutils.BeanUtilsBean;
+import org.apache.commons.beanutils.ConvertUtilsBean2;
+import org.apache.commons.beanutils.FluentPropertyBeanIntrospector;
+import org.apache.commons.beanutils.PropertyUtilsBean;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.behavior.DynamicProperties;
+import org.apache.nifi.annotation.behavior.DynamicProperty;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SystemResource;
+import org.apache.nifi.annotation.behavior.SystemResourceConsideration;
+import org.apache.nifi.annotation.behavior.TriggerSerially;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.SeeAlso;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.context.PropertyContext;
+import org.apache.nifi.expression.ExpressionLanguageScope;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessSessionFactory;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.processors.aws.AbstractAWSCredentialsProviderProcessor;
+import
org.apache.nifi.processors.aws.kinesis.stream.record.AbstractKinesisRecordProcessor;
+import
org.apache.nifi.processors.aws.kinesis.stream.record.KinesisRecordProcessorRaw;
+import
org.apache.nifi.processors.aws.kinesis.stream.record.KinesisRecordProcessorRecord;
+import org.apache.nifi.serialization.RecordReaderFactory;
+import org.apache.nifi.serialization.RecordSetWriterFactory;
+import org.apache.nifi.serialization.record.RecordFieldType;
+
+import java.lang.reflect.InvocationTargetException;
+import java.net.InetAddress;
+import java.net.UnknownHostException;
+import java.time.LocalDateTime;
+import java.time.ZoneId;
+import java.time.ZoneOffset;
+import java.time.format.DateTimeFormatter;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.Date;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import java.util.Set;
+import java.util.UUID;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.Future;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+import java.util.stream.Collectors;
+
+@InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN)
+@TriggerSerially
+@Tags({"amazon", "aws", "kinesis", "consume", "stream"})
+@CapabilityDescription("Reads data from the specified AWS Kinesis stream and
outputs a FlowFile for every processed Record (raw) " +
+" or a FlowFile for a batch of processed records if a Record Reader
and Record Writer are configured. " +
+
[jira] [Created] (NIFI-8531) Upgrade Kinesis Client Library to 2.x
Chris Sampson created NIFI-8531:
---
Summary: Upgrade Kinesis Client Library to 2.x
Key: NIFI-8531
URL: https://issues.apache.org/jira/browse/NIFI-8531
Project: Apache NiFi
Issue Type: Improvement
Reporter: Chris Sampson
The Kinesis Client Library [KCL] version 1.x is used by the
{{ConsumeKinesisStream}} processor (NIFI-2892).
It would be better to upgrade to KCL 2.x as it is more up-to-date and provides
more advanced features.
For example, the {{WorkerStateChangeListener}} in 2.x has an
{{onAllInitializationAttemptsFailed}} method that could be used to better track
{{Worker}} startup issues (instead of relying on checking for unexpected
{{WorkerState}} values in our processor's {{onTrigger}} method).
There are also more configuration options available that would be worth
considering as processor properties/noting in the processor documentation.
N.B. upgrading to KCL 2.x relies upon upgrading to AWS Java SDK 2.x first
(NIFI-8287).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi] mattyb149 commented on pull request #5027: NIFI-8470: PutDatabaseRecord truncates strings to 1 character for CHAR(n) field
mattyb149 commented on pull request #5027: URL: https://github.com/apache/nifi/pull/5027#issuecomment-839014787 The commit message should have the Jira case and a description of the problem fixed (not what file(s) changed), something like: `NIFI-8470: Fix DataTypeUtils for CHAR() sequences` You can do this by the following commands on your branch: ``` git reset --soft HEAD^ git commit -m "NIFI-8470: Fix DataTypeUtils for CHAR() sequences" git push -f origin NIFI-8470 ``` Then I can approve the GitHub Actions run and review/merge. Thank you for your contribution! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] mattyb149 closed pull request #5026: NIFI-8470
mattyb149 closed pull request #5026: URL: https://github.com/apache/nifi/pull/5026 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] martinzink commented on a change in pull request #1066: MINIFICPP-1532: Create a processor to capture resource consumption me…
martinzink commented on a change in pull request #1066:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1066#discussion_r630440856
##
File path: libminifi/test/unit/MemoryUsageTest.cpp
##
@@ -47,3 +47,16 @@ TEST_CASE("Test new and legacy total system memory query
equivalency") {
REQUIRE(GetTotalMemoryLegacy() ==
utils::OsUtils::getSystemTotalPhysicalMemory());
}
#endif
+
+
+#ifdef WIN32
+TEST_CASE("Test Paging file size", "[testpagingfile]") {
+ constexpr bool cout_enabled = true;
+ const auto total_paging_file_size = utils::OsUtils::getTotalPagingFileSize();
+
+ if (cout_enabled) {
+std::cout << "Total Paging file size: " << total_paging_file_size << "
bytes" << std::endl;
Review comment:
I wanted these tests to be able to print out these hardware/platform
specific informations so it can be verified manually on various platforms if
needed without bothoring with log files.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (NIFI-8367) Remove unnecessary use of Joda-Time
[ https://issues.apache.org/jira/browse/NIFI-8367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17342804#comment-17342804 ] David Handermann commented on NIFI-8367: References to joda-time can also be removed for nifi-standard-content-viewer after Avro is upgraded to a more recent version that no longer relies on Joda Time. > Remove unnecessary use of Joda-Time > --- > > Key: NIFI-8367 > URL: https://issues.apache.org/jira/browse/NIFI-8367 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.13.2 >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Time Spent: 0.5h > Remaining Estimate: 0h > > A small number of classes use the [Joda-Time|https://www.joda.org/joda-time/] > library for date and time operations. These references should be replaced > with JSR-310 java.time classes so that the joda-time dependency can be > removed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-8367) Remove unnecessary use of Joda-Time
[ https://issues.apache.org/jira/browse/NIFI-8367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17342803#comment-17342803 ] ASF subversion and git services commented on NIFI-8367: --- Commit 340b13033f1012a2de2cec582b019ad0e01ead4a in nifi's branch refs/heads/main from David Handermann [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=340b130 ] NIFI-8367 Removed joda-time from nifi-standard-processors > Remove unnecessary use of Joda-Time > --- > > Key: NIFI-8367 > URL: https://issues.apache.org/jira/browse/NIFI-8367 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.13.2 >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > A small number of classes use the [Joda-Time|https://www.joda.org/joda-time/] > library for date and time operations. These references should be replaced > with JSR-310 java.time classes so that the joda-time dependency can be > removed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] markap14 merged pull request #5067: NIFI-8367 Removed joda-time from nifi-standard-processors
markap14 merged pull request #5067: URL: https://github.com/apache/nifi/pull/5067 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] markap14 commented on pull request #5067: NIFI-8367 Removed joda-time from nifi-standard-processors
markap14 commented on pull request #5067: URL: https://github.com/apache/nifi/pull/5067#issuecomment-838993109 Thanks for catching that @exceptionfactory. Looks like all still builds with all tests passing. +1 will merge to main -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-6685) Sort and Align Process Groups automatically
[ https://issues.apache.org/jira/browse/NIFI-6685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-6685: - Summary: Sort and Align Process Groups automatically (was: Sort and aling) > Sort and Align Process Groups automatically > --- > > Key: NIFI-6685 > URL: https://issues.apache.org/jira/browse/NIFI-6685 > Project: Apache NiFi > Issue Type: Wish > Components: Core UI >Reporter: Nitin Ullal >Assignee: Guillaume Schaer >Priority: Major > Attachments: nifi.png > > > It would be nice to have a feature that sorts and aligns Process Groups by > their name and aligns them to a grid (for example 3x3). > Check out attached screenshot. Every time I add something I have to manually > sort and align. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-8518) Include edge case for JSON keys that contain whitespace in EvaluateJsonPath documentation
[
https://issues.apache.org/jira/browse/NIFI-8518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17342796#comment-17342796
]
ASF subversion and git services commented on NIFI-8518:
---
Commit 86d5b08d232e54e5af0beac1fcc6c1509b0b303e in nifi's branch
refs/heads/main from matthewlannan
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=86d5b08 ]
Adding addititonal documentation to EvalluateJsonPath per
https://issues.apache.org/jira/browse/NIFI-8518
> Include edge case for JSON keys that contain whitespace in EvaluateJsonPath
> documentation
> -
>
> Key: NIFI-8518
> URL: https://issues.apache.org/jira/browse/NIFI-8518
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Documentation Website
>Affects Versions: 1.13.2
> Environment: Mac, BigSur 11.2.3, 32MB RAM, 8 Core Intel i9, 1 TB
> storage
>Reporter: Matthew Lannan
>Priority: Minor
> Labels: documentation, easyfix
> Fix For: 1.14.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> I would say this is borderline buggish - since
> {code:java}
> "bike color":{code}
> is a valid JSON key. However, EvaluateJsonPath will say
> {code:java}
> $.bike color{code}
> is not valid.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (NIFI-8518) Include edge case for JSON keys that contain whitespace in EvaluateJsonPath documentation
[
https://issues.apache.org/jira/browse/NIFI-8518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Payne resolved NIFI-8518.
--
Fix Version/s: 1.14.0
Resolution: Fixed
> Include edge case for JSON keys that contain whitespace in EvaluateJsonPath
> documentation
> -
>
> Key: NIFI-8518
> URL: https://issues.apache.org/jira/browse/NIFI-8518
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Documentation Website
>Affects Versions: 1.13.2
> Environment: Mac, BigSur 11.2.3, 32MB RAM, 8 Core Intel i9, 1 TB
> storage
>Reporter: Matthew Lannan
>Priority: Minor
> Labels: documentation, easyfix
> Fix For: 1.14.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> I would say this is borderline buggish - since
> {code:java}
> "bike color":{code}
> is a valid JSON key. However, EvaluateJsonPath will say
> {code:java}
> $.bike color{code}
> is not valid.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi] markap14 commented on pull request #5058: NIFI-8518
markap14 commented on pull request #5058: URL: https://github.com/apache/nifi/pull/5058#issuecomment-838977051 Thanks @mlannan. Changes look good to me. +1 merged to main. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] markap14 merged pull request #5058: NIFI-8518
markap14 merged pull request #5058: URL: https://github.com/apache/nifi/pull/5058 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on pull request #5065: NIFI-8528 Migrate NiFi Registry into NiFi codebase
exceptionfactory commented on pull request #5065: URL: https://github.com/apache/nifi/pull/5065#issuecomment-838950445 > @exceptionfactory > > > It may need to be a separate task, but the increased build time raises the question of more granular builds so that changes to NiFi Registry only result in building NiFi Registry. Of course changes to core components will necessitate complete builds. What do you think? > > Yeah, the increased build time really feels to be a genuine issue. However I'm afraid we would want at least one full clean build no matter what, just to be sure. > > Maybe it would work the other way around - _don't build registry_ if it hasn't change. > The reason being nifi depends on registry, not the other way around. So if the registry changes, we will need to build nifi for sure. Maybe this is not that apparent yet but there are plans to move some artifacts from registry to the _nifi-api_. > > But I'm not sure what would be the best course of action. Thanks for the reply @tpalfy. I agree that getting a successful build of your PR with Registry across all three environments sounds good. Perhaps a follow-on issue to update the GitHub workflow configuration, including both timeout adjustments and selective builds, is the best way to go. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-8518) Include edge case for JSON keys that contain whitespace in EvaluateJsonPath documentation
[
https://issues.apache.org/jira/browse/NIFI-8518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matthew Lannan updated NIFI-8518:
-
Summary: Include edge case for JSON keys that contain whitespace in
EvaluateJsonPath documentation (was: Include edge case for JSON keys with
whitespace in EvaluateJsonPath documentation)
> Include edge case for JSON keys that contain whitespace in EvaluateJsonPath
> documentation
> -
>
> Key: NIFI-8518
> URL: https://issues.apache.org/jira/browse/NIFI-8518
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Documentation Website
>Affects Versions: 1.13.2
> Environment: Mac, BigSur 11.2.3, 32MB RAM, 8 Core Intel i9, 1 TB
> storage
>Reporter: Matthew Lannan
>Priority: Minor
> Labels: documentation, easyfix
>
> I would say this is borderline buggish - since
> {code:java}
> "bike color":{code}
> is a valid JSON key. However, EvaluateJsonPath will say
> {code:java}
> $.bike color{code}
> is not valid.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (NIFI-8220) Establish a secure by default configuration for NiFi
[ https://issues.apache.org/jira/browse/NIFI-8220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17342777#comment-17342777 ] David Handermann commented on NIFI-8220: Sub-task NIFI-8516 brings together work from the other sub-tasks and addresses points 1, 2, and 3 from the description. The Single User Authorizer implemented in sub-task NIFI-8363 allows the the authenticated user to perform all actions and does not restrict access to any components. Regarding point 4, [~joewitt], did you have a particular approach in mind? One option could be updating the Single User Authorizer to disallow access to restricted processors. Updating the configuration to use the standard file-based and managed authorizer would be one way require explicit changes to enable restricted components. Or do you think point 4 is necessary as part of implementing this epic? > Establish a secure by default configuration for NiFi > > > Key: NIFI-8220 > URL: https://issues.apache.org/jira/browse/NIFI-8220 > Project: Apache NiFi > Issue Type: Epic > Components: Tools and Build >Reporter: Joe Witt >Assignee: Joe Witt >Priority: Blocker > Fix For: 1.14.0 > > > Inspired by this tweet > https://twitter.com/_escctrl_/status/1359280656174510081?s=21 and the > resulting discussion here > https://lists.apache.org/thread.html/rc590f21807192a0dce18293c2d5b47392a6fd8a1ef26d77fbd6ee695%40%3Cdev.nifi.apache.org%3E > It is time to change our config model. It was also setup to be easy to use. > We've seen these silly setups on the Internet before but has gotten > ridiculous. We need to take action. > Will create a set of one or more JIRAs to roughly do the following. > 1. Disable HTTP by default. If a user wants to enable to it for whatever > reason then also make them enable a new property which says something to the > effect of 'allow completely non secure access to the entire nifi instance - > not recommended' > 2. Enable HTTPS with one way authentication by default which would be the > client authenticating the server whereby the server has a server cert. We > could either make that cert a self-signed (and thus not trusted by client's > by default) cert or give a way for the user to run through command line > process to make a legit cert. > 3. If not already configured with an authorization provider supply and out of > the box provider which supports only a single auto generated at first startup > user/password enabling access to the NiFi system. > 4. Disable all restricted processors by default. Require the user to > explicitly enable them. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-8516) Set Default HTTPS and Single User Authentication
[ https://issues.apache.org/jira/browse/NIFI-8516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-8516: --- Status: Patch Available (was: Open) > Set Default HTTPS and Single User Authentication > > > Key: NIFI-8516 > URL: https://issues.apache.org/jira/browse/NIFI-8516 > Project: Apache NiFi > Issue Type: Sub-task >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Labels: HTTPS, security > Time Spent: 10m > Remaining Estimate: 0h > > Building on the implementation of single user authentication in NIFI-8363 and > self-signed certificate generation in NIFI-8403, the default configuration > should be updated to enable these features. > This change should disable default access over unencrypted and > unauthenticated channels. The project README should be updated to reflect the > default behavior with instructions on retrieving the generated username and > password from application logs. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] exceptionfactory opened a new pull request #5068: NIFI-8516 Enabled HTTPS and Single User Authentication by default
exceptionfactory opened a new pull request #5068:
URL: https://github.com/apache/nifi/pull/5068
Description of PR
NIFI-8516 Implements the stated goals of NIFI-8220 providing a basic layer
of security for the default installation of NiFi.
The default configuration disables HTTP on port 8080 and enables HTTPS on
port 9443 using a self-signed certificate generated during the bootstrap
process. The default configuration also leverages the Single User Login
Identity Provider and Authorizer to generate a random username and password for
authentication. The NiFi README.md includes updated instructions and
screenshots describing how to access NiFi using the generated credentials.
These changes will not impact existing installations of NiFi following an
upgrade. Neither the self-signed certificate nor the generated username and
password are intended for production deployments. The purpose of these default
configuration changes is to provide a greater level of security for default
installations of NiFi while maintain a straightforward experience for new users.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [X] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [X] Does your PR title start with **NIFI-** where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [X] Has your PR been rebased against the latest commit within the target
branch (typically `main`)?
- [X] Is your initial contribution a single, squashed commit? _Additional
commits in response to PR reviewer feedback should be made on this branch and
pushed to allow change tracking. Do not `squash` or use `--force` when pushing
to allow for clean monitoring of changes._
### For code changes:
- [X] Have you ensured that the full suite of tests is executed via `mvn
-Pcontrib-check clean install` at the root `nifi` folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [X] Have you verified that the full build is successful on JDK 8?
- [ ] Have you verified that the full build is successful on JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main
`LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main
`NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [X] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI for
build issues and submit an update to your PR as soon as possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] tpalfy commented on pull request #5065: NIFI-8528 Migrate NiFi Registry into NiFi codebase
tpalfy commented on pull request #5065: URL: https://github.com/apache/nifi/pull/5065#issuecomment-838844759 @exceptionfactory > It may need to be a separate task, but the increased build time raises the question of more granular builds so that changes to NiFi Registry only result in building NiFi Registry. Of course changes to core components will necessitate complete builds. What do you think? Yeah, the increased build time really feels to be a genuine issue. However I'm afraid we would want at least one full clean build no matter what, just to be sure. Maybe it would work the other way around - _don't build registry_ if it hasn't change. The reason being nifi depends on registry, not the other way around. So if the registry changes, we will need to build nifi for sure. Maybe this is not that apparent yet but there are plans to move some artifacts from registry to the _nifi-api_. But I'm not sure what would be the best course of action. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] fgerlits commented on a change in pull request #1066: MINIFICPP-1532: Create a processor to capture resource consumption me…
fgerlits commented on a change in pull request #1066:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1066#discussion_r630356615
##
File path: extensions/pdh/tests/PerformanceDataMonitorTests.cpp
##
@@ -0,0 +1,268 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include
+#include
+#include
+#include
+#include
+
+#include "TestBase.h"
+#include "processors/PutFile.h"
+#include "utils/file/FileUtils.h"
+#include "PerformanceDataMonitor.h"
+#include "rapidjson/filereadstream.h"
+
+using PutFile = org::apache::nifi::minifi::processors::PutFile;
+using PerformanceDataMonitor =
org::apache::nifi::minifi::processors::PerformanceDataMonitor;
+using PerformanceDataCounter =
org::apache::nifi::minifi::processors::PerformanceDataCounter;
+
+class PerformanceDataMonitorTester {
+ public:
+ PerformanceDataMonitorTester() {
+LogTestController::getInstance().setTrace();
+dir_ = test_controller_.createTempDirectory("/tmp/gt.XX");
Review comment:
This really shouldn't compile (I don't know why VS allows it), as the
parameter is a non-const `char*`; the input string even gets modified. Please
use `minifi::utils::createTempDir(_controller_)` instead.
##
File path: extensions/pdh/tests/PerformanceDataCounterTests.cpp
##
@@ -0,0 +1,185 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "TestBase.h"
+#include "PDHCounters.h"
+#include "MemoryConsumptionCounter.h"
+
+using PDHCounter = org::apache::nifi::minifi::processors::PDHCounter;
+using PDHCounterArray = org::apache::nifi::minifi::processors::PDHCounterArray;
+using PDHCounterBase = org::apache::nifi::minifi::processors::PDHCounterBase;
+using MemoryConsumptionCounter =
org::apache::nifi::minifi::processors::MemoryConsumptionCounter;
+
+
+TEST_CASE("PDHCounterNameTests", "[pdhcounternametests]") {
+ PDHCounterBase* test_counter =
PDHCounterBase::createPDHCounter("\\System\\Threads");
+ REQUIRE(nullptr != dynamic_cast (test_counter));
+ REQUIRE("\\System\\Threads" == test_counter->getName());
+ REQUIRE("System" == test_counter->getObjectName());
+ REQUIRE("Threads" == test_counter->getCounterName());
+}
+
+TEST_CASE("PDHCounterArrayNameTests", "[pdhcounterarraytests]") {
+ PDHCounterBase* test_counter_array =
PDHCounterBase::createPDHCounter("\\LogicalDisk(*)\\% Free Space");
+ REQUIRE(nullptr != dynamic_cast (test_counter_array));
+ REQUIRE("\\LogicalDisk(*)\\% Free Space" == test_counter_array->getName());
+ REQUIRE("LogicalDisk" == test_counter_array->getObjectName());
+ REQUIRE("% Free Space" == test_counter_array->getCounterName());
+}
+
+TEST_CASE("PDHCountersInvalidNameTests", "[pdhcountersinvalidnametests]") {
+ REQUIRE(nullptr == PDHCounterBase::createPDHCounter("Invalid Name"));
+ REQUIRE(nullptr == PDHCounterBase::createPDHCounter(""));
+ REQUIRE(nullptr == PDHCounterBase::createPDHCounter("Something\\Counter"));
+ REQUIRE(nullptr ==
PDHCounterBase::createPDHCounter("\\Too\\Many\\Separators"));
+ REQUIRE(nullptr ==
PDHCounterBase::createPDHCounter("Too\\Many\\Separators"));
+ REQUIRE(nullptr != PDHCounterBase::createPDHCounter("\\Valid\\Counter"));
+}
+
+class TestablePDHCounter : public PDHCounter {
+ public:
+ explicit TestablePDHCounter(const std::string& query_name, bool is_double =
true) : PDHCounter(query_name, is_double) {
+ }
+};
+
+class TestablePDHCounterArray : public PDHCounterArray {
+ public:
+ explicit
[GitHub] [nifi] turcsanyip commented on a change in pull request #4822: NIFI-2892 Implement AWS Kinesis Stream Get Processor
turcsanyip commented on a change in pull request #4822:
URL: https://github.com/apache/nifi/pull/4822#discussion_r629948591
##
File path:
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/kinesis/stream/ConsumeKinesisStream.java
##
@@ -0,0 +1,704 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.aws.kinesis.stream;
+
+import
com.amazonaws.services.kinesis.clientlibrary.interfaces.v2.IRecordProcessorFactory;
+import
com.amazonaws.services.kinesis.clientlibrary.lib.worker.InitialPositionInStream;
+import
com.amazonaws.services.kinesis.clientlibrary.lib.worker.KinesisClientLibConfiguration;
+import com.amazonaws.services.kinesis.clientlibrary.lib.worker.Worker;
+import com.amazonaws.services.kinesis.metrics.impl.NullMetricsFactory;
+import org.apache.commons.beanutils.BeanUtilsBean;
+import org.apache.commons.beanutils.ConvertUtilsBean2;
+import org.apache.commons.beanutils.FluentPropertyBeanIntrospector;
+import org.apache.commons.beanutils.PropertyUtilsBean;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.behavior.DynamicProperties;
+import org.apache.nifi.annotation.behavior.DynamicProperty;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SystemResource;
+import org.apache.nifi.annotation.behavior.SystemResourceConsideration;
+import org.apache.nifi.annotation.behavior.TriggerSerially;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.SeeAlso;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.context.PropertyContext;
+import org.apache.nifi.expression.ExpressionLanguageScope;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessSessionFactory;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.processors.aws.AbstractAWSCredentialsProviderProcessor;
+import
org.apache.nifi.processors.aws.kinesis.stream.record.AbstractKinesisRecordProcessor;
+import
org.apache.nifi.processors.aws.kinesis.stream.record.KinesisRecordProcessorRaw;
+import
org.apache.nifi.processors.aws.kinesis.stream.record.KinesisRecordProcessorRecord;
+import org.apache.nifi.serialization.RecordReaderFactory;
+import org.apache.nifi.serialization.RecordSetWriterFactory;
+import org.apache.nifi.serialization.record.RecordFieldType;
+
+import java.lang.reflect.InvocationTargetException;
+import java.net.InetAddress;
+import java.net.UnknownHostException;
+import java.time.LocalDateTime;
+import java.time.ZoneId;
+import java.time.ZoneOffset;
+import java.time.format.DateTimeFormatter;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.Date;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import java.util.Set;
+import java.util.UUID;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.Future;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+import java.util.stream.Collectors;
+
+@InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN)
+@TriggerSerially
+@Tags({"amazon", "aws", "kinesis", "consume", "stream"})
+@CapabilityDescription("Reads data from the specified AWS Kinesis stream and
outputs a FlowFile for every processed Record (raw) " +
+" or a FlowFile for a batch of processed records if a Record Reader
and Record Writer are configured. " +
+"At-least-once
[GitHub] [nifi] markap14 commented on pull request #4903: NIFI-8330: Fix compilation of Jython scripts for scripting components
markap14 commented on pull request #4903: URL: https://github.com/apache/nifi/pull/4903#issuecomment-838816994 @mattyb149 I tried this out with the latest commit, and it didn't appear to solve the underlying problem. There's no `onPropertyModified` to indicate that the script needs to be recompiled. I created a simple dataflow: GenerateFlowFile -> ExecuteScript -> UpdateAttribute. Configured ExecuteScript with the following script: ``` flowfile = session.get() flowfile = session.putAttribute(flowfile, 'greeting', 'hello') session.transfer(flowfile, REL_SUCCESS) session.commit() ``` It appeared to run correctly. Then I stopped the Processor, and changed the script to: ``` flowfile = session.get() flowfile = session.putAttribute(flowfile, 'greeting', 'good-bye') session.transfer(flowfile, REL_SUCCESS) session.commit() ``` Ran it again. The output had an attribute of `greeting` with a value of `hello` - NOT `good-bye`. So after I changed the text of the script, it did not recompile. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] gresockj edited a comment on pull request #5065: NIFI-8528 Migrate NiFi Registry into NiFi codebase
gresockj edited a comment on pull request #5065: URL: https://github.com/apache/nifi/pull/5065#issuecomment-838739151 Verified build and deployment on MacOS. LGTM (Once the Windows build is fixed, that is) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] gresockj commented on pull request #5065: NIFI-8528 Migrate NiFi Registry into NiFi codebase
gresockj commented on pull request #5065: URL: https://github.com/apache/nifi/pull/5065#issuecomment-838739151 Verified build and deployment on MacOS. LGTM -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (NIFI-6685) Sort and aling
[ https://issues.apache.org/jira/browse/NIFI-6685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guillaume Schaer reassigned NIFI-6685: -- Assignee: Guillaume Schaer > Sort and aling > -- > > Key: NIFI-6685 > URL: https://issues.apache.org/jira/browse/NIFI-6685 > Project: Apache NiFi > Issue Type: Wish > Components: Core UI >Reporter: Nitin Ullal >Assignee: Guillaume Schaer >Priority: Major > Attachments: nifi.png > > > It would be nice to have a feature that sorts and aligns Process Groups by > their name and aligns them to a grid (for example 3x3). > Check out attached screenshot. Every time I add something I have to manually > sort and align. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] gschaer commented on pull request #5064: NIFI-8007: Setup background grid
gschaer commented on pull request #5064: URL: https://github.com/apache/nifi/pull/5064#issuecomment-838734495 > How difficult would it be to have 3 levels of grid display: off, major gridlines only, major and minor? Good point. Let me see what I can do. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-4913) Specifying "run.as" in bootstrap.properties causes environment variables to not be passed to NiFi
[ https://issues.apache.org/jira/browse/NIFI-4913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17342682#comment-17342682 ] ASF subversion and git services commented on NIFI-4913: --- Commit 48a9054731c2246be913e1e191b93e082f7ec0f8 in nifi's branch refs/heads/main from Sushil Kumar [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=48a9054 ] NIFI-4913 - Added "-E" with sudo to read environment variables set in current shell environment while running nifi with bootstrap property "run.as" set as to some user NIFI-4913 - Updated administration-guide.adoc Signed-off-by: Nathan Gough This closes #3750. > Specifying "run.as" in bootstrap.properties causes environment variables to > not be passed to NiFi > - > > Key: NIFI-4913 > URL: https://issues.apache.org/jira/browse/NIFI-4913 > Project: Apache NiFi > Issue Type: Bug > Components: Configuration >Affects Versions: 1.5.0 >Reporter: Jonathan Peterson >Assignee: Sushil Kumar >Priority: Minor > Labels: beginner > Time Spent: 1h 50m > Remaining Estimate: 0h > > I solved this by including "-E" after "sudo" in nifi.sh [1]. > [1] > [https://github.com/apache/nifi/blob/rel/nifi-1.5.0/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-resources/src/main/resources/bin/nifi.sh#L312] > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-4913) Specifying "run.as" in bootstrap.properties causes environment variables to not be passed to NiFi
[ https://issues.apache.org/jira/browse/NIFI-4913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17342681#comment-17342681 ] ASF subversion and git services commented on NIFI-4913: --- Commit 48a9054731c2246be913e1e191b93e082f7ec0f8 in nifi's branch refs/heads/main from Sushil Kumar [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=48a9054 ] NIFI-4913 - Added "-E" with sudo to read environment variables set in current shell environment while running nifi with bootstrap property "run.as" set as to some user NIFI-4913 - Updated administration-guide.adoc Signed-off-by: Nathan Gough This closes #3750. > Specifying "run.as" in bootstrap.properties causes environment variables to > not be passed to NiFi > - > > Key: NIFI-4913 > URL: https://issues.apache.org/jira/browse/NIFI-4913 > Project: Apache NiFi > Issue Type: Bug > Components: Configuration >Affects Versions: 1.5.0 >Reporter: Jonathan Peterson >Assignee: Sushil Kumar >Priority: Minor > Labels: beginner > Time Spent: 1h 50m > Remaining Estimate: 0h > > I solved this by including "-E" after "sudo" in nifi.sh [1]. > [1] > [https://github.com/apache/nifi/blob/rel/nifi-1.5.0/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-resources/src/main/resources/bin/nifi.sh#L312] > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] thenatog closed pull request #3750: NIFI-4913 Added preserve.environment to read environment variables while using run.as
thenatog closed pull request #3750: URL: https://github.com/apache/nifi/pull/3750 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] adam-markovics commented on a change in pull request #1066: MINIFICPP-1532: Create a processor to capture resource consumption me…
adam-markovics commented on a change in pull request #1066:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1066#discussion_r630147443
##
File path: libminifi/test/unit/MemoryUsageTest.cpp
##
@@ -20,22 +20,22 @@
#include "utils/OsUtils.h"
#include "../TestBase.h"
-TEST_CASE("Test memory usage", "[testmemoryusage]") {
+TEST_CASE("Test Physical memory usage", "[testphysicalmemoryusage]") {
constexpr bool cout_enabled = true;
Review comment:
What is the purpose of this constant?
##
File path: extensions/pdh/tests/PerformanceDataMonitorTests.cpp
##
@@ -0,0 +1,268 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include
+#include
+#include
+#include
+#include
+
+#include "TestBase.h"
+#include "processors/PutFile.h"
+#include "utils/file/FileUtils.h"
+#include "PerformanceDataMonitor.h"
+#include "rapidjson/filereadstream.h"
+
+using PutFile = org::apache::nifi::minifi::processors::PutFile;
Review comment:
It's simpler just like
`using org::apache::nifi::minifi::processors::PutFile;`
Also for the others.
##
File path: extensions/pdh/PDHCounters.h
##
@@ -0,0 +1,98 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#pragma once
+
+#include
+#include
+#include
+#include
+
+#include "PerformanceDataCounter.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace processors {
+
+class PDHCounterBase : public PerformanceDataCounter {
+ public:
+ virtual ~PDHCounterBase() {}
+ static PDHCounterBase* createPDHCounter(const std::string& query_name, bool
is_double = true);
+
+ const std::string& getName() const;
+ virtual std::string getObjectName() const;
+ virtual std::string getCounterName() const;
+ virtual PDH_STATUS addToQuery(PDH_HQUERY& pdh_query) = 0;
+ virtual PDH_STATUS collectData() = 0;
+ protected:
+ PDHCounterBase(const std::string& query_name, bool is_double) :
PerformanceDataCounter(),
+is_double_format_(is_double), pdh_english_counter_name_(query_name) {
+ }
+ DWORD getDWFormat() const;
+ const bool is_double_format_;
+ std::string pdh_english_counter_name_;
+};
+
+class PDHCounter : public PDHCounterBase {
+ public:
+ void addToJson(rapidjson::Value& body, rapidjson::Document::AllocatorType&
alloc) const override;
+
+ PDH_STATUS addToQuery(PDH_HQUERY& pdh_query) override;
+ PDH_STATUS collectData() override;
+
+ protected:
+ friend PDHCounterBase* PDHCounterBase::createPDHCounter(const std::string&
query_name, bool is_double);
+ explicit PDHCounter(const std::string& query_name, bool is_double) :
PDHCounterBase(query_name, is_double),
+counter_(), current_value_() {
+ }
+ rapidjson::Value getValue() const;
+
+ PDH_HCOUNTER counter_;
+ PDH_FMT_COUNTERVALUE current_value_;
+};
+
+class PDHCounterArray : public PDHCounterBase {
+ public:
+ virtual ~PDHCounterArray() { clearCurrentData(); }
+ void addToJson(rapidjson::Value& body, rapidjson::Document::AllocatorType&
alloc) const override;
+
+ std::string getObjectName() const override;
+ PDH_STATUS addToQuery(PDH_HQUERY& pdh_query) override;
+ PDH_STATUS collectData() override;
+
+ protected:
+ friend PDHCounterBase* PDHCounterBase::createPDHCounter(const std::string&
query_name, bool is_double);
+ explicit PDHCounterArray(std::string query_name, bool is_double) :
PDHCounterBase(query_name, is_double),
Review comment:
If you take query_name by value,
[GitHub] [nifi] markap14 commented on pull request #5059: NIFI-8519 Adding HDFS support for NAR autoload
markap14 commented on pull request #5059:
URL: https://github.com/apache/nifi/pull/5059#issuecomment-838659499
Thanks for the contribution @simonbence! Looking through this i think ideas
are sound. However, this is updating the `nifi-api` module, and once it's been
released it's permanent. So we need to be extremely careful about anything that
we put in there. Naming, Exceptions that should be thrown, and the correct
level of abstraction. So I have some thoughts around this.
- What this 'extension' is really doing has nothing to do with Auto-Loading.
It has to do with sourcing the NAR's from some other location. And given the
interface provided in the PR, things like `start()` and `stop()` don't make a
lot of sense to me, to be honest. When I see that, I feel like the component
should be starting some background thread or something maybe so that it can
poll? But then the background thread is created elsewhere. I think what would
be a more appropriate level of abstraction would be:
```
public interface NarProvider {
/**
* Initializes the NAR Provider based on the given set of properties
*/
void initialize(NarProviderInitializationContext context);
/**
* Performs a listing of all NAR's that are available.
*/
Collection listNars() throws IOException;
/**
* Fetches the NAR at the given location. The location should be one of
the values returned by listNars().
*/
InputStream fetchNarContents(String location) throws IOException;
}
```
And here, `NarProviderInitializlationContext` would perhaps just have a
single method:
```
Map getProperties();
```
But this abstraction gives us the freedom to update it in the future.
I also think it's important to consider how those properties get provided to
the implementation. The way that we typically handle something like this within
NiFi would be to have the framework look at `nifi.properties` and pull out the
properties appropriate for the instance. So, perhaps we would look for any
property that has the format:
```
nifi.library.nar.autoload..implementation
nifi.library.nar.autoload..XYZ
```
And then build up a Map where the key is the property name
after and the value is the property value. For example, if we have in
nifi.properties:
```
nifi.library.nar.autoload.hdfs.implementation=org.apache.nifi.hdfs.FetchNarHdfs
nifi.library.nar.autoload.hdfs.kerberos=username
nifi.library.nar.autoload.hdfs.hello=world
```
This would produce a Map that could be provided to the `initialize` method
(via the context):
kerberos => username
hello => world
(No need to include `implementation` necessarily because it's used by the
framework to know the name of the class to use but doesn't really need to be
passed to the class itself).
A few additional thoughts on the implementation:
- With this approach, we only need to have extensions for determining what's
available and downloading it. The Auto-Loader should not be pluggable, and it
should be responsible for calling out to each NarProvider to get a listing, and
then triggering the download of each NAR.
- It makes sense to support multiple Nar Providers, even multiple of the
same type. It's very reasonable to imagine an organization in which there are
multiple teams hosting nars in different locations, and a user may want to
fetch from multiple directories, etc. This would all work nicely with the above
property naming scheme.
- We should always include debug-level logging indicating which NAR's were
returned when performing the listing, and this should indicate how long it took
to perform that listing.
- Should log at an INFO level each time a NAR is downloaded and how long
that took
- NAR's should be written with a dot-name (`.my-bundle.nar`) and then
renamed so that the Extension Loader is able to detect it only when it's done.
It probably would make sense to download into a sub-directory of the
`extensions` directory. I.e., if the nar loader is configured to monitor the
`./extensions` directory we could create a directory `./extensions/hdfs` and
then update the Nar Loader to also scan that directory. This would make it
easier to understand where each of the NAR's came from if we have multiple
NarProviders.
- It's possible that over time users will continually add new versions of a
NAR to the same directory. And over time this will result in loading massive
amounts of NAR's into NiFi. Those that get removed will remain in NiFi. We will
need some way to age off old NARs. Perhaps after performing a listing, we
should detect any nar's that were previously downloaded from that NarProvider
and not present in the Listing. Then, if any of those is not in use (i.e., no
extensions are in the flow that use that bundle) then we can remove the bundle
and its NarLoader.
[GitHub] [nifi] markobean commented on pull request #4780: NIFI-8126 Include Total Queued Duration in ConnectionStatus metrics
markobean commented on pull request #4780: URL: https://github.com/apache/nifi/pull/4780#issuecomment-838653339 @markap14 we are going to be cutting a release of an internal fork, and we'd like to include this ticket. Do you have any additional comments? Or, if it looks good, could you please commit? Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] thenatog commented on pull request #3750: NIFI-4913 Added preserve.environment to read environment variables while using run.as
thenatog commented on pull request #3750: URL: https://github.com/apache/nifi/pull/3750#issuecomment-838616353 Great, looks good, +1 will merge. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-8367) Remove unnecessary use of Joda-Time
[ https://issues.apache.org/jira/browse/NIFI-8367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-8367: --- Status: Patch Available (was: In Progress) > Remove unnecessary use of Joda-Time > --- > > Key: NIFI-8367 > URL: https://issues.apache.org/jira/browse/NIFI-8367 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.13.2 >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > A small number of classes use the [Joda-Time|https://www.joda.org/joda-time/] > library for date and time operations. These references should be replaced > with JSR-310 java.time classes so that the joda-time dependency can be > removed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-8470) PutDatabaseRecord truncates strings to 1 character when inserting into PostgreSQL CHAR(n) field
[ https://issues.apache.org/jira/browse/NIFI-8470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-8470: --- Component/s: (was: NiFi Stateless) > PutDatabaseRecord truncates strings to 1 character when inserting into > PostgreSQL CHAR(n) field > --- > > Key: NIFI-8470 > URL: https://issues.apache.org/jira/browse/NIFI-8470 > Project: Apache NiFi > Issue Type: Bug > Environment: Linux CentOS 7.9, PostgreSQL 13, OpenJDK 1.8.0 >Reporter: Sandro Tolaini >Priority: Major > Time Spent: 1h 20m > Remaining Estimate: 0h > > When inserting records into a CHAR(5) column in a PostgreSQL table, the value > is truncated to first character -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-8470) PutDatabaseRecord truncates strings to 1 character when inserting into PostgreSQL CHAR(n) field
[ https://issues.apache.org/jira/browse/NIFI-8470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-8470: --- Affects Version/s: (was: 1.13.2) Status: Patch Available (was: Open) > PutDatabaseRecord truncates strings to 1 character when inserting into > PostgreSQL CHAR(n) field > --- > > Key: NIFI-8470 > URL: https://issues.apache.org/jira/browse/NIFI-8470 > Project: Apache NiFi > Issue Type: Bug > Components: NiFi Stateless > Environment: Linux CentOS 7.9, PostgreSQL 13, OpenJDK 1.8.0 >Reporter: Sandro Tolaini >Priority: Major > Time Spent: 1h 20m > Remaining Estimate: 0h > > When inserting records into a CHAR(5) column in a PostgreSQL table, the value > is truncated to first character -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] exceptionfactory opened a new pull request #5067: NIFI-8367 Removed joda-time from nifi-standard-processors
exceptionfactory opened a new pull request #5067:
URL: https://github.com/apache/nifi/pull/5067
Description of PR
NIFI-8367 Removes joda-time direct dependency from
`nifi-standard-processors`. Recent changes to `InvokeHTTP` removed the last
direct reference to Joda Time. The `nifi-standard-content-viewer` module still
relies on Joda Time for handling certain Avro fields and will remain necessary
until the Avro dependency is upgraded.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [X] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [X] Does your PR title start with **NIFI-** where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [X] Has your PR been rebased against the latest commit within the target
branch (typically `main`)?
- [X] Is your initial contribution a single, squashed commit? _Additional
commits in response to PR reviewer feedback should be made on this branch and
pushed to allow change tracking. Do not `squash` or use `--force` when pushing
to allow for clean monitoring of changes._
### For code changes:
- [X] Have you ensured that the full suite of tests is executed via `mvn
-Pcontrib-check clean install` at the root `nifi` folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [X] Have you verified that the full build is successful on JDK 8?
- [ ] Have you verified that the full build is successful on JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main
`LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main
`NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI for
build issues and submit an update to your PR as soon as possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (NIFI-1465) Upgrade encryption of sensitive properties
[
https://issues.apache.org/jira/browse/NIFI-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann updated NIFI-1465:
---
Status: Patch Available (was: In Progress)
> Upgrade encryption of sensitive properties
> --
>
> Key: NIFI-1465
> URL: https://issues.apache.org/jira/browse/NIFI-1465
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 0.5.0
>Reporter: Andy LoPresto
>Assignee: David Handermann
>Priority: Major
> Labels: encryption, security
> Fix For: 1.14.0
>
> Original Estimate: 120h
> Remaining Estimate: 120h
>
> Currently, NiFi accepts a password and encryption algorithm in
> {{nifi.properties}} which are used to encrypt all sensitive processor
> properties throughout the application. The password defaults to empty and the
> algorithm defaults to {{PBEWITHMD5AND256BITAES-CBC-OPENSSL}}. This algorithm:
> * uses a digest function ({{MD5}}) which is not cryptographically secure
> [1][2][3][4]
> * uses a single iteration count [5][6]
> * limits password input to 16 characters on JVMs without the unlimited
> strength cryptographic jurisdiction policy files installed [NIFI-1255]
> all of which combine to make it extremely insecure. We should change the
> default algorithm to use a strong key derivation function (KDF) [7] which
> will properly derive a key to protect the sensitive properties.
> Because existing systems have already encrypted the properties using a key
> derived from the original settings, we should provide a translation/upgrade
> utility to seamlessly convert the stored values from the old password &
> algorithm combination to the new.
> [1] http://security.stackexchange.com/a/19908/16485
> [2] http://security.stackexchange.com/a/31846/16485
> [3]
> http://security.stackexchange.com/questions/52461/how-weak-is-md5-as-a-password-hashing-function
> [4] http://security.stackexchange.com/a/31410/16485
> [5] http://security.stackexchange.com/a/29139/16485
> [6] https://www.openssl.org/docs/manmaster/crypto/EVP_BytesToKey.html
> [7]
> https://cwiki.apache.org/confluence/display/NIFI/Key+Derivation+Function+Explanations
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (NIFI-8417) emoji in processor group name cause nifi to not start up
[ https://issues.apache.org/jira/browse/NIFI-8417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann resolved NIFI-8417. Fix Version/s: 1.14.0 Assignee: Hsin-Ying Lee Resolution: Fixed > emoji in processor group name cause nifi to not start up > > > Key: NIFI-8417 > URL: https://issues.apache.org/jira/browse/NIFI-8417 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.13.0, 1.13.1, 1.13.2 > Environment: RHEL 7, jdk8 >Reporter: Adam Turley >Assignee: Hsin-Ying Lee >Priority: Blocker > Fix For: 1.14.0 > > Attachments: Screen Shot 2021-04-12 at 9.17.29 AM.png > > Time Spent: 1h 40m > Remaining Estimate: 0h > > if a processor group includes an emoji in the name, nifi will work normally > until restart. Once restarted nifi will endlessly crash with the following > error message: > 2021-04-12 16:06:56,100 ERROR [main] org.apache.nifi.web.server.JettyServer > Unable to load flow due to: > org.apache.nifi.controller.serialization.FlowSerializationException: > org.xml.sax.SAXParseException; lineNumber: 341336; columnNumber: 21; > Character reference "" is an invalid XML character. > org.apache.nifi.controller.serialization.FlowSerializationException: > org.xml.sax.SAXParseException; lineNumber: 341336; columnNumber: 21; > Character reference "" is an invalid XML character. > at > org.apache.nifi.cluster.protocol.StandardDataFlow.parseFlowBytes(StandardDataFlow.java:136) > at > org.apache.nifi.cluster.protocol.StandardDataFlow.getFlowDocument(StandardDataFlow.java:96) > at > org.apache.nifi.controller.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:185) > at > org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1413) > at > org.apache.nifi.persistence.StandardXMLFlowConfigurationDAO.load(StandardXMLFlowConfigurationDAO.java:89) > at > org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:810) > at > org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:458) > at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:1133) > at org.apache.nifi.NiFi.(NiFi.java:159) > at org.apache.nifi.NiFi.(NiFi.java:71) > at org.apache.nifi.NiFi.main(NiFi.java:303) > Caused by: org.xml.sax.SAXParseException: Character reference "" is > an invalid XML character. > at > org.apache.xerces.util.ErrorHandlerWrapper.createSAXParseException(Unknown > Source) > at org.apache.xerces.util.ErrorHandlerWrapper.fatalError(Unknown Source) > at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source) > at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source) > at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source) > at org.apache.xerces.impl.XMLScanner.reportFatalError(Unknown Source) > at org.apache.xerces.impl.XMLScanner.scanCharReferenceValue(Unknown Source) > at > org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanCharReference(Unknown > Source) > at > org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown > Source) > at > org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown > Source) > at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source) > at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source) > at org.apache.xerces.parsers.XMLParser.parse(Unknown Source) > at org.apache.xerces.parsers.DOMParser.parse(Unknown Source) > at org.apache.xerces.jaxp.DocumentBuilderImpl.parse(Unknown Source) > at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:121) > at > org.apache.nifi.cluster.protocol.StandardDataFlow.parseFlowBytes(StandardDataFlow.java:134) > ... 10 common frames omitted -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-8417) emoji in processor group name cause nifi to not start up
[ https://issues.apache.org/jira/browse/NIFI-8417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17342563#comment-17342563 ] ASF subversion and git services commented on NIFI-8417: --- Commit 1791f4cc35f3977e44312aed4a2ecd173b8c43c4 in nifi's branch refs/heads/main from s9514171 [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=1791f4c ] NIFI-8417 emoji in processor group name cause nifi to not start up - Excluded Xalan dependency from spring-security-saml2-core This closes #5012 Signed-off-by: David Handermann > emoji in processor group name cause nifi to not start up > > > Key: NIFI-8417 > URL: https://issues.apache.org/jira/browse/NIFI-8417 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.13.0, 1.13.1, 1.13.2 > Environment: RHEL 7, jdk8 >Reporter: Adam Turley >Priority: Blocker > Attachments: Screen Shot 2021-04-12 at 9.17.29 AM.png > > Time Spent: 1.5h > Remaining Estimate: 0h > > if a processor group includes an emoji in the name, nifi will work normally > until restart. Once restarted nifi will endlessly crash with the following > error message: > 2021-04-12 16:06:56,100 ERROR [main] org.apache.nifi.web.server.JettyServer > Unable to load flow due to: > org.apache.nifi.controller.serialization.FlowSerializationException: > org.xml.sax.SAXParseException; lineNumber: 341336; columnNumber: 21; > Character reference "" is an invalid XML character. > org.apache.nifi.controller.serialization.FlowSerializationException: > org.xml.sax.SAXParseException; lineNumber: 341336; columnNumber: 21; > Character reference "" is an invalid XML character. > at > org.apache.nifi.cluster.protocol.StandardDataFlow.parseFlowBytes(StandardDataFlow.java:136) > at > org.apache.nifi.cluster.protocol.StandardDataFlow.getFlowDocument(StandardDataFlow.java:96) > at > org.apache.nifi.controller.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:185) > at > org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1413) > at > org.apache.nifi.persistence.StandardXMLFlowConfigurationDAO.load(StandardXMLFlowConfigurationDAO.java:89) > at > org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:810) > at > org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:458) > at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:1133) > at org.apache.nifi.NiFi.(NiFi.java:159) > at org.apache.nifi.NiFi.(NiFi.java:71) > at org.apache.nifi.NiFi.main(NiFi.java:303) > Caused by: org.xml.sax.SAXParseException: Character reference "" is > an invalid XML character. > at > org.apache.xerces.util.ErrorHandlerWrapper.createSAXParseException(Unknown > Source) > at org.apache.xerces.util.ErrorHandlerWrapper.fatalError(Unknown Source) > at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source) > at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source) > at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source) > at org.apache.xerces.impl.XMLScanner.reportFatalError(Unknown Source) > at org.apache.xerces.impl.XMLScanner.scanCharReferenceValue(Unknown Source) > at > org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanCharReference(Unknown > Source) > at > org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown > Source) > at > org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown > Source) > at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source) > at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source) > at org.apache.xerces.parsers.XMLParser.parse(Unknown Source) > at org.apache.xerces.parsers.DOMParser.parse(Unknown Source) > at org.apache.xerces.jaxp.DocumentBuilderImpl.parse(Unknown Source) > at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:121) > at > org.apache.nifi.cluster.protocol.StandardDataFlow.parseFlowBytes(StandardDataFlow.java:134) > ... 10 common frames omitted -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] asfgit closed pull request #5012: NIFI-8417 emoji in processor group name cause nifi to not start up
asfgit closed pull request #5012: URL: https://github.com/apache/nifi/pull/5012 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-registry] moranr commented on pull request #319: NIFIREG-395 - Implemented the ability to import and export versioned flows through the UI
moranr commented on pull request #319: URL: https://github.com/apache/nifi-registry/pull/319#issuecomment-838476301 > I think the most consistent label would be "Latest (Version #)". @mtien-apache I like that suggestion. > "Export Version" is a good suggestion. Like Bryan, I don't think that change is critical. Also agree here, but not critical. Typically you see Upload/Download or Import/Export used in together. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] exceptionfactory edited a comment on pull request #5012: NIFI-8417 emoji in processor group name cause nifi to not start up
exceptionfactory edited a comment on pull request #5012: URL: https://github.com/apache/nifi/pull/5012#issuecomment-838458447 Confirmed that SAML authentication works with the Xalan dependency. +1 Merging. Thanks for tracking this down @s9514171! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on pull request #5012: NIFI-8417 emoji in processor group name cause nifi to not start up
exceptionfactory commented on pull request #5012: URL: https://github.com/apache/nifi/pull/5012#issuecomment-838458447 Confirmed that SAML authentication works with the Xalan dependency. +1 Merging. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] fgerlits commented on a change in pull request #1066: MINIFICPP-1532: Create a processor to capture resource consumption me…
fgerlits commented on a change in pull request #1066:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1066#discussion_r629991357
##
File path: CMakeLists.txt
##
@@ -501,6 +501,14 @@ if (ENABLE_ALL OR ENABLE_AWS)
createExtension(AWS-EXTENSIONS "AWS EXTENSIONS" "This enables AWS
support" "extensions/aws" "${TEST_DIR}/aws-tests")
endif()
+## PDH Extentions
Review comment:
typo: Exten(t -> s)ions
##
File path: extensions/pdh/MemoryConsumptionCounter.h
##
@@ -0,0 +1,55 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#pragma once
+
+#include "PerformanceDataCounter.h"
+#include
+#include "utils/OsUtils.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace processors {
+
+class MemoryConsumptionCounter : public PerformanceDataCounter {
+ public:
+ MemoryConsumptionCounter() {
+ }
+
+ void addToJson(rapidjson::Value& body, rapidjson::Document::AllocatorType&
alloc) const override {
+rapidjson::Value& group_node = acquireNode(std::string("Memory"), body,
alloc);
+
+rapidjson::Value total_pysical_memory;
Review comment:
typo: pysical -> physical
##
File path: extensions/pdh/PerformanceDataMonitor.h
##
@@ -0,0 +1,102 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#pragma once
+
+#include
+#include
+#include
+#include
+#include
+
+#include "core/Processor.h"
+
+#include "rapidjson/stream.h"
+#include "rapidjson/writer.h"
+#include "rapidjson/prettywriter.h"
+
+#include "PerformanceDataCounter.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace processors {
+
+// PerformanceDataMonitor Class
+class PerformanceDataMonitor : public core::Processor {
+ public:
+ static constexpr char const* JSON_FORMAT_STR = "JSON";
+ static constexpr char const* OPEN_TELEMETRY_FORMAT_STR = "OpenTelemetry";
+
+ explicit PerformanceDataMonitor(const std::string& name, utils::Identifier
uuid = utils::Identifier())
+ : Processor(name, uuid), output_format_(OutputFormat::kJSON),
+ logger_(logging::LoggerFactory::getLogger()),
+ pdh_query_(nullptr), resource_consumption_counters_() {
+ }
+ ~PerformanceDataMonitor() override;
+ static constexpr char const* ProcessorName = "PerformanceDataMonitor";
+ // Supported Properties
+ static core::Property PredefinedGroups;
+ static core::Property CustomPDHCounters;
+ static core::Property OutputFormatProperty;
+ // Supported Relationships
+ static core::Relationship Success;
+ // Nest Callback Class for write stream
+ class WriteCallback : public OutputStreamCallback {
Review comment:
This class could be useful elsewhere, too. If you gave it a better name
(eg. `JSONFlowFileWriterCallback`), I think it could live in `libminifi/utils`.
##
File path: extensions/pdh/PerformanceDataMonitor.cpp
##
@@ -0,0 +1,283 @@
+/**
+ * @file GenerateFlowFile.cpp
+ * GenerateFlowFile class implementation
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *
[GitHub] [nifi] markobean commented on pull request #5064: NIFI-8007: Setup background grid
markobean commented on pull request #5064: URL: https://github.com/apache/nifi/pull/5064#issuecomment-838420712 How difficult would it be to have 3 levels of grid display: off, major gridlines only, major and minor? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-8491) Add Inherited Parameter Contexts tab on Parameter Context view in UI
[ https://issues.apache.org/jira/browse/NIFI-8491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Gresock updated NIFI-8491: - Description: On the Create/Update Parameter Context view, we'll need to add a third tab, "Inherited Parameter Contexts". This should allow selection of one or more existing Parameter Contexts, as well as ordering of the inherited parameter contexts similar to the prioritizers in the connections view (see image). > Add Inherited Parameter Contexts tab on Parameter Context view in UI > > > Key: NIFI-8491 > URL: https://issues.apache.org/jira/browse/NIFI-8491 > Project: Apache NiFi > Issue Type: Sub-task >Reporter: Joseph Gresock >Priority: Major > Attachments: prioritizers.png > > > On the Create/Update Parameter Context view, we'll need to add a third tab, > "Inherited Parameter Contexts". This should allow selection of one or more > existing Parameter Contexts, as well as ordering of the inherited parameter > contexts similar to the prioritizers in the connections view (see image). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-8491) Add Inherited Parameter Contexts tab on Parameter Context view in UI
[ https://issues.apache.org/jira/browse/NIFI-8491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Gresock updated NIFI-8491: - Attachment: prioritizers.png > Add Inherited Parameter Contexts tab on Parameter Context view in UI > > > Key: NIFI-8491 > URL: https://issues.apache.org/jira/browse/NIFI-8491 > Project: Apache NiFi > Issue Type: Sub-task >Reporter: Joseph Gresock >Priority: Major > Attachments: prioritizers.png > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-8491) Add Inherited Parameter Contexts tab on Parameter Context view in UI
[ https://issues.apache.org/jira/browse/NIFI-8491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Gresock updated NIFI-8491: - Summary: Add Inherited Parameter Contexts tab on Parameter Context view in UI (was: UI implementation of Composite Parameter Contexts) > Add Inherited Parameter Contexts tab on Parameter Context view in UI > > > Key: NIFI-8491 > URL: https://issues.apache.org/jira/browse/NIFI-8491 > Project: Apache NiFi > Issue Type: Sub-task >Reporter: Joseph Gresock >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
