[GitHub] nifi issue #2683: NIFI-5146 Only support HTTP or HTTPS operation for NiFi AP...
Github user alopresto commented on the issue: https://github.com/apache/nifi/pull/2683 @kevdoran Fixed the copy/paste error and added an explicit unit test to ensure that mistake is not made again. Thanks. ---
[jira] [Commented] (NIFI-5146) Ability to configure HTTP and HTTPS simultaneously causes HostHeader issues
[ https://issues.apache.org/jira/browse/NIFI-5146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16468218#comment-16468218 ] ASF GitHub Bot commented on NIFI-5146: -- Github user alopresto commented on the issue: https://github.com/apache/nifi/pull/2683 @kevdoran Fixed the copy/paste error and added an explicit unit test to ensure that mistake is not made again. Thanks. > Ability to configure HTTP and HTTPS simultaneously causes HostHeader issues > --- > > Key: NIFI-5146 > URL: https://issues.apache.org/jira/browse/NIFI-5146 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.6.0 >Reporter: Aldrin Piri >Assignee: Andy LoPresto >Priority: Major > Labels: hostname, http, https, security > Fix For: 1.7.0 > > > The host header whitelisting evaluation is only done when NiFi is configured > in secure mode, determined by the setting of an HTTPS port. (see > https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-jetty/src/main/java/org/apache/nifi/web/server/JettyServer.java#L161 > and > [https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-jetty/src/main/java/org/apache/nifi/web/server/HostHeaderHandler.java#L190).] > However, in the case where both are enabled, the HTTP port is not enumerated > in possible combinations and explicit inclusions of a given socket that would > be HTTP is stripped via > [https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-jetty/src/main/java/org/apache/nifi/web/server/HostHeaderHandler.java#L143.] > It is possible that concurrently running HTTP and HTTPS no longer makes > sense, in which case we could evaluate the relevant properties and prevent > startup for an unintended configuration. Alternatively, we would need to > adjust the custom hostname interpretation to also include consideration for > the HTTP port. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4942) NiFi Toolkit - Allow migration of master key without previous password
[ https://issues.apache.org/jira/browse/NIFI-4942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16468210#comment-16468210 ] ASF GitHub Bot commented on NIFI-4942: -- Github user joewitt commented on the issue: https://github.com/apache/nifi/pull/2690 much much better! I'm going to remove the extraneous commented Ignore lines and merge to master. Thanks > NiFi Toolkit - Allow migration of master key without previous password > -- > > Key: NIFI-4942 > URL: https://issues.apache.org/jira/browse/NIFI-4942 > Project: Apache NiFi > Issue Type: Improvement > Components: Tools and Build >Affects Versions: 1.5.0 >Reporter: Yolanda M. Davis >Assignee: Andy LoPresto >Priority: Major > Fix For: 1.7.0 > > Attachments: > TEST-org.apache.nifi.properties.ConfigEncryptionToolTest.xml > > > Currently the encryption cli in nifi toolkit requires that, in order to > migrate from one master key to the next, the previous master key or password > should be provided. In cases where the provisioning tool doesn't have the > previous value available this becomes challenging to provide and may be prone > to error. In speaking with [~alopresto] we can allow toolkit to support a > mode of execution such that the master key can be updated without requiring > the previous password. Also documentation around it's usage should be updated > to be clear in describing the purpose and the type of environment where this > command should be used (admin only access etc). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2690: NIFI-4942 Resolved test failures in JCE limited mode.
Github user joewitt commented on the issue: https://github.com/apache/nifi/pull/2690 much much better! I'm going to remove the extraneous commented Ignore lines and merge to master. Thanks ---
[jira] [Commented] (NIFI-5109) AbstractListProcessor stop calling performListing after primary node reelection
[
https://issues.apache.org/jira/browse/NIFI-5109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16468163#comment-16468163
]
ASF GitHub Bot commented on NIFI-5109:
--
Github user ijokarumawak commented on the issue:
https://github.com/apache/nifi/pull/2657
Maybe I'm thinking the issue oversimplified, but changing the line 378
`return` to `break` would address the issue while ensuring that we don't reset
the flag until we know we've pulled the latest state, wouldn't it?
```
if (LATEST_LISTED_ENTRY_TIMESTAMP_KEY.equals(k)) {
minTimestampToListMillis = Long.parseLong(v);
// If our determined timestamp is the same as that
of our last listing, skip this execution as there are no updates
if
(minTimestampToListMillis.equals(this.lastListedLatestEntryTimestampMillis)) {
context.yield();
return; // How about changing this to 'break'?
} else {
this.lastListedLatestEntryTimestampMillis =
minTimestampToListMillis;
}
} else if
(LAST_PROCESSED_LATEST_ENTRY_TIMESTAMP_KEY.equals(k)) {
this.lastProcessedLatestEntryTimestampMillis =
Long.parseLong(v);
} else if (k.startsWith(IDENTIFIER_PREFIX)) {
latestIdentifiersProcessed.add(v);
}
```
> AbstractListProcessor stop calling performListing after primary node
> reelection
> ---
>
> Key: NIFI-5109
> URL: https://issues.apache.org/jira/browse/NIFI-5109
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.6.0
>Reporter: Maksym Viazovskyi
>Priority: Major
> Attachments:
> 0001-NIFI-2109-Reset-justElectedPrimaryNode-flag-right-af.patch,
> PutAndListSFTP.xml, docker-compose.yml
>
> Original Estimate: 72h
> Remaining Estimate: 72h
>
> AbstractListProcessor stops performing listing of entities when after
> reelection the same node becomes Primary.
> STEPS TO REPRODUCE:
> 0. Make sure that you have docker 18.03.0-ce
> 1. Download attached [^docker-compose.yml] and [^PutAndListSFTP.xml]
> 2. Run *_docker-compose up -d_* in the folder where you downloaded
> docker-compose.yml. This command will run 3 containers - Zookeeper, NiFi
> 1.6.0 in Cluster Mode with 1 node, SFTPs server
> 3. After NiFi is up and running (check http://localhost:8080/nifi) upload
> PutAndListSFTP.xml template and place it on NiFi canvas. It has simple logic
> - generate flow file each 10 seconds and put it on SFTP, in parallel
> ListSFTP processor "listens" for the new files and pushes data about them to
> NiFi log with LogAttribute processor
> 4. Update PutSFTP and ListSFTP processors with password admin (to check SFTP
> container use following credentials SFTP://admin:admin@localhost:)
> 5. Run the flow
> 6. Check the NiFi log (*_docker logs -f --tail 100 nifi_*) - you should see
> attributes for the uploaded files on SFTP
> 7. Check that generated each 10 seconds files appear on SFTP /upload folder
> 8. Run following command to restart Zookeeper to put NiFi in reelection
> state: *_docker restart zookeeper_*
> 9. After reelection phase finished check that new files appears on SFTP, but
> NO new files info appears in NiFi log
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi issue #2657: NIFI-5109 Reset justElectedPrimaryNode flag right after re...
Github user ijokarumawak commented on the issue:
https://github.com/apache/nifi/pull/2657
Maybe I'm thinking the issue oversimplified, but changing the line 378
`return` to `break` would address the issue while ensuring that we don't reset
the flag until we know we've pulled the latest state, wouldn't it?
```
if (LATEST_LISTED_ENTRY_TIMESTAMP_KEY.equals(k)) {
minTimestampToListMillis = Long.parseLong(v);
// If our determined timestamp is the same as that
of our last listing, skip this execution as there are no updates
if
(minTimestampToListMillis.equals(this.lastListedLatestEntryTimestampMillis)) {
context.yield();
return; // How about changing this to 'break'?
} else {
this.lastListedLatestEntryTimestampMillis =
minTimestampToListMillis;
}
} else if
(LAST_PROCESSED_LATEST_ENTRY_TIMESTAMP_KEY.equals(k)) {
this.lastProcessedLatestEntryTimestampMillis =
Long.parseLong(v);
} else if (k.startsWith(IDENTIFIER_PREFIX)) {
latestIdentifiersProcessed.add(v);
}
```
---
[jira] [Updated] (NIFI-4942) NiFi Toolkit - Allow migration of master key without previous password
[ https://issues.apache.org/jira/browse/NIFI-4942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andy LoPresto updated NIFI-4942: Status: Patch Available (was: Open) > NiFi Toolkit - Allow migration of master key without previous password > -- > > Key: NIFI-4942 > URL: https://issues.apache.org/jira/browse/NIFI-4942 > Project: Apache NiFi > Issue Type: Improvement > Components: Tools and Build >Affects Versions: 1.5.0 >Reporter: Yolanda M. Davis >Assignee: Andy LoPresto >Priority: Major > Fix For: 1.7.0 > > Attachments: > TEST-org.apache.nifi.properties.ConfigEncryptionToolTest.xml > > > Currently the encryption cli in nifi toolkit requires that, in order to > migrate from one master key to the next, the previous master key or password > should be provided. In cases where the provisioning tool doesn't have the > previous value available this becomes challenging to provide and may be prone > to error. In speaking with [~alopresto] we can allow toolkit to support a > mode of execution such that the master key can be updated without requiring > the previous password. Also documentation around it's usage should be updated > to be clear in describing the purpose and the type of environment where this > command should be used (admin only access etc). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (NIFI-5159) SimpleCsvFileLookupService doesn't resolve 'variables' properly
[
https://issues.apache.org/jira/browse/NIFI-5159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard resolved NIFI-5159.
--
Resolution: Fixed
Thanks for reporting this [~daniel.arnaiz]. This issue has already been fixed
with NIFI-4149. You can build the lookup services NAR from master if you need
the fix.
> SimpleCsvFileLookupService doesn't resolve 'variables' properly
> ---
>
> Key: NIFI-5159
> URL: https://issues.apache.org/jira/browse/NIFI-5159
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.6.0
>Reporter: Daniel Arnaiz
>Assignee: Pierre Villard
>Priority: Major
> Fix For: 1.7.0
>
>
> - Create a new process Group.
> - Add "Variables" from UI to the process group scope:
> csv=d:\tmp\mycsv.csv
> - Create a controller service SimpleCsvFileLookupService in Group scope.
> - Configure attribute "CSV File" as ${csv}
> - Enable the controller service, so got an error:
> 2018-05-07 08:56:11,160 DEBUG [Timer-Driven Process Thread-10]
> o.a.n.lookup.SimpleCsvFileLookupService
> SimpleCsvFileLookupService[id=c7639ef7-9932-3ae8-d4df-b0eaf5dce379] Loading
> lookup table from file: ${csv}
> java.io.FileNotFoundException: ${csv} (El sistema no puede encontrar el
> archivo especificado)
> at java.io.FileInputStream.open0(Native Method)
> at java.io.FileInputStream.open(FileInputStream.java:195)
> at java.io.FileInputStream.(FileInputStream.java:138)
> at java.io.FileInputStream.(FileInputStream.java:93)
> at java.io.FileReader.(FileReader.java:58)
> at
> org.apache.nifi.lookup.SimpleCsvFileLookupService.loadCache(SimpleCsvFileLookupService.java:139)
> at
> org.apache.nifi.lookup.SimpleCsvFileLookupService.onEnabled(SimpleCsvFileLookupService.java:190)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (NIFI-5159) SimpleCsvFileLookupService doesn't resolve 'variables' properly
[
https://issues.apache.org/jira/browse/NIFI-5159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard reassigned NIFI-5159:
Assignee: Pierre Villard
> SimpleCsvFileLookupService doesn't resolve 'variables' properly
> ---
>
> Key: NIFI-5159
> URL: https://issues.apache.org/jira/browse/NIFI-5159
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.6.0
>Reporter: Daniel Arnaiz
>Assignee: Pierre Villard
>Priority: Major
> Fix For: 1.7.0
>
>
> - Create a new process Group.
> - Add "Variables" from UI to the process group scope:
> csv=d:\tmp\mycsv.csv
> - Create a controller service SimpleCsvFileLookupService in Group scope.
> - Configure attribute "CSV File" as ${csv}
> - Enable the controller service, so got an error:
> 2018-05-07 08:56:11,160 DEBUG [Timer-Driven Process Thread-10]
> o.a.n.lookup.SimpleCsvFileLookupService
> SimpleCsvFileLookupService[id=c7639ef7-9932-3ae8-d4df-b0eaf5dce379] Loading
> lookup table from file: ${csv}
> java.io.FileNotFoundException: ${csv} (El sistema no puede encontrar el
> archivo especificado)
> at java.io.FileInputStream.open0(Native Method)
> at java.io.FileInputStream.open(FileInputStream.java:195)
> at java.io.FileInputStream.(FileInputStream.java:138)
> at java.io.FileInputStream.(FileInputStream.java:93)
> at java.io.FileReader.(FileReader.java:58)
> at
> org.apache.nifi.lookup.SimpleCsvFileLookupService.loadCache(SimpleCsvFileLookupService.java:139)
> at
> org.apache.nifi.lookup.SimpleCsvFileLookupService.onEnabled(SimpleCsvFileLookupService.java:190)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (NIFI-5159) SimpleCsvFileLookupService doesn't resolve 'variables' properly
[
https://issues.apache.org/jira/browse/NIFI-5159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-5159:
-
Fix Version/s: 1.7.0
> SimpleCsvFileLookupService doesn't resolve 'variables' properly
> ---
>
> Key: NIFI-5159
> URL: https://issues.apache.org/jira/browse/NIFI-5159
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.6.0
>Reporter: Daniel Arnaiz
>Priority: Major
> Fix For: 1.7.0
>
>
> - Create a new process Group.
> - Add "Variables" from UI to the process group scope:
> csv=d:\tmp\mycsv.csv
> - Create a controller service SimpleCsvFileLookupService in Group scope.
> - Configure attribute "CSV File" as ${csv}
> - Enable the controller service, so got an error:
> 2018-05-07 08:56:11,160 DEBUG [Timer-Driven Process Thread-10]
> o.a.n.lookup.SimpleCsvFileLookupService
> SimpleCsvFileLookupService[id=c7639ef7-9932-3ae8-d4df-b0eaf5dce379] Loading
> lookup table from file: ${csv}
> java.io.FileNotFoundException: ${csv} (El sistema no puede encontrar el
> archivo especificado)
> at java.io.FileInputStream.open0(Native Method)
> at java.io.FileInputStream.open(FileInputStream.java:195)
> at java.io.FileInputStream.(FileInputStream.java:138)
> at java.io.FileInputStream.(FileInputStream.java:93)
> at java.io.FileReader.(FileReader.java:58)
> at
> org.apache.nifi.lookup.SimpleCsvFileLookupService.loadCache(SimpleCsvFileLookupService.java:139)
> at
> org.apache.nifi.lookup.SimpleCsvFileLookupService.onEnabled(SimpleCsvFileLookupService.java:190)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (NIFI-5159) SimpleCsvFileLookupService doesn't resolve 'variables' properly
[
https://issues.apache.org/jira/browse/NIFI-5159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-5159:
-
Component/s: (was: Core Framework)
Extensions
> SimpleCsvFileLookupService doesn't resolve 'variables' properly
> ---
>
> Key: NIFI-5159
> URL: https://issues.apache.org/jira/browse/NIFI-5159
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.6.0
>Reporter: Daniel Arnaiz
>Assignee: Pierre Villard
>Priority: Major
> Fix For: 1.7.0
>
>
> - Create a new process Group.
> - Add "Variables" from UI to the process group scope:
> csv=d:\tmp\mycsv.csv
> - Create a controller service SimpleCsvFileLookupService in Group scope.
> - Configure attribute "CSV File" as ${csv}
> - Enable the controller service, so got an error:
> 2018-05-07 08:56:11,160 DEBUG [Timer-Driven Process Thread-10]
> o.a.n.lookup.SimpleCsvFileLookupService
> SimpleCsvFileLookupService[id=c7639ef7-9932-3ae8-d4df-b0eaf5dce379] Loading
> lookup table from file: ${csv}
> java.io.FileNotFoundException: ${csv} (El sistema no puede encontrar el
> archivo especificado)
> at java.io.FileInputStream.open0(Native Method)
> at java.io.FileInputStream.open(FileInputStream.java:195)
> at java.io.FileInputStream.(FileInputStream.java:138)
> at java.io.FileInputStream.(FileInputStream.java:93)
> at java.io.FileReader.(FileReader.java:58)
> at
> org.apache.nifi.lookup.SimpleCsvFileLookupService.loadCache(SimpleCsvFileLookupService.java:139)
> at
> org.apache.nifi.lookup.SimpleCsvFileLookupService.onEnabled(SimpleCsvFileLookupService.java:190)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-950) Perform component validation asynchronously
[ https://issues.apache.org/jira/browse/NIFI-950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16467889#comment-16467889 ] ASF GitHub Bot commented on NIFI-950: - Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/2689 @mcgilman I am closing this PR because it needs to be rebased and I ran into an issue with a unit test during the rebase. Will open a new PR once that has been addressed. > Perform component validation asynchronously > --- > > Key: NIFI-950 > URL: https://issues.apache.org/jira/browse/NIFI-950 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Joseph Percivall >Priority: Major > Attachments: self_reference_flow_fix.xml > > > I created a flow that is a self referencing http loop. The flow was working > fine but I wanted to save the template for later testing. I downloaded the > the flow as a template. Then I tried testing a thread.sleep in the beginning > of onConfigured, createSSLContext, and validate methods of > StandardSSLContextService. I did a mvn clean install in the > nifi-nar-bundles/nifi-standard-services/nifi-ssl-context-bundle/nifi-ssl-context-service > directory. Then a mvn clean install in the nifi-assembly directory. After I > imported the template the UI became very slow when clicking to different > windows of the UI such as configuring a processor and the controller services > window. > I then stashed my changes and rebuilt the files. Once again I imported my > template, and attempting to configure a processor or accessing the controller > services window became very slow. > The flow xml is attached. > --- > The description and attachment showed an issue where long running validation > caused the UI to become unresponsive. This validation should be done > asynchronously so that the UI always remains responsive. Initial thoughts... > - new state to indicate that validation is in progress > - a mechanism for refreshing validation results > - time out for waiting for validation to complete? or need to always be > validating all components in case their validity is based on something > environmental (like a configuration file that is modified outside of the > application)? > - provide better support for components that are running and become invalid > -- related to this we need to provide guidance regarding the difference > between become invalid and when we should use features like bulletins and > yielding to rely runtime issues -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-950) Perform component validation asynchronously
[ https://issues.apache.org/jira/browse/NIFI-950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16467890#comment-16467890 ] ASF GitHub Bot commented on NIFI-950: - Github user markap14 closed the pull request at: https://github.com/apache/nifi/pull/2689 > Perform component validation asynchronously > --- > > Key: NIFI-950 > URL: https://issues.apache.org/jira/browse/NIFI-950 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Joseph Percivall >Priority: Major > Attachments: self_reference_flow_fix.xml > > > I created a flow that is a self referencing http loop. The flow was working > fine but I wanted to save the template for later testing. I downloaded the > the flow as a template. Then I tried testing a thread.sleep in the beginning > of onConfigured, createSSLContext, and validate methods of > StandardSSLContextService. I did a mvn clean install in the > nifi-nar-bundles/nifi-standard-services/nifi-ssl-context-bundle/nifi-ssl-context-service > directory. Then a mvn clean install in the nifi-assembly directory. After I > imported the template the UI became very slow when clicking to different > windows of the UI such as configuring a processor and the controller services > window. > I then stashed my changes and rebuilt the files. Once again I imported my > template, and attempting to configure a processor or accessing the controller > services window became very slow. > The flow xml is attached. > --- > The description and attachment showed an issue where long running validation > caused the UI to become unresponsive. This validation should be done > asynchronously so that the UI always remains responsive. Initial thoughts... > - new state to indicate that validation is in progress > - a mechanism for refreshing validation results > - time out for waiting for validation to complete? or need to always be > validating all components in case their validity is based on something > environmental (like a configuration file that is modified outside of the > application)? > - provide better support for components that are running and become invalid > -- related to this we need to provide guidance regarding the difference > between become invalid and when we should use features like bulletins and > yielding to rely runtime issues -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #2689: NIFI-950: Perform component validation asynchronous...
Github user markap14 closed the pull request at: https://github.com/apache/nifi/pull/2689 ---
[GitHub] nifi issue #2689: NIFI-950: Perform component validation asynchronously, oth...
Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/2689 @mcgilman I am closing this PR because it needs to be rebased and I ran into an issue with a unit test during the rebase. Will open a new PR once that has been addressed. ---
[jira] [Updated] (NIFI-5136) Leaked component references preventing GC of components and class loaders
[ https://issues.apache.org/jira/browse/NIFI-5136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-5136: - Resolution: Fixed Status: Resolved (was: Patch Available) > Leaked component references preventing GC of components and class loaders > - > > Key: NIFI-5136 > URL: https://issues.apache.org/jira/browse/NIFI-5136 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.2.0, 1.3.0, 1.4.0, 1.5.0, 1.6.0 >Reporter: Bryan Bende >Assignee: Bryan Bende >Priority: Major > > A user on the mailing list reported that after some time of creating/deleting > HDFS processors, it appeared that the classes/instances were still around and > eventually the NiFi instance would get out of memory and need to be restarted. > After investigation there are multiple issues preventing garbage collection > of deleted components. One issue is specific to the HDFS processors, the > other issues are for all components... > 1) The LogRepository still has a reference to a ComponentLogger which has a > reference to the component > 2) The processor scheduler has a map of scheduled states which has references > to processors that have been deleted > 3) The Hadoop processors start a thread that is never stopped when the > processor is stopped/deleted, this means the class loader can't be cleaned up > b/c the Runnable came from the InstanceClassLoader of the deleted processor > 4) Importing a flow from registry will instantiate an instance of each > component to ensure the incoming types are valid, but the InstanceClassLoader > and ComponentLogger are not cleaned up for these temp instances > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5136) Leaked component references preventing GC of components and class loaders
[ https://issues.apache.org/jira/browse/NIFI-5136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16467861#comment-16467861 ] ASF GitHub Bot commented on NIFI-5136: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2668 > Leaked component references preventing GC of components and class loaders > - > > Key: NIFI-5136 > URL: https://issues.apache.org/jira/browse/NIFI-5136 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.2.0, 1.3.0, 1.4.0, 1.5.0, 1.6.0 >Reporter: Bryan Bende >Assignee: Bryan Bende >Priority: Major > > A user on the mailing list reported that after some time of creating/deleting > HDFS processors, it appeared that the classes/instances were still around and > eventually the NiFi instance would get out of memory and need to be restarted. > After investigation there are multiple issues preventing garbage collection > of deleted components. One issue is specific to the HDFS processors, the > other issues are for all components... > 1) The LogRepository still has a reference to a ComponentLogger which has a > reference to the component > 2) The processor scheduler has a map of scheduled states which has references > to processors that have been deleted > 3) The Hadoop processors start a thread that is never stopped when the > processor is stopped/deleted, this means the class loader can't be cleaned up > b/c the Runnable came from the InstanceClassLoader of the deleted processor > 4) Importing a flow from registry will instantiate an instance of each > component to ensure the incoming types are valid, but the InstanceClassLoader > and ComponentLogger are not cleaned up for these temp instances > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4942) NiFi Toolkit - Allow migration of master key without previous password
[ https://issues.apache.org/jira/browse/NIFI-4942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16467857#comment-16467857 ] ASF GitHub Bot commented on NIFI-4942: -- GitHub user alopresto opened a pull request: https://github.com/apache/nifi/pull/2690 NIFI-4942 Resolved test failures in JCE limited mode. Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [x] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/alopresto/nifi NIFI-4942-jce-issue Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2690.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2690 commit 0006a0c06dd44656b8f25f6ecebaa68565ff4c1c Author: Andy LoPresto Date: 2018-05-08T19:28:10Z NIFI-4942 Resolved test failures in JCE limited mode. > NiFi Toolkit - Allow migration of master key without previous password > -- > > Key: NIFI-4942 > URL: https://issues.apache.org/jira/browse/NIFI-4942 > Project: Apache NiFi > Issue Type: Improvement > Components: Tools and Build >Affects Versions: 1.5.0 >Reporter: Yolanda M. Davis >Assignee: Andy LoPresto >Priority: Major > Fix For: 1.7.0 > > Attachments: > TEST-org.apache.nifi.properties.ConfigEncryptionToolTest.xml > > > Currently the encryption cli in nifi toolkit requires that, in order to > migrate from one master key to the next, the previous master key or password > should be provided. In cases where the provisioning tool doesn't have the > previous value available this becomes challenging to provide and may be prone > to error. In speaking with [~alopresto] we can allow toolkit to support a > mode of execution such that the master key can be updated without requiring > the previous password. Also documentation around it's usage should be updated > to be clear in describing the purpose and the type of environment where this > command should be used (admin only access etc). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #2668: NIFI-5136 Ensure processor references are removed f...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2668 ---
[jira] [Commented] (NIFI-5136) Leaked component references preventing GC of components and class loaders
[ https://issues.apache.org/jira/browse/NIFI-5136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16467856#comment-16467856 ] ASF GitHub Bot commented on NIFI-5136: -- Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/2668 @bbende thanks for addressing these issues! I can see that we've tackled a handful of spots that could be leaking the references, and I've seen the heap dumps showing that they are no longer concerns. The changes to the Abstract Hadoop Processors are less desirable than I'd prefer, but I agree that the API doesn't really expose what it would need to expose in order to do this more effectively, so it's a reasonable workaround. The LogRepositoryFactory change in FlowController's create* methods is good - i hadn't realized that we were doing that there, within the nar loader. With this particular scenario, I don't think it's really replicable in a unit test. All that could be done in unit tests would be to verify that very trivial functionality like "removeLogger() calls remove() on the underlying map" which make for bad unit testing IMO. You could attempt to create some very involved integration tests, but they would involve mocking out so much that it would be hard to know what is really being verified. On top of that, you still would not be able to verify that the appropriate objects are reclaimable via garbage collection. This all looks good to me, though. Can verify behavior and code changes look good. +1 will merge to master. > Leaked component references preventing GC of components and class loaders > - > > Key: NIFI-5136 > URL: https://issues.apache.org/jira/browse/NIFI-5136 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.2.0, 1.3.0, 1.4.0, 1.5.0, 1.6.0 >Reporter: Bryan Bende >Assignee: Bryan Bende >Priority: Major > > A user on the mailing list reported that after some time of creating/deleting > HDFS processors, it appeared that the classes/instances were still around and > eventually the NiFi instance would get out of memory and need to be restarted. > After investigation there are multiple issues preventing garbage collection > of deleted components. One issue is specific to the HDFS processors, the > other issues are for all components... > 1) The LogRepository still has a reference to a ComponentLogger which has a > reference to the component > 2) The processor scheduler has a map of scheduled states which has references > to processors that have been deleted > 3) The Hadoop processors start a thread that is never stopped when the > processor is stopped/deleted, this means the class loader can't be cleaned up > b/c the Runnable came from the InstanceClassLoader of the deleted processor > 4) Importing a flow from registry will instantiate an instance of each > component to ensure the incoming types are valid, but the InstanceClassLoader > and ComponentLogger are not cleaned up for these temp instances > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5136) Leaked component references preventing GC of components and class loaders
[ https://issues.apache.org/jira/browse/NIFI-5136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16467858#comment-16467858 ] ASF subversion and git services commented on NIFI-5136: --- Commit 92b4a3208fddd44f0d7e3d618761b9db238b758c in nifi's branch refs/heads/master from [~bbende] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=92b4a32 ] NIFI-5136 Ensure processor references are removed from LogRepository and from ProcessScheduler - Forcing FileSystem statistics thread to be interrupted when HDFS processors are stopped - Stop creating temp components during import from registry, use bundle info instead This closes #2668. Signed-off-by: Mark Payne > Leaked component references preventing GC of components and class loaders > - > > Key: NIFI-5136 > URL: https://issues.apache.org/jira/browse/NIFI-5136 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.2.0, 1.3.0, 1.4.0, 1.5.0, 1.6.0 >Reporter: Bryan Bende >Assignee: Bryan Bende >Priority: Major > > A user on the mailing list reported that after some time of creating/deleting > HDFS processors, it appeared that the classes/instances were still around and > eventually the NiFi instance would get out of memory and need to be restarted. > After investigation there are multiple issues preventing garbage collection > of deleted components. One issue is specific to the HDFS processors, the > other issues are for all components... > 1) The LogRepository still has a reference to a ComponentLogger which has a > reference to the component > 2) The processor scheduler has a map of scheduled states which has references > to processors that have been deleted > 3) The Hadoop processors start a thread that is never stopped when the > processor is stopped/deleted, this means the class loader can't be cleaned up > b/c the Runnable came from the InstanceClassLoader of the deleted processor > 4) Importing a flow from registry will instantiate an instance of each > component to ensure the incoming types are valid, but the InstanceClassLoader > and ComponentLogger are not cleaned up for these temp instances > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #2690: NIFI-4942 Resolved test failures in JCE limited mod...
GitHub user alopresto opened a pull request: https://github.com/apache/nifi/pull/2690 NIFI-4942 Resolved test failures in JCE limited mode. Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [x] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/alopresto/nifi NIFI-4942-jce-issue Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2690.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2690 commit 0006a0c06dd44656b8f25f6ecebaa68565ff4c1c Author: Andy LoPresto Date: 2018-05-08T19:28:10Z NIFI-4942 Resolved test failures in JCE limited mode. ---
[GitHub] nifi issue #2668: NIFI-5136 Ensure processor references are removed from Log...
Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/2668 @bbende thanks for addressing these issues! I can see that we've tackled a handful of spots that could be leaking the references, and I've seen the heap dumps showing that they are no longer concerns. The changes to the Abstract Hadoop Processors are less desirable than I'd prefer, but I agree that the API doesn't really expose what it would need to expose in order to do this more effectively, so it's a reasonable workaround. The LogRepositoryFactory change in FlowController's create* methods is good - i hadn't realized that we were doing that there, within the nar loader. With this particular scenario, I don't think it's really replicable in a unit test. All that could be done in unit tests would be to verify that very trivial functionality like "removeLogger() calls remove() on the underlying map" which make for bad unit testing IMO. You could attempt to create some very involved integration tests, but they would involve mocking out so much that it would be hard to know what is really being verified. On top of that, you still would not be able to verify that the appropriate objects are reclaimable via garbage collection. This all looks good to me, though. Can verify behavior and code changes look good. +1 will merge to master. ---
[jira] [Updated] (NIFI-5176) NiFi needs to be buildable on Java 9
[ https://issues.apache.org/jira/browse/NIFI-5176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Storck updated NIFI-5176: -- Description: While retaining a source/target comptability of 1.8, NiFi needs to be buildable on Java 9. The following issues have been encountered while attempting to run a Java 1.8-built NiFi on Java 9: ||Issue||Solution|| |Groovy compiler not parsing groovy code correctly on Java 9|Updated maven-compiler-plugin to 3.7.0, and included dependencies for groovy-eclipse-compiler:2.9.3-01 and groovy-eclipse-batch:2.4.15-01| |Antler isn't able to process grammars| | was:While retaining a source/target comptability of 1.8, NiFi needs to be buildable on Java 9. > NiFi needs to be buildable on Java 9 > > > Key: NIFI-5176 > URL: https://issues.apache.org/jira/browse/NIFI-5176 > Project: Apache NiFi > Issue Type: Sub-task >Reporter: Jeff Storck >Assignee: Jeff Storck >Priority: Major > > While retaining a source/target comptability of 1.8, NiFi needs to be > buildable on Java 9. > The following issues have been encountered while attempting to run a Java > 1.8-built NiFi on Java 9: > ||Issue||Solution|| > |Groovy compiler not parsing groovy code correctly on Java 9|Updated > maven-compiler-plugin to 3.7.0, and included dependencies for > groovy-eclipse-compiler:2.9.3-01 and groovy-eclipse-batch:2.4.15-01| > |Antler isn't able to process grammars| | -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (NIFI-5175) NiFi built in Java 1.8 needs to run on Java 9
[ https://issues.apache.org/jira/browse/NIFI-5175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Storck updated NIFI-5175: -- Description: The following issues have been encountered while attempting to run a Java 1.8-built NiFi on Java 9: ||Issue||Solution|| |JAXB complains that classes do not have a default no-arg public constructor|Add no-arg public constructors| > NiFi built in Java 1.8 needs to run on Java 9 > - > > Key: NIFI-5175 > URL: https://issues.apache.org/jira/browse/NIFI-5175 > Project: Apache NiFi > Issue Type: Sub-task >Reporter: Jeff Storck >Assignee: Jeff Storck >Priority: Major > > The following issues have been encountered while attempting to run a Java > 1.8-built NiFi on Java 9: > ||Issue||Solution|| > |JAXB complains that classes do not have a default no-arg public > constructor|Add no-arg public constructors| -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (NIFI-5176) NiFi needs to be buildable on Java 9
Jeff Storck created NIFI-5176: - Summary: NiFi needs to be buildable on Java 9 Key: NIFI-5176 URL: https://issues.apache.org/jira/browse/NIFI-5176 Project: Apache NiFi Issue Type: Sub-task Reporter: Jeff Storck Assignee: Jeff Storck While retaining a source/target comptability of 1.8, NiFi needs to be buildable on Java 9. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (NIFI-5173) Graph search control fails to demonstrate component selection
[
https://issues.apache.org/jira/browse/NIFI-5173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Aversa updated NIFI-5173:
--
Description: When using the graph search control to locate a component
within a flow, the searched item fails to render as selected on the graph. The
item is positioned correctly, but is not highlighted accordingly. Preliminary
research indicated that within the *nfActions.show* method, the
*selection.classed('selected',true);* is being reset by a subsequent call.
(was: When using the graph search control to locate a component within a flow,
the searched item fails to render as selected on the graph. The item is
positioned correctly, but is not highlighted accordingly. Preliminary research
indicated that within the *nfActions.show* method, the
*selection.classed('selected',true);* is being reset by another call.)
> Graph search control fails to demonstrate component selection
> -
>
> Key: NIFI-5173
> URL: https://issues.apache.org/jira/browse/NIFI-5173
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core UI
>Affects Versions: 1.6.0
>Reporter: Alex Aversa
>Priority: Minor
>
> When using the graph search control to locate a component within a flow, the
> searched item fails to render as selected on the graph. The item is
> positioned correctly, but is not highlighted accordingly. Preliminary
> research indicated that within the *nfActions.show* method, the
> *selection.classed('selected',true);* is being reset by a subsequent call.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (NIFI-5175) NiFi built in Java 1.8 needs to run on Java 9
Jeff Storck created NIFI-5175: - Summary: NiFi built in Java 1.8 needs to run on Java 9 Key: NIFI-5175 URL: https://issues.apache.org/jira/browse/NIFI-5175 Project: Apache NiFi Issue Type: Sub-task Reporter: Jeff Storck Assignee: Jeff Storck -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (NIFI-5174) NiFi Compatibility with Java 9
Jeff Storck created NIFI-5174: - Summary: NiFi Compatibility with Java 9 Key: NIFI-5174 URL: https://issues.apache.org/jira/browse/NIFI-5174 Project: Apache NiFi Issue Type: Task Reporter: Jeff Storck Assignee: Jeff Storck -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (NIFI-5173) Graph search control fails to demonstrate component selection
[
https://issues.apache.org/jira/browse/NIFI-5173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Aversa updated NIFI-5173:
--
Priority: Minor (was: Major)
> Graph search control fails to demonstrate component selection
> -
>
> Key: NIFI-5173
> URL: https://issues.apache.org/jira/browse/NIFI-5173
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core UI
>Affects Versions: 1.6.0
>Reporter: Alex Aversa
>Priority: Minor
>
> When using the graph search control to locate a component within a flow, the
> searched item fails to render as selected on the graph. The item is
> positioned correctly, but is not highlighted accordingly. Preliminary
> research indicated that within the *nfActions.show* method, the
> *selection.classed('selected',true);* is being reset by another call.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (NIFI-5173) Graph search control fails to demonstrate component selection
Alex Aversa created NIFI-5173:
-
Summary: Graph search control fails to demonstrate component
selection
Key: NIFI-5173
URL: https://issues.apache.org/jira/browse/NIFI-5173
Project: Apache NiFi
Issue Type: Bug
Components: Core UI
Affects Versions: 1.6.0
Reporter: Alex Aversa
When using the graph search control to locate a component within a flow, the
searched item fails to render as selected on the graph. The item is positioned
correctly, but is not highlighted accordingly. Preliminary research indicated
that within the *nfActions.show* method, the
*selection.classed('selected',true);* is being reset by another call.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (MINIFICPP-489) Linter issues
Aldrin Piri created MINIFICPP-489: - Summary: Linter issues Key: MINIFICPP-489 URL: https://issues.apache.org/jira/browse/MINIFICPP-489 Project: NiFi MiNiFi C++ Issue Type: Task Reporter: Aldrin Piri Assignee: Aldrin Piri Fix For: 0.5.0 There are currently some linter issues that have cropped up in Travis's absence. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-minifi-cpp pull request #321: MINIFICPP-394: Implement MQTT C2 protocol...
Github user asfgit closed the pull request at: https://github.com/apache/nifi-minifi-cpp/pull/321 ---
[jira] [Commented] (MINIFICPP-394) Implement MQTT C2 protocol
[ https://issues.apache.org/jira/browse/MINIFICPP-394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16467784#comment-16467784 ] ASF GitHub Bot commented on MINIFICPP-394: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi-minifi-cpp/pull/321 > Implement MQTT C2 protocol > --- > > Key: MINIFICPP-394 > URL: https://issues.apache.org/jira/browse/MINIFICPP-394 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: marco polo >Assignee: marco polo >Priority: Major > Fix For: 0.5.0 > > > Implement MQTT C2 protocol as we likely won't want to use the REST protocol > long term -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (MINIFICPP-394) Implement MQTT C2 protocol
[ https://issues.apache.org/jira/browse/MINIFICPP-394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aldrin Piri resolved MINIFICPP-394. --- Resolution: Fixed Assignee: marco polo Fix Version/s: 0.5.0 > Implement MQTT C2 protocol > --- > > Key: MINIFICPP-394 > URL: https://issues.apache.org/jira/browse/MINIFICPP-394 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: marco polo >Assignee: marco polo >Priority: Major > Fix For: 0.5.0 > > > Implement MQTT C2 protocol as we likely won't want to use the REST protocol > long term -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-394) Implement MQTT C2 protocol
[ https://issues.apache.org/jira/browse/MINIFICPP-394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16467780#comment-16467780 ] ASF GitHub Bot commented on MINIFICPP-394: -- Github user apiri commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/321 Was able to verify #321 and rebase this on that. Build looks good here. > Implement MQTT C2 protocol > --- > > Key: MINIFICPP-394 > URL: https://issues.apache.org/jira/browse/MINIFICPP-394 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: marco polo >Priority: Major > > Implement MQTT C2 protocol as we likely won't want to use the REST protocol > long term -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-minifi-cpp issue #321: MINIFICPP-394: Implement MQTT C2 protocol along ...
Github user apiri commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/321 Was able to verify #321 and rebase this on that. Build looks good here. ---
[jira] [Commented] (NIFI-950) Perform component validation asynchronously
[ https://issues.apache.org/jira/browse/NIFI-950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16467774#comment-16467774 ] ASF GitHub Bot commented on NIFI-950: - Github user mcgilman commented on the issue: https://github.com/apache/nifi/pull/2689 Will review... > Perform component validation asynchronously > --- > > Key: NIFI-950 > URL: https://issues.apache.org/jira/browse/NIFI-950 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Joseph Percivall >Priority: Major > Attachments: self_reference_flow_fix.xml > > > I created a flow that is a self referencing http loop. The flow was working > fine but I wanted to save the template for later testing. I downloaded the > the flow as a template. Then I tried testing a thread.sleep in the beginning > of onConfigured, createSSLContext, and validate methods of > StandardSSLContextService. I did a mvn clean install in the > nifi-nar-bundles/nifi-standard-services/nifi-ssl-context-bundle/nifi-ssl-context-service > directory. Then a mvn clean install in the nifi-assembly directory. After I > imported the template the UI became very slow when clicking to different > windows of the UI such as configuring a processor and the controller services > window. > I then stashed my changes and rebuilt the files. Once again I imported my > template, and attempting to configure a processor or accessing the controller > services window became very slow. > The flow xml is attached. > --- > The description and attachment showed an issue where long running validation > caused the UI to become unresponsive. This validation should be done > asynchronously so that the UI always remains responsive. Initial thoughts... > - new state to indicate that validation is in progress > - a mechanism for refreshing validation results > - time out for waiting for validation to complete? or need to always be > validating all components in case their validity is based on something > environmental (like a configuration file that is modified outside of the > application)? > - provide better support for components that are running and become invalid > -- related to this we need to provide guidance regarding the difference > between become invalid and when we should use features like bulletins and > yielding to rely runtime issues -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2689: NIFI-950: Perform component validation asynchronously, oth...
Github user mcgilman commented on the issue: https://github.com/apache/nifi/pull/2689 Will review... ---
[jira] [Commented] (NIFI-950) Perform component validation asynchronously
[ https://issues.apache.org/jira/browse/NIFI-950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16467771#comment-16467771 ] ASF GitHub Bot commented on NIFI-950: - GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/2689 NIFI-950: Perform component validation asynchronously, other cluster-related performance enhancements This PR addresses NIFI-950 and a handful of other JIRA's. They were all done in a single PR because the modifications tend to build upon one another, and also because this PR really addresses the issue of making nifi clustering more responsive (in terms of UI) and more stable. Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-950 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2689.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2689 commit 05f77401395b4274f2b09bf43b812af325e81690 Author: Mark Payne Date: 2018-04-11T19:36:54Z NIFI-950: Make component validation asynchronous NIFI-950: Still seeing some slow response times when instantiating a large template in cluster mode so making some minor tweaks based on the results of CPU profiling NIFI-5112: Refactored FlowSerializer so that it creates the desired intermediate data model that can be serialized, separate from serializing. This allows us to hold the FlowController's Read Lock only while creating the data model, not while actually serializing the data. Configured Jersey Client in ThreadPoolRequestReplicator not to look for features using the Service Loader for every request. Updated Template object to hold a DOM Node that represents the template contents instead of having to serialize the DTO, then parse the serialized form as a DOM object each time that it needs to be serialized. NIFI-5112: Change ThreadPoolRequestReplicator to use OkHttp client instead of Jersey Client NIFI-5111: Ensure that if a node is no longer cluster coordinator, that it clears any stale heartbeats. NIFI-5110: Notify StandardProcessScheduler when a component is removed so that it will clean up any resource related to component lifecycle. commit 3ef86dc0c4e2c4855972cd0c200b7b9982e0c040 Author: Mark Payne Date: 2018-04-24T19:52:36Z NIFI-950: Avoid gathering the Status objects for entire flow when we don't need them; removed unnecessary code commit a7c7fcb7c3fc72f8fc0a32e6ebdc4795bc12a8b4 Author: Mark Payne Date: 2018-05-03T20:07:10Z NIFI-950: Bug fixes commit 434aa1dd85c53a7cc58497aad42246692a19a739 Author: Mark Payne Date: 2018-05-08T17:35:44Z NIFI-950: Bug fix; added validation status to ProcessorDTO, ControllerServiceDTO, ReportingTaskDTO; updated DebugFlow to allow for pause time to be set in the customValidate method for testing functionality > Perform component validation asynchronously > --- > >
[GitHub] nifi pull request #2689: NIFI-950: Perform component validation asynchronous...
GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/2689 NIFI-950: Perform component validation asynchronously, other cluster-related performance enhancements This PR addresses NIFI-950 and a handful of other JIRA's. They were all done in a single PR because the modifications tend to build upon one another, and also because this PR really addresses the issue of making nifi clustering more responsive (in terms of UI) and more stable. Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-950 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2689.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2689 commit 05f77401395b4274f2b09bf43b812af325e81690 Author: Mark Payne Date: 2018-04-11T19:36:54Z NIFI-950: Make component validation asynchronous NIFI-950: Still seeing some slow response times when instantiating a large template in cluster mode so making some minor tweaks based on the results of CPU profiling NIFI-5112: Refactored FlowSerializer so that it creates the desired intermediate data model that can be serialized, separate from serializing. This allows us to hold the FlowController's Read Lock only while creating the data model, not while actually serializing the data. Configured Jersey Client in ThreadPoolRequestReplicator not to look for features using the Service Loader for every request. Updated Template object to hold a DOM Node that represents the template contents instead of having to serialize the DTO, then parse the serialized form as a DOM object each time that it needs to be serialized. NIFI-5112: Change ThreadPoolRequestReplicator to use OkHttp client instead of Jersey Client NIFI-5111: Ensure that if a node is no longer cluster coordinator, that it clears any stale heartbeats. NIFI-5110: Notify StandardProcessScheduler when a component is removed so that it will clean up any resource related to component lifecycle. commit 3ef86dc0c4e2c4855972cd0c200b7b9982e0c040 Author: Mark Payne Date: 2018-04-24T19:52:36Z NIFI-950: Avoid gathering the Status objects for entire flow when we don't need them; removed unnecessary code commit a7c7fcb7c3fc72f8fc0a32e6ebdc4795bc12a8b4 Author: Mark Payne Date: 2018-05-03T20:07:10Z NIFI-950: Bug fixes commit 434aa1dd85c53a7cc58497aad42246692a19a739 Author: Mark Payne Date: 2018-05-08T17:35:44Z NIFI-950: Bug fix; added validation status to ProcessorDTO, ControllerServiceDTO, ReportingTaskDTO; updated DebugFlow to allow for pause time to be set in the customValidate method for testing functionality ---
[jira] [Created] (NIFI-5172) PutElasticSearchRecord does to fail individual recods
Juan C. Sequeiros created NIFI-5172: --- Summary: PutElasticSearchRecord does to fail individual recods Key: NIFI-5172 URL: https://issues.apache.org/jira/browse/NIFI-5172 Project: Apache NiFi Issue Type: Improvement Affects Versions: 1.6.0 Reporter: Juan C. Sequeiros My observation and not sure if working as expected but when I send my output from MergeRecord ( set to 1 ) max number of records and one of those records has an invalid timestamp "bogusdata" value ES rejects it, rightly so since on ES we have a schema template more granular and is expecting timestamp as type "date". >From USER forums Matt Burgess: Yes the current behavior is to move the entire input flowfile to failure if any errors occur. Some other record-aware processors create separate flow files for failed and successful records, but PutElasticsearchHttpRecord does not (yet) do that. Please feel free to write a Jira for this improvement. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (MINIFICPP-488) Resolve build issue with EL value
[ https://issues.apache.org/jira/browse/MINIFICPP-488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aldrin Piri updated MINIFICPP-488: -- Fix Version/s: 0.5.0 > Resolve build issue with EL value > - > > Key: MINIFICPP-488 > URL: https://issues.apache.org/jira/browse/MINIFICPP-488 > Project: NiFi MiNiFi C++ > Issue Type: Bug >Reporter: marco polo >Assignee: marco polo >Priority: Major > Fix For: 0.5.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (MINIFICPP-488) Resolve build issue with EL value
[ https://issues.apache.org/jira/browse/MINIFICPP-488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aldrin Piri resolved MINIFICPP-488. --- Resolution: Fixed > Resolve build issue with EL value > - > > Key: MINIFICPP-488 > URL: https://issues.apache.org/jira/browse/MINIFICPP-488 > Project: NiFi MiNiFi C++ > Issue Type: Bug >Reporter: marco polo >Assignee: marco polo >Priority: Major > Fix For: 0.5.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-488) Resolve build issue with EL value
[ https://issues.apache.org/jira/browse/MINIFICPP-488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16467736#comment-16467736 ] ASF GitHub Bot commented on MINIFICPP-488: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi-minifi-cpp/pull/324 > Resolve build issue with EL value > - > > Key: MINIFICPP-488 > URL: https://issues.apache.org/jira/browse/MINIFICPP-488 > Project: NiFi MiNiFi C++ > Issue Type: Bug >Reporter: marco polo >Assignee: marco polo >Priority: Major > Fix For: 0.5.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-minifi-cpp pull request #324: MINIFICPP-488: Use already implemented ui...
Github user asfgit closed the pull request at: https://github.com/apache/nifi-minifi-cpp/pull/324 ---
[GitHub] nifi pull request #2680: NIFI-5156: Updated GCP SDK to latest version
Github user zenfenan commented on a diff in the pull request: https://github.com/apache/nifi/pull/2680#discussion_r186804537 --- Diff: nifi-nar-bundles/nifi-gcp-bundle/pom.xml --- @@ -26,6 +26,22 @@ 1.7.0-SNAPSHOT pom + +0.46.0-alpha --- End diff -- Cool. Then I'll try to update it and build and check. If everything works, I'll update the PR. ---
[jira] [Commented] (NIFI-5156) Update 'Google Cloud SDK' version and refactor GCP processors' code
[ https://issues.apache.org/jira/browse/NIFI-5156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16467716#comment-16467716 ] ASF GitHub Bot commented on NIFI-5156: -- Github user zenfenan commented on a diff in the pull request: https://github.com/apache/nifi/pull/2680#discussion_r186804537 --- Diff: nifi-nar-bundles/nifi-gcp-bundle/pom.xml --- @@ -26,6 +26,22 @@ 1.7.0-SNAPSHOT pom + +0.46.0-alpha --- End diff -- Cool. Then I'll try to update it and build and check. If everything works, I'll update the PR. > Update 'Google Cloud SDK' version and refactor GCP processors' code > --- > > Key: NIFI-5156 > URL: https://issues.apache.org/jira/browse/NIFI-5156 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.5.0, 1.6.0 >Reporter: Sivaprasanna Sethuraman >Assignee: Sivaprasanna Sethuraman >Priority: Major > > The current version of nifi-gcp-bundle has the following problems: > # It is of a very old version > # The bundle uses > [google-cloud|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-gcp-bundle/nifi-gcp-processors/pom.xml#L65-L82] > which is a uber/fat jar that comes with all google-cloud-java SDKs making > the overall NAR bundle heavy. > The following improvements are identified and have to be made: > * Update the SDK to a more recent version > * Introduce google-cloud-bom and then use the necessary SDK (storage, > bigquery, etc) as and when needed, thus reducing the overall NAR size. > * Refactor and make necessary changes to the current version of GCS > processors, if needed because of the update -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #2671: NiFi-5102 - Adding Processors for MarkLogic DB
Github user joewitt commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2671#discussion_r186803533
--- Diff:
nifi-nar-bundles/nifi-marklogic-bundle/nifi-marklogic-processors/pom.xml ---
@@ -0,0 +1,88 @@
+
+
+http://maven.apache.org/POM/4.0.0";
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd";>
+
+4.0.0
+
+
+org.apache.nifi
+nifi-marklogic-bundle
+1.7.0-SNAPSHOT
+
+
+nifi-marklogic-processors
+
+
+
+org.apache.nifi
+nifi-api
+${nifi.version}
+
+
+org.apache.nifi
+nifi-utils
+${nifi.version}
--- End diff --
i'm editing this and other findings on a local branch. I'll share them in
a patch
---
[jira] [Commented] (NIFI-5156) Update 'Google Cloud SDK' version and refactor GCP processors' code
[ https://issues.apache.org/jira/browse/NIFI-5156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16467704#comment-16467704 ] ASF GitHub Bot commented on NIFI-5156: -- Github user pvillard31 commented on a diff in the pull request: https://github.com/apache/nifi/pull/2680#discussion_r186799348 --- Diff: nifi-nar-bundles/nifi-gcp-bundle/pom.xml --- @@ -26,6 +26,22 @@ 1.7.0-SNAPSHOT pom + +0.46.0-alpha --- End diff -- 0.47.0-alpha is now available > Update 'Google Cloud SDK' version and refactor GCP processors' code > --- > > Key: NIFI-5156 > URL: https://issues.apache.org/jira/browse/NIFI-5156 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.5.0, 1.6.0 >Reporter: Sivaprasanna Sethuraman >Assignee: Sivaprasanna Sethuraman >Priority: Major > > The current version of nifi-gcp-bundle has the following problems: > # It is of a very old version > # The bundle uses > [google-cloud|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-gcp-bundle/nifi-gcp-processors/pom.xml#L65-L82] > which is a uber/fat jar that comes with all google-cloud-java SDKs making > the overall NAR bundle heavy. > The following improvements are identified and have to be made: > * Update the SDK to a more recent version > * Introduce google-cloud-bom and then use the necessary SDK (storage, > bigquery, etc) as and when needed, thus reducing the overall NAR size. > * Refactor and make necessary changes to the current version of GCS > processors, if needed because of the update -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #2680: NIFI-5156: Updated GCP SDK to latest version
Github user pvillard31 commented on a diff in the pull request: https://github.com/apache/nifi/pull/2680#discussion_r186799348 --- Diff: nifi-nar-bundles/nifi-gcp-bundle/pom.xml --- @@ -26,6 +26,22 @@ 1.7.0-SNAPSHOT pom + +0.46.0-alpha --- End diff -- 0.47.0-alpha is now available ---
[jira] [Commented] (MINIFICPP-488) Resolve build issue with EL value
[ https://issues.apache.org/jira/browse/MINIFICPP-488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16467698#comment-16467698 ] ASF GitHub Bot commented on MINIFICPP-488: -- Github user apiri commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/324 reviewing > Resolve build issue with EL value > - > > Key: MINIFICPP-488 > URL: https://issues.apache.org/jira/browse/MINIFICPP-488 > Project: NiFi MiNiFi C++ > Issue Type: Bug >Reporter: marco polo >Assignee: marco polo >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-minifi-cpp issue #324: MINIFICPP-488: Use already implemented uint64_t ...
Github user apiri commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/324 reviewing ---
[GitHub] nifi issue #2509: NIFI-543 Added annotation to indicate processor should run...
Github user mcgilman commented on the issue: https://github.com/apache/nifi/pull/2509 @zenfenan I'm a +1 on the front end changes. I believe that @markap14 was reviewing the back end changes. Let's make sure he's good as well before it's merged. Thanks. ---
[jira] [Commented] (NIFI-543) Provide extensions a way to indicate that they can run only on primary node, if clustered
[ https://issues.apache.org/jira/browse/NIFI-543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16467695#comment-16467695 ] ASF GitHub Bot commented on NIFI-543: - Github user mcgilman commented on the issue: https://github.com/apache/nifi/pull/2509 @zenfenan I'm a +1 on the front end changes. I believe that @markap14 was reviewing the back end changes. Let's make sure he's good as well before it's merged. Thanks. > Provide extensions a way to indicate that they can run only on primary node, > if clustered > - > > Key: NIFI-543 > URL: https://issues.apache.org/jira/browse/NIFI-543 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core Framework, Documentation & Website, Extensions >Reporter: Mark Payne >Assignee: Sivaprasanna Sethuraman >Priority: Major > > There are Processors that are known to be problematic if run from multiple > nodes simultaneously. These processors should be able to use a > @PrimaryNodeOnly annotation (or something similar) to indicate that they can > be scheduled to run only on primary node if run in a cluster. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-394) Implement MQTT C2 protocol
[ https://issues.apache.org/jira/browse/MINIFICPP-394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16467684#comment-16467684 ] ASF GitHub Bot commented on MINIFICPP-394: -- Github user phrocker commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/321 @apiri https://github.com/apache/nifi-minifi-cpp/pull/324 should solve the build failure. I see I have conflicts so I'll rebase too > Implement MQTT C2 protocol > --- > > Key: MINIFICPP-394 > URL: https://issues.apache.org/jira/browse/MINIFICPP-394 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: marco polo >Priority: Major > > Implement MQTT C2 protocol as we likely won't want to use the REST protocol > long term -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-minifi-cpp issue #321: MINIFICPP-394: Implement MQTT C2 protocol along ...
Github user phrocker commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/321 @apiri https://github.com/apache/nifi-minifi-cpp/pull/324 should solve the build failure. I see I have conflicts so I'll rebase too ---
[jira] [Commented] (MINIFICPP-488) Resolve build issue with EL value
[ https://issues.apache.org/jira/browse/MINIFICPP-488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16467683#comment-16467683 ] ASF GitHub Bot commented on MINIFICPP-488: -- GitHub user phrocker opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/324 MINIFICPP-488: Use already implemented uint64_t prototype Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/phrocker/nifi-minifi-cpp MINIFICPP-488 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/324.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #324 commit 09e305047300709b9449e9d7fcbc48ee087332a0 Author: Marc Parisi Date: 2018-05-08T16:54:13Z MINIFICPP-488: Use already implemented uint64_t prototype > Resolve build issue with EL value > - > > Key: MINIFICPP-488 > URL: https://issues.apache.org/jira/browse/MINIFICPP-488 > Project: NiFi MiNiFi C++ > Issue Type: Bug >Reporter: marco polo >Assignee: marco polo >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-minifi-cpp pull request #324: MINIFICPP-488: Use already implemented ui...
GitHub user phrocker opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/324 MINIFICPP-488: Use already implemented uint64_t prototype Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/phrocker/nifi-minifi-cpp MINIFICPP-488 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/324.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #324 commit 09e305047300709b9449e9d7fcbc48ee087332a0 Author: Marc Parisi Date: 2018-05-08T16:54:13Z MINIFICPP-488: Use already implemented uint64_t prototype ---
[jira] [Created] (MINIFICPP-488) Resolve build issue with EL value
marco polo created MINIFICPP-488: Summary: Resolve build issue with EL value Key: MINIFICPP-488 URL: https://issues.apache.org/jira/browse/MINIFICPP-488 Project: NiFi MiNiFi C++ Issue Type: Bug Reporter: marco polo Assignee: marco polo -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFIREG-140) Nifi Registry not able to start - NoClassDefFoundError org/apache/nifi/registry/util/FileUtils
[
https://issues.apache.org/jira/browse/NIFIREG-140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16467662#comment-16467662
]
ASF GitHub Bot commented on NIFIREG-140:
Github user asfgit closed the pull request at:
https://github.com/apache/nifi-registry/pull/114
> Nifi Registry not able to start - NoClassDefFoundError
> org/apache/nifi/registry/util/FileUtils
> --
>
> Key: NIFIREG-140
> URL: https://issues.apache.org/jira/browse/NIFIREG-140
> Project: NiFi Registry
> Issue Type: Bug
>Affects Versions: 0.1.0
>Reporter: Gaurang Shah
>Priority: Major
> Fix For: 0.2.0
>
>
> while trying to start the nifi registry I am getting following error.
> nifi registry version: 0.1.0
>
> {code:java}
> 2018-02-06 00:11:52,665 INFO [main] org.apache.nifi.registry.NiFiRegistry
> Launching NiFi Registry...
> 2018-02-06 00:11:52,676 INFO [main] org.apache.nifi.registry.NiFiRegistry
> Read property protection key from conf/bootstrap.conf
> 2018-02-06 00:11:52,799 INFO [main] o.a.n.r.security.crypto.CryptoKeyLoader
> No encryption key present in the bootstrap.conf file at
> C:\nifi-registry-0.1.0-bin\nifi-registry-0.1.0\conf\bootstrap.conf
> 2018-02-06 00:11:52,807 INFO [main] o.a.n.r.p.NiFiRegistryPropertiesLoader
> Loaded 26 properties from
> C:\nifi-registry-0.1.0-bin\nifi-registry-0.1.0\conf\nifi-registry.properties
> 2018-02-06 00:11:52,811 INFO [main] org.apache.nifi.registry.NiFiRegistry
> Loaded 26 properties
> 2018-02-06 00:11:52,813 INFO [main] org.apache.nifi.registry.NiFiRegistry
> NiFi Registry started without Bootstrap Port information provided; will not
> listen for requests from Bootstrap
> 2018-02-06 00:11:52,820 ERROR [main] org.apache.nifi.registry.NiFiRegistry
> Failure to launch NiFi Registry due to java.lang.NoClassDefFoundError:
> org/apache/nifi/registry/util/FileUtils
> java.lang.NoClassDefFoundError: org/apache/nifi/registry/util/FileUtils
> at org.apache.nifi.registry.NiFiRegistry.(NiFiRegistry.java:97)
> ~[nifi-registry-runtime-0.1.0.jar:0.1.0]
> at org.apache.nifi.registry.NiFiRegistry.main(NiFiRegistry.java:158)
> ~[nifi-registry-runtime-0.1.0.jar:0.1.0]
> Caused by: java.lang.ClassNotFoundException:
> org.apache.nifi.registry.util.FileUtils
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381) ~[na:1.8.0_161]
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[na:1.8.0_161]
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:338)
> ~[na:1.8.0_161]
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[na:1.8.0_161]
> ... 2 common frames omitted
> 2018-02-06 00:11:52,824 INFO [Thread-1] org.apache.nifi.registry.NiFiRegistry
> Initiating shutdown of Jetty web server...
> 2018-02-06 00:11:52,824 INFO [Thread-1] org.apache.nifi.registry.NiFiRegistry
> Jetty web server shutdown completed (nicely or otherwise).
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (NIFIREG-140) Nifi Registry not able to start - NoClassDefFoundError org/apache/nifi/registry/util/FileUtils
[
https://issues.apache.org/jira/browse/NIFIREG-140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Bende resolved NIFIREG-140.
-
Resolution: Fixed
Fix Version/s: 0.2.0
> Nifi Registry not able to start - NoClassDefFoundError
> org/apache/nifi/registry/util/FileUtils
> --
>
> Key: NIFIREG-140
> URL: https://issues.apache.org/jira/browse/NIFIREG-140
> Project: NiFi Registry
> Issue Type: Bug
>Affects Versions: 0.1.0
>Reporter: Gaurang Shah
>Priority: Major
> Fix For: 0.2.0
>
>
> while trying to start the nifi registry I am getting following error.
> nifi registry version: 0.1.0
>
> {code:java}
> 2018-02-06 00:11:52,665 INFO [main] org.apache.nifi.registry.NiFiRegistry
> Launching NiFi Registry...
> 2018-02-06 00:11:52,676 INFO [main] org.apache.nifi.registry.NiFiRegistry
> Read property protection key from conf/bootstrap.conf
> 2018-02-06 00:11:52,799 INFO [main] o.a.n.r.security.crypto.CryptoKeyLoader
> No encryption key present in the bootstrap.conf file at
> C:\nifi-registry-0.1.0-bin\nifi-registry-0.1.0\conf\bootstrap.conf
> 2018-02-06 00:11:52,807 INFO [main] o.a.n.r.p.NiFiRegistryPropertiesLoader
> Loaded 26 properties from
> C:\nifi-registry-0.1.0-bin\nifi-registry-0.1.0\conf\nifi-registry.properties
> 2018-02-06 00:11:52,811 INFO [main] org.apache.nifi.registry.NiFiRegistry
> Loaded 26 properties
> 2018-02-06 00:11:52,813 INFO [main] org.apache.nifi.registry.NiFiRegistry
> NiFi Registry started without Bootstrap Port information provided; will not
> listen for requests from Bootstrap
> 2018-02-06 00:11:52,820 ERROR [main] org.apache.nifi.registry.NiFiRegistry
> Failure to launch NiFi Registry due to java.lang.NoClassDefFoundError:
> org/apache/nifi/registry/util/FileUtils
> java.lang.NoClassDefFoundError: org/apache/nifi/registry/util/FileUtils
> at org.apache.nifi.registry.NiFiRegistry.(NiFiRegistry.java:97)
> ~[nifi-registry-runtime-0.1.0.jar:0.1.0]
> at org.apache.nifi.registry.NiFiRegistry.main(NiFiRegistry.java:158)
> ~[nifi-registry-runtime-0.1.0.jar:0.1.0]
> Caused by: java.lang.ClassNotFoundException:
> org.apache.nifi.registry.util.FileUtils
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381) ~[na:1.8.0_161]
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[na:1.8.0_161]
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:338)
> ~[na:1.8.0_161]
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[na:1.8.0_161]
> ... 2 common frames omitted
> 2018-02-06 00:11:52,824 INFO [Thread-1] org.apache.nifi.registry.NiFiRegistry
> Initiating shutdown of Jetty web server...
> 2018-02-06 00:11:52,824 INFO [Thread-1] org.apache.nifi.registry.NiFiRegistry
> Jetty web server shutdown completed (nicely or otherwise).
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi-registry pull request #114: NIFIREG-140: Fix classpath for Windows
Github user asfgit closed the pull request at: https://github.com/apache/nifi-registry/pull/114 ---
[jira] [Commented] (NIFIREG-140) Nifi Registry not able to start - NoClassDefFoundError org/apache/nifi/registry/util/FileUtils
[
https://issues.apache.org/jira/browse/NIFIREG-140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16467661#comment-16467661
]
ASF GitHub Bot commented on NIFIREG-140:
Github user bbende commented on the issue:
https://github.com/apache/nifi-registry/pull/114
Looks good, going to merge, thanks for the contribution!
> Nifi Registry not able to start - NoClassDefFoundError
> org/apache/nifi/registry/util/FileUtils
> --
>
> Key: NIFIREG-140
> URL: https://issues.apache.org/jira/browse/NIFIREG-140
> Project: NiFi Registry
> Issue Type: Bug
>Affects Versions: 0.1.0
>Reporter: Gaurang Shah
>Priority: Major
>
> while trying to start the nifi registry I am getting following error.
> nifi registry version: 0.1.0
>
> {code:java}
> 2018-02-06 00:11:52,665 INFO [main] org.apache.nifi.registry.NiFiRegistry
> Launching NiFi Registry...
> 2018-02-06 00:11:52,676 INFO [main] org.apache.nifi.registry.NiFiRegistry
> Read property protection key from conf/bootstrap.conf
> 2018-02-06 00:11:52,799 INFO [main] o.a.n.r.security.crypto.CryptoKeyLoader
> No encryption key present in the bootstrap.conf file at
> C:\nifi-registry-0.1.0-bin\nifi-registry-0.1.0\conf\bootstrap.conf
> 2018-02-06 00:11:52,807 INFO [main] o.a.n.r.p.NiFiRegistryPropertiesLoader
> Loaded 26 properties from
> C:\nifi-registry-0.1.0-bin\nifi-registry-0.1.0\conf\nifi-registry.properties
> 2018-02-06 00:11:52,811 INFO [main] org.apache.nifi.registry.NiFiRegistry
> Loaded 26 properties
> 2018-02-06 00:11:52,813 INFO [main] org.apache.nifi.registry.NiFiRegistry
> NiFi Registry started without Bootstrap Port information provided; will not
> listen for requests from Bootstrap
> 2018-02-06 00:11:52,820 ERROR [main] org.apache.nifi.registry.NiFiRegistry
> Failure to launch NiFi Registry due to java.lang.NoClassDefFoundError:
> org/apache/nifi/registry/util/FileUtils
> java.lang.NoClassDefFoundError: org/apache/nifi/registry/util/FileUtils
> at org.apache.nifi.registry.NiFiRegistry.(NiFiRegistry.java:97)
> ~[nifi-registry-runtime-0.1.0.jar:0.1.0]
> at org.apache.nifi.registry.NiFiRegistry.main(NiFiRegistry.java:158)
> ~[nifi-registry-runtime-0.1.0.jar:0.1.0]
> Caused by: java.lang.ClassNotFoundException:
> org.apache.nifi.registry.util.FileUtils
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381) ~[na:1.8.0_161]
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[na:1.8.0_161]
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:338)
> ~[na:1.8.0_161]
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[na:1.8.0_161]
> ... 2 common frames omitted
> 2018-02-06 00:11:52,824 INFO [Thread-1] org.apache.nifi.registry.NiFiRegistry
> Initiating shutdown of Jetty web server...
> 2018-02-06 00:11:52,824 INFO [Thread-1] org.apache.nifi.registry.NiFiRegistry
> Jetty web server shutdown completed (nicely or otherwise).
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi-registry issue #114: NIFIREG-140: Fix classpath for Windows
Github user bbende commented on the issue: https://github.com/apache/nifi-registry/pull/114 Looks good, going to merge, thanks for the contribution! ---
[jira] [Commented] (NIFIREG-162) Add Git backed persistence provider
[ https://issues.apache.org/jira/browse/NIFIREG-162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16467636#comment-16467636 ] ASF GitHub Bot commented on NIFIREG-162: Github user kevdoran commented on the issue: https://github.com/apache/nifi-registry/pull/112 Nice work @ijokarumawak and everyone who contributed to this. This is a major feature to have as part of NiFi Registry and will certainly be useful. > Add Git backed persistence provider > --- > > Key: NIFIREG-162 > URL: https://issues.apache.org/jira/browse/NIFIREG-162 > Project: NiFi Registry > Issue Type: Improvement >Reporter: Koji Kawamura >Assignee: Koji Kawamura >Priority: Major > Fix For: 0.2.0 > > > Currently, NiFi Registry provides FileSystemFlowPersistenceProvider, which > stores Flow snapshot files into local file system. It simply manages snapshot > versions by creating directories with version numbers. > While it works, there are also demands for using Git as a version control and > persistence mechanism. > A Git backend persistence repository would be beneficial in following aspects: > * Git is a SCM (Source Control Management) that manages commits, branches, > file diffs, patches natively and provide ways to contribute and apply changes > among users > * Local and remote Git repositories can construct a distributed reliable > storage > * There are several Git repository services on the internet which can be used > as remote Git repositories those can be used as backup storages > There are few things with current NiFi Registry framework and existing > FileSystemFlowPersistenceProvider those may not be Git friendly: > * Bucket id and Flow id are UUID and not recognizable by human, if those > files have human readable names, many Git commands and tools can be used > easier. > * Current serialized Flow snapshots are binary files having header bytes and > XML encoded flow contents. If those are pure ASCII format, Git can provide > better diffs among commits, that can provide better UX in terms of > controlling Flow snapshot versions > * NiFi Registry userid which can be used as author in Git commit is not > available in FlowSnapshotContext > Also, if we are going to add another Persistence Provider implementation, we > also need to provide a way to migrate existing persisted files so that those > can be used by new one. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-registry issue #112: NIFIREG-162: Support Git backed PersistenceProvide...
Github user kevdoran commented on the issue: https://github.com/apache/nifi-registry/pull/112 Nice work @ijokarumawak and everyone who contributed to this. This is a major feature to have as part of NiFi Registry and will certainly be useful. ---
[jira] [Commented] (NIFI-5109) AbstractListProcessor stop calling performListing after primary node reelection
[ https://issues.apache.org/jira/browse/NIFI-5109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16467617#comment-16467617 ] ASF GitHub Bot commented on NIFI-5109: -- Github user joewitt commented on the issue: https://github.com/apache/nifi/pull/2657 @viazovskyi i think it is important to address that issue (ensuring we dont reset the state check until we know we've pulled latest state if available) before we move forward. > AbstractListProcessor stop calling performListing after primary node > reelection > --- > > Key: NIFI-5109 > URL: https://issues.apache.org/jira/browse/NIFI-5109 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.6.0 >Reporter: Maksym Viazovskyi >Priority: Major > Attachments: > 0001-NIFI-2109-Reset-justElectedPrimaryNode-flag-right-af.patch, > PutAndListSFTP.xml, docker-compose.yml > > Original Estimate: 72h > Remaining Estimate: 72h > > AbstractListProcessor stops performing listing of entities when after > reelection the same node becomes Primary. > STEPS TO REPRODUCE: > 0. Make sure that you have docker 18.03.0-ce > 1. Download attached [^docker-compose.yml] and [^PutAndListSFTP.xml] > 2. Run *_docker-compose up -d_* in the folder where you downloaded > docker-compose.yml. This command will run 3 containers - Zookeeper, NiFi > 1.6.0 in Cluster Mode with 1 node, SFTPs server > 3. After NiFi is up and running (check http://localhost:8080/nifi) upload > PutAndListSFTP.xml template and place it on NiFi canvas. It has simple logic > - generate flow file each 10 seconds and put it on SFTP, in parallel > ListSFTP processor "listens" for the new files and pushes data about them to > NiFi log with LogAttribute processor > 4. Update PutSFTP and ListSFTP processors with password admin (to check SFTP > container use following credentials SFTP://admin:admin@localhost:) > 5. Run the flow > 6. Check the NiFi log (*_docker logs -f --tail 100 nifi_*) - you should see > attributes for the uploaded files on SFTP > 7. Check that generated each 10 seconds files appear on SFTP /upload folder > 8. Run following command to restart Zookeeper to put NiFi in reelection > state: *_docker restart zookeeper_* > 9. After reelection phase finished check that new files appears on SFTP, but > NO new files info appears in NiFi log > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2657: NIFI-5109 Reset justElectedPrimaryNode flag right after re...
Github user joewitt commented on the issue: https://github.com/apache/nifi/pull/2657 @viazovskyi i think it is important to address that issue (ensuring we dont reset the state check until we know we've pulled latest state if available) before we move forward. ---
[jira] [Commented] (NIFI-543) Provide extensions a way to indicate that they can run only on primary node, if clustered
[ https://issues.apache.org/jira/browse/NIFI-543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16467612#comment-16467612 ] ASF GitHub Bot commented on NIFI-543: - Github user zenfenan commented on the issue: https://github.com/apache/nifi/pull/2509 @mcgilman Ready to be merged? > Provide extensions a way to indicate that they can run only on primary node, > if clustered > - > > Key: NIFI-543 > URL: https://issues.apache.org/jira/browse/NIFI-543 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core Framework, Documentation & Website, Extensions >Reporter: Mark Payne >Assignee: Sivaprasanna Sethuraman >Priority: Major > > There are Processors that are known to be problematic if run from multiple > nodes simultaneously. These processors should be able to use a > @PrimaryNodeOnly annotation (or something similar) to indicate that they can > be scheduled to run only on primary node if run in a cluster. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-minifi-cpp pull request #323: MINFICPP-487: Change lib minifi version t...
Github user asfgit closed the pull request at: https://github.com/apache/nifi-minifi-cpp/pull/323 ---
[GitHub] nifi issue #2509: NIFI-543 Added annotation to indicate processor should run...
Github user zenfenan commented on the issue: https://github.com/apache/nifi/pull/2509 @mcgilman Ready to be merged? ---
[jira] [Resolved] (NIFIREG-155) Cannot access nifi once secure
[ https://issues.apache.org/jira/browse/NIFIREG-155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Bende resolved NIFIREG-155. - Resolution: Fixed Assignee: Bryan Bende Closing this for now. If it is believed to be a real issue then please re-open. > Cannot access nifi once secure > -- > > Key: NIFIREG-155 > URL: https://issues.apache.org/jira/browse/NIFIREG-155 > Project: NiFi Registry > Issue Type: Bug >Affects Versions: 0.1.0 > Environment: Amazon Linux 2 LTS > OpenJDK 1.8.0_161-b14 > T2.small >Reporter: Scott Howell >Assignee: Bryan Bende >Priority: Minor > > I have setup nifi-registry unsecure and was able to access the UI. When I > switched to a secured instances of nifii-registry the ui is no longer > available. There are no errors in the nifi-registry-app.log. > > I am running this through an ELB on AWS. I was able to use this when it was > unsecure but not when running securely. I am also not able to see the > healthcheck hitting nifi-registry like I do when looking at my nifi > instances. > > Since this project is so new I figured I would reach out and see if this > capability doesn't work yet or if I am doing something wrong. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFIREG-162) Add Git backed persistence provider
[ https://issues.apache.org/jira/browse/NIFIREG-162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16467600#comment-16467600 ] ASF GitHub Bot commented on NIFIREG-162: Github user asfgit closed the pull request at: https://github.com/apache/nifi-registry/pull/112 > Add Git backed persistence provider > --- > > Key: NIFIREG-162 > URL: https://issues.apache.org/jira/browse/NIFIREG-162 > Project: NiFi Registry > Issue Type: Improvement >Reporter: Koji Kawamura >Assignee: Koji Kawamura >Priority: Major > Fix For: 0.2.0 > > > Currently, NiFi Registry provides FileSystemFlowPersistenceProvider, which > stores Flow snapshot files into local file system. It simply manages snapshot > versions by creating directories with version numbers. > While it works, there are also demands for using Git as a version control and > persistence mechanism. > A Git backend persistence repository would be beneficial in following aspects: > * Git is a SCM (Source Control Management) that manages commits, branches, > file diffs, patches natively and provide ways to contribute and apply changes > among users > * Local and remote Git repositories can construct a distributed reliable > storage > * There are several Git repository services on the internet which can be used > as remote Git repositories those can be used as backup storages > There are few things with current NiFi Registry framework and existing > FileSystemFlowPersistenceProvider those may not be Git friendly: > * Bucket id and Flow id are UUID and not recognizable by human, if those > files have human readable names, many Git commands and tools can be used > easier. > * Current serialized Flow snapshots are binary files having header bytes and > XML encoded flow contents. If those are pure ASCII format, Git can provide > better diffs among commits, that can provide better UX in terms of > controlling Flow snapshot versions > * NiFi Registry userid which can be used as author in Git commit is not > available in FlowSnapshotContext > Also, if we are going to add another Persistence Provider implementation, we > also need to provide a way to migrate existing persisted files so that those > can be used by new one. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (NIFIREG-162) Add Git backed persistence provider
[ https://issues.apache.org/jira/browse/NIFIREG-162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Bende resolved NIFIREG-162. - Resolution: Fixed Fix Version/s: 0.2.0 > Add Git backed persistence provider > --- > > Key: NIFIREG-162 > URL: https://issues.apache.org/jira/browse/NIFIREG-162 > Project: NiFi Registry > Issue Type: Improvement >Reporter: Koji Kawamura >Assignee: Koji Kawamura >Priority: Major > Fix For: 0.2.0 > > > Currently, NiFi Registry provides FileSystemFlowPersistenceProvider, which > stores Flow snapshot files into local file system. It simply manages snapshot > versions by creating directories with version numbers. > While it works, there are also demands for using Git as a version control and > persistence mechanism. > A Git backend persistence repository would be beneficial in following aspects: > * Git is a SCM (Source Control Management) that manages commits, branches, > file diffs, patches natively and provide ways to contribute and apply changes > among users > * Local and remote Git repositories can construct a distributed reliable > storage > * There are several Git repository services on the internet which can be used > as remote Git repositories those can be used as backup storages > There are few things with current NiFi Registry framework and existing > FileSystemFlowPersistenceProvider those may not be Git friendly: > * Bucket id and Flow id are UUID and not recognizable by human, if those > files have human readable names, many Git commands and tools can be used > easier. > * Current serialized Flow snapshots are binary files having header bytes and > XML encoded flow contents. If those are pure ASCII format, Git can provide > better diffs among commits, that can provide better UX in terms of > controlling Flow snapshot versions > * NiFi Registry userid which can be used as author in Git commit is not > available in FlowSnapshotContext > Also, if we are going to add another Persistence Provider implementation, we > also need to provide a way to migrate existing persisted files so that those > can be used by new one. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-registry pull request #112: NIFIREG-162: Support Git backed Persistence...
Github user asfgit closed the pull request at: https://github.com/apache/nifi-registry/pull/112 ---
[jira] [Commented] (NIFIREG-162) Add Git backed persistence provider
[ https://issues.apache.org/jira/browse/NIFIREG-162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16467584#comment-16467584 ] ASF GitHub Bot commented on NIFIREG-162: Github user bbende commented on the issue: https://github.com/apache/nifi-registry/pull/112 @ijokarumawak thanks for the updates, everything looks good, going to merge... thanks for all the work on this! > Add Git backed persistence provider > --- > > Key: NIFIREG-162 > URL: https://issues.apache.org/jira/browse/NIFIREG-162 > Project: NiFi Registry > Issue Type: Improvement >Reporter: Koji Kawamura >Assignee: Koji Kawamura >Priority: Major > > Currently, NiFi Registry provides FileSystemFlowPersistenceProvider, which > stores Flow snapshot files into local file system. It simply manages snapshot > versions by creating directories with version numbers. > While it works, there are also demands for using Git as a version control and > persistence mechanism. > A Git backend persistence repository would be beneficial in following aspects: > * Git is a SCM (Source Control Management) that manages commits, branches, > file diffs, patches natively and provide ways to contribute and apply changes > among users > * Local and remote Git repositories can construct a distributed reliable > storage > * There are several Git repository services on the internet which can be used > as remote Git repositories those can be used as backup storages > There are few things with current NiFi Registry framework and existing > FileSystemFlowPersistenceProvider those may not be Git friendly: > * Bucket id and Flow id are UUID and not recognizable by human, if those > files have human readable names, many Git commands and tools can be used > easier. > * Current serialized Flow snapshots are binary files having header bytes and > XML encoded flow contents. If those are pure ASCII format, Git can provide > better diffs among commits, that can provide better UX in terms of > controlling Flow snapshot versions > * NiFi Registry userid which can be used as author in Git commit is not > available in FlowSnapshotContext > Also, if we are going to add another Persistence Provider implementation, we > also need to provide a way to migrate existing persisted files so that those > can be used by new one. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-registry issue #112: NIFIREG-162: Support Git backed PersistenceProvide...
Github user bbende commented on the issue: https://github.com/apache/nifi-registry/pull/112 @ijokarumawak thanks for the updates, everything looks good, going to merge... thanks for all the work on this! ---
[jira] [Commented] (NIFI-5166) Create deep learning classification and regression processor
[ https://issues.apache.org/jira/browse/NIFI-5166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16467567#comment-16467567 ] ASF GitHub Bot commented on NIFI-5166: -- Github user mans2singh commented on the issue: https://github.com/apache/nifi/pull/2686 Good Morning Nifi Folks: The appveyor build is passing for this PR but travis-ci build is failing with the following message: `[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.20.1:test (default-test) on project nifi-cdc-mysql-processors: There are test failures. [ERROR] [ERROR] Please refer to /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-cdc/nifi-cdc-mysql-bundle/nifi-cdc-mysql-processors/target/surefire-reports for the individual test results. [ERROR] Please refer to dump files (if any exist) [date]-jvmRun[N].dump, [date].dumpstream and [date]-jvmRun[N].dumpstream.` Can you please advice on how to resolve this error ? Thanks Mans > Create deep learning classification and regression processor > > > Key: NIFI-5166 > URL: https://issues.apache.org/jira/browse/NIFI-5166 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Affects Versions: 1.6.0 >Reporter: Mans Singh >Assignee: Mans Singh >Priority: Minor > Labels: Learning, classification,, deep, regression, > Fix For: 1.7.0 > > Original Estimate: 168h > Remaining Estimate: 168h > > We need a deep learning classification and regression processor. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2686: NIFI-5166 - Deep learning classification and regression pr...
Github user mans2singh commented on the issue: https://github.com/apache/nifi/pull/2686 Good Morning Nifi Folks: The appveyor build is passing for this PR but travis-ci build is failing with the following message: `[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.20.1:test (default-test) on project nifi-cdc-mysql-processors: There are test failures. [ERROR] [ERROR] Please refer to /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-cdc/nifi-cdc-mysql-bundle/nifi-cdc-mysql-processors/target/surefire-reports for the individual test results. [ERROR] Please refer to dump files (if any exist) [date]-jvmRun[N].dump, [date].dumpstream and [date]-jvmRun[N].dumpstream.` Can you please advice on how to resolve this error ? Thanks Mans ---
[jira] [Commented] (NIFI-5109) AbstractListProcessor stop calling performListing after primary node reelection
[ https://issues.apache.org/jira/browse/NIFI-5109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16467539#comment-16467539 ] ASF GitHub Bot commented on NIFI-5109: -- Github user viazovskyi commented on the issue: https://github.com/apache/nifi/pull/2657 @joewitt Hi Joe, is there any update? > AbstractListProcessor stop calling performListing after primary node > reelection > --- > > Key: NIFI-5109 > URL: https://issues.apache.org/jira/browse/NIFI-5109 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.6.0 >Reporter: Maksym Viazovskyi >Priority: Major > Attachments: > 0001-NIFI-2109-Reset-justElectedPrimaryNode-flag-right-af.patch, > PutAndListSFTP.xml, docker-compose.yml > > Original Estimate: 72h > Remaining Estimate: 72h > > AbstractListProcessor stops performing listing of entities when after > reelection the same node becomes Primary. > STEPS TO REPRODUCE: > 0. Make sure that you have docker 18.03.0-ce > 1. Download attached [^docker-compose.yml] and [^PutAndListSFTP.xml] > 2. Run *_docker-compose up -d_* in the folder where you downloaded > docker-compose.yml. This command will run 3 containers - Zookeeper, NiFi > 1.6.0 in Cluster Mode with 1 node, SFTPs server > 3. After NiFi is up and running (check http://localhost:8080/nifi) upload > PutAndListSFTP.xml template and place it on NiFi canvas. It has simple logic > - generate flow file each 10 seconds and put it on SFTP, in parallel > ListSFTP processor "listens" for the new files and pushes data about them to > NiFi log with LogAttribute processor > 4. Update PutSFTP and ListSFTP processors with password admin (to check SFTP > container use following credentials SFTP://admin:admin@localhost:) > 5. Run the flow > 6. Check the NiFi log (*_docker logs -f --tail 100 nifi_*) - you should see > attributes for the uploaded files on SFTP > 7. Check that generated each 10 seconds files appear on SFTP /upload folder > 8. Run following command to restart Zookeeper to put NiFi in reelection > state: *_docker restart zookeeper_* > 9. After reelection phase finished check that new files appears on SFTP, but > NO new files info appears in NiFi log > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2657: NIFI-5109 Reset justElectedPrimaryNode flag right after re...
Github user viazovskyi commented on the issue: https://github.com/apache/nifi/pull/2657 @joewitt Hi Joe, is there any update? ---
[jira] [Resolved] (NIFIREG-171) Upgrade moment.js
[ https://issues.apache.org/jira/browse/NIFIREG-171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman resolved NIFIREG-171. - Resolution: Fixed Fix Version/s: 0.2.0 > Upgrade moment.js > - > > Key: NIFIREG-171 > URL: https://issues.apache.org/jira/browse/NIFIREG-171 > Project: NiFi Registry > Issue Type: Improvement >Reporter: Scott Aslan >Assignee: Scott Aslan >Priority: Major > Fix For: 0.2.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (NIFI-5167) Reporting Task Controller Service UI failure
[ https://issues.apache.org/jira/browse/NIFI-5167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman updated NIFI-5167: -- Status: Patch Available (was: Open) > Reporting Task Controller Service UI failure > > > Key: NIFI-5167 > URL: https://issues.apache.org/jira/browse/NIFI-5167 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.6.0 >Reporter: Mark Bean >Assignee: Matt Gilman >Priority: Major > > The UI for Controller Services related to a Reporting Task can become > non-responsive. Test case: Create a Reporting Task which requires a > Controller Service (e.g. SiteToSiteBulletinReportingTask.) When configuring > the properties for the Reporting Task, select the Controller Service property > (e.g. SSL Context Service) and choose "create new service". Then, configure > the resultant Controller Service (e.g. StandardRestrictedSSLContextService.) > When choosing "Apply", the properties popup window is not dismissed. Yet, the > properties appear to apply successfully because on subsequent configuration, > the properties remain as previously set. Also, a message in the app.log > indicates: > "INFO [Flow Service Tasks Thread-2] o.a.nifi.controller.StandardFlowService > Saved flow controller org.apache.nifi.controller.FlowController@443b57d7 // > Another save pending = false" > The Controller Service configuration window should dismiss appropriately when > "Apply" button is selected. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5167) Reporting Task Controller Service UI failure
[ https://issues.apache.org/jira/browse/NIFI-5167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16467529#comment-16467529 ] ASF GitHub Bot commented on NIFI-5167: -- Github user scottyaslan commented on the issue: https://github.com/apache/nifi/pull/2688 Will review... > Reporting Task Controller Service UI failure > > > Key: NIFI-5167 > URL: https://issues.apache.org/jira/browse/NIFI-5167 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.6.0 >Reporter: Mark Bean >Assignee: Matt Gilman >Priority: Major > > The UI for Controller Services related to a Reporting Task can become > non-responsive. Test case: Create a Reporting Task which requires a > Controller Service (e.g. SiteToSiteBulletinReportingTask.) When configuring > the properties for the Reporting Task, select the Controller Service property > (e.g. SSL Context Service) and choose "create new service". Then, configure > the resultant Controller Service (e.g. StandardRestrictedSSLContextService.) > When choosing "Apply", the properties popup window is not dismissed. Yet, the > properties appear to apply successfully because on subsequent configuration, > the properties remain as previously set. Also, a message in the app.log > indicates: > "INFO [Flow Service Tasks Thread-2] o.a.nifi.controller.StandardFlowService > Saved flow controller org.apache.nifi.controller.FlowController@443b57d7 // > Another save pending = false" > The Controller Service configuration window should dismiss appropriately when > "Apply" button is selected. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5167) Reporting Task Controller Service UI failure
[ https://issues.apache.org/jira/browse/NIFI-5167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16467527#comment-16467527 ] ASF GitHub Bot commented on NIFI-5167: -- GitHub user mcgilman opened a pull request: https://github.com/apache/nifi/pull/2688 NIFI-5167: Updating how the nf-reporting-task module is injected to the nf-controller-service module NIFI-5167: - Updating how the nf-reporting-task module is injected to the nf-controller-service module. Steps to reproduce the issue is in the JIRA. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mcgilman/nifi NIFI-5167 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2688.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2688 commit 00ae0f39559065f1fa4b3e6daf672e57a43c5842 Author: Matt Gilman Date: 2018-05-08T15:00:43Z NIFI-5167: - Updating how the nf-reporting-task module is injected to the nf-controller-service module. > Reporting Task Controller Service UI failure > > > Key: NIFI-5167 > URL: https://issues.apache.org/jira/browse/NIFI-5167 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.6.0 >Reporter: Mark Bean >Assignee: Matt Gilman >Priority: Major > > The UI for Controller Services related to a Reporting Task can become > non-responsive. Test case: Create a Reporting Task which requires a > Controller Service (e.g. SiteToSiteBulletinReportingTask.) When configuring > the properties for the Reporting Task, select the Controller Service property > (e.g. SSL Context Service) and choose "create new service". Then, configure > the resultant Controller Service (e.g. StandardRestrictedSSLContextService.) > When choosing "Apply", the properties popup window is not dismissed. Yet, the > properties appear to apply successfully because on subsequent configuration, > the properties remain as previously set. Also, a message in the app.log > indicates: > "INFO [Flow Service Tasks Thread-2] o.a.nifi.controller.StandardFlowService > Saved flow controller org.apache.nifi.controller.FlowController@443b57d7 // > Another save pending = false" > The Controller Service configuration window should dismiss appropriately when > "Apply" button is selected. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2688: NIFI-5167: Updating how the nf-reporting-task module is in...
Github user scottyaslan commented on the issue: https://github.com/apache/nifi/pull/2688 Will review... ---
[GitHub] nifi pull request #2688: NIFI-5167: Updating how the nf-reporting-task modul...
GitHub user mcgilman opened a pull request: https://github.com/apache/nifi/pull/2688 NIFI-5167: Updating how the nf-reporting-task module is injected to the nf-controller-service module NIFI-5167: - Updating how the nf-reporting-task module is injected to the nf-controller-service module. Steps to reproduce the issue is in the JIRA. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mcgilman/nifi NIFI-5167 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2688.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2688 commit 00ae0f39559065f1fa4b3e6daf672e57a43c5842 Author: Matt Gilman Date: 2018-05-08T15:00:43Z NIFI-5167: - Updating how the nf-reporting-task module is injected to the nf-controller-service module. ---
[jira] [Commented] (NIFI-5167) Reporting Task Controller Service UI failure
[ https://issues.apache.org/jira/browse/NIFI-5167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16467518#comment-16467518 ] Matt Gilman commented on NIFI-5167: --- Thanks for the extra details. Yeah, we just need to adjust how the modules are injected. > Reporting Task Controller Service UI failure > > > Key: NIFI-5167 > URL: https://issues.apache.org/jira/browse/NIFI-5167 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.6.0 >Reporter: Mark Bean >Assignee: Matt Gilman >Priority: Major > > The UI for Controller Services related to a Reporting Task can become > non-responsive. Test case: Create a Reporting Task which requires a > Controller Service (e.g. SiteToSiteBulletinReportingTask.) When configuring > the properties for the Reporting Task, select the Controller Service property > (e.g. SSL Context Service) and choose "create new service". Then, configure > the resultant Controller Service (e.g. StandardRestrictedSSLContextService.) > When choosing "Apply", the properties popup window is not dismissed. Yet, the > properties appear to apply successfully because on subsequent configuration, > the properties remain as previously set. Also, a message in the app.log > indicates: > "INFO [Flow Service Tasks Thread-2] o.a.nifi.controller.StandardFlowService > Saved flow controller org.apache.nifi.controller.FlowController@443b57d7 // > Another save pending = false" > The Controller Service configuration window should dismiss appropriately when > "Apply" button is selected. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5167) Reporting Task Controller Service UI failure
[ https://issues.apache.org/jira/browse/NIFI-5167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16467515#comment-16467515 ] Mark Bean commented on NIFI-5167: - [~mcgilman] Sounds like you're on track, but further investigation by a co-worker shows that there is a circular dependency between controller service and reporting tasks in the javascript. Basically, the controller service is trying to reload the reporting task which has not yet been initialized. I can get additional detail from him if needed. > Reporting Task Controller Service UI failure > > > Key: NIFI-5167 > URL: https://issues.apache.org/jira/browse/NIFI-5167 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.6.0 >Reporter: Mark Bean >Assignee: Matt Gilman >Priority: Major > > The UI for Controller Services related to a Reporting Task can become > non-responsive. Test case: Create a Reporting Task which requires a > Controller Service (e.g. SiteToSiteBulletinReportingTask.) When configuring > the properties for the Reporting Task, select the Controller Service property > (e.g. SSL Context Service) and choose "create new service". Then, configure > the resultant Controller Service (e.g. StandardRestrictedSSLContextService.) > When choosing "Apply", the properties popup window is not dismissed. Yet, the > properties appear to apply successfully because on subsequent configuration, > the properties remain as previously set. Also, a message in the app.log > indicates: > "INFO [Flow Service Tasks Thread-2] o.a.nifi.controller.StandardFlowService > Saved flow controller org.apache.nifi.controller.FlowController@443b57d7 // > Another save pending = false" > The Controller Service configuration window should dismiss appropriately when > "Apply" button is selected. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (NIFI-5167) Reporting Task Controller Service UI failure
[ https://issues.apache.org/jira/browse/NIFI-5167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman reassigned NIFI-5167: - Assignee: Matt Gilman > Reporting Task Controller Service UI failure > > > Key: NIFI-5167 > URL: https://issues.apache.org/jira/browse/NIFI-5167 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.6.0 >Reporter: Mark Bean >Assignee: Matt Gilman >Priority: Major > > The UI for Controller Services related to a Reporting Task can become > non-responsive. Test case: Create a Reporting Task which requires a > Controller Service (e.g. SiteToSiteBulletinReportingTask.) When configuring > the properties for the Reporting Task, select the Controller Service property > (e.g. SSL Context Service) and choose "create new service". Then, configure > the resultant Controller Service (e.g. StandardRestrictedSSLContextService.) > When choosing "Apply", the properties popup window is not dismissed. Yet, the > properties appear to apply successfully because on subsequent configuration, > the properties remain as previously set. Also, a message in the app.log > indicates: > "INFO [Flow Service Tasks Thread-2] o.a.nifi.controller.StandardFlowService > Saved flow controller org.apache.nifi.controller.FlowController@443b57d7 // > Another save pending = false" > The Controller Service configuration window should dismiss appropriately when > "Apply" button is selected. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5167) Reporting Task Controller Service UI failure
[ https://issues.apache.org/jira/browse/NIFI-5167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16467509#comment-16467509 ] Matt Gilman commented on NIFI-5167: --- [~markbean] [~mosermw] Thanks guys! I just replicated locally. Will be posting a PR shortly. > Reporting Task Controller Service UI failure > > > Key: NIFI-5167 > URL: https://issues.apache.org/jira/browse/NIFI-5167 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.6.0 >Reporter: Mark Bean >Priority: Major > > The UI for Controller Services related to a Reporting Task can become > non-responsive. Test case: Create a Reporting Task which requires a > Controller Service (e.g. SiteToSiteBulletinReportingTask.) When configuring > the properties for the Reporting Task, select the Controller Service property > (e.g. SSL Context Service) and choose "create new service". Then, configure > the resultant Controller Service (e.g. StandardRestrictedSSLContextService.) > When choosing "Apply", the properties popup window is not dismissed. Yet, the > properties appear to apply successfully because on subsequent configuration, > the properties remain as previously set. Also, a message in the app.log > indicates: > "INFO [Flow Service Tasks Thread-2] o.a.nifi.controller.StandardFlowService > Saved flow controller org.apache.nifi.controller.FlowController@443b57d7 // > Another save pending = false" > The Controller Service configuration window should dismiss appropriately when > "Apply" button is selected. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5167) Reporting Task Controller Service UI failure
[ https://issues.apache.org/jira/browse/NIFI-5167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16467506#comment-16467506 ] Michael Moser commented on NIFI-5167: - More information ... this produces a Javascript error that reads: Uncaught TypeError: Cannot read property 'reload' of undefined at Object. (nf-canvas-all.js?1.6.0:53) > Reporting Task Controller Service UI failure > > > Key: NIFI-5167 > URL: https://issues.apache.org/jira/browse/NIFI-5167 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.6.0 >Reporter: Mark Bean >Priority: Major > > The UI for Controller Services related to a Reporting Task can become > non-responsive. Test case: Create a Reporting Task which requires a > Controller Service (e.g. SiteToSiteBulletinReportingTask.) When configuring > the properties for the Reporting Task, select the Controller Service property > (e.g. SSL Context Service) and choose "create new service". Then, configure > the resultant Controller Service (e.g. StandardRestrictedSSLContextService.) > When choosing "Apply", the properties popup window is not dismissed. Yet, the > properties appear to apply successfully because on subsequent configuration, > the properties remain as previously set. Also, a message in the app.log > indicates: > "INFO [Flow Service Tasks Thread-2] o.a.nifi.controller.StandardFlowService > Saved flow controller org.apache.nifi.controller.FlowController@443b57d7 // > Another save pending = false" > The Controller Service configuration window should dismiss appropriately when > "Apply" button is selected. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFIREG-171) Upgrade moment.js
[ https://issues.apache.org/jira/browse/NIFIREG-171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16467468#comment-16467468 ] ASF GitHub Bot commented on NIFIREG-171: Github user asfgit closed the pull request at: https://github.com/apache/nifi-registry/pull/116 > Upgrade moment.js > - > > Key: NIFIREG-171 > URL: https://issues.apache.org/jira/browse/NIFIREG-171 > Project: NiFi Registry > Issue Type: Improvement >Reporter: Scott Aslan >Assignee: Scott Aslan >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-registry pull request #116: [NIFIREG-171] update moment.js
Github user asfgit closed the pull request at: https://github.com/apache/nifi-registry/pull/116 ---
[jira] [Commented] (NIFI-5041) Add convenient SPNEGO/Kerberos authentication support to LivySessionController
[
https://issues.apache.org/jira/browse/NIFI-5041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16467452#comment-16467452
]
ASF GitHub Bot commented on NIFI-5041:
--
Github user peter-toth commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2630#discussion_r186738068
--- Diff:
nifi-nar-bundles/nifi-extension-utils/nifi-hadoop-utils/src/main/java/org/apache/nifi/hadoop/KerberosConfiguration.java
---
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.hadoop;
+
+import org.apache.hadoop.security.authentication.util.KerberosUtil;
+
+import javax.security.auth.login.AppConfigurationEntry;
+import java.util.HashMap;
+import java.util.Map;
+
+/**
+ * Modified Kerberos configuration class from {@link
org.apache.hadoop.security.authentication.client.KerberosAuthenticator.KerberosConfiguration}
+ * that requires authentication from a keytab.
+ */
+public class KerberosConfiguration extends
javax.security.auth.login.Configuration {
--- End diff --
@mattyb149, @joewitt can you please help me and review this PR and let me
know if I need to change anything.
> Add convenient SPNEGO/Kerberos authentication support to LivySessionController
> --
>
> Key: NIFI-5041
> URL: https://issues.apache.org/jira/browse/NIFI-5041
> Project: Apache NiFi
> Issue Type: Improvement
>Affects Versions: 1.5.0
>Reporter: Peter Toth
>Priority: Minor
>
> Livy requires SPNEGO/Kerberos authentication on a secured cluster. Initiating
> such an authentication from NiFi is a viable by providing a
> java.security.auth.login.config system property
> (https://docs.oracle.com/javase/8/docs/technotes/guides/security/jgss/lab/part6.html),
> but this is a bit cumbersome and needs kinit running outside of NiFi.
> An alternative and more sophisticated solution would be to do the SPNEGO
> negotiation programmatically.
> * This solution would add some new properties to the LivySessionController
> to fetch kerberos principal and password/keytab
> * Add the required HTTP Negotiate header (with an SPNEGO token) to the
> HttpURLConnection to do the authentication programmatically
> (https://tools.ietf.org/html/rfc4559)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi pull request #2630: NIFI-5041 Adds SPNEGO authentication to LivySession...
Github user peter-toth commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2630#discussion_r186738068
--- Diff:
nifi-nar-bundles/nifi-extension-utils/nifi-hadoop-utils/src/main/java/org/apache/nifi/hadoop/KerberosConfiguration.java
---
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.hadoop;
+
+import org.apache.hadoop.security.authentication.util.KerberosUtil;
+
+import javax.security.auth.login.AppConfigurationEntry;
+import java.util.HashMap;
+import java.util.Map;
+
+/**
+ * Modified Kerberos configuration class from {@link
org.apache.hadoop.security.authentication.client.KerberosAuthenticator.KerberosConfiguration}
+ * that requires authentication from a keytab.
+ */
+public class KerberosConfiguration extends
javax.security.auth.login.Configuration {
--- End diff --
@mattyb149, @joewitt can you please help me and review this PR and let me
know if I need to change anything.
---
[jira] [Resolved] (MINIFICPP-463) Implement escape/unescape URL EL functions
[ https://issues.apache.org/jira/browse/MINIFICPP-463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Christianson resolved MINIFICPP-463. --- Resolution: Fixed > Implement escape/unescape URL EL functions > -- > > Key: MINIFICPP-463 > URL: https://issues.apache.org/jira/browse/MINIFICPP-463 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Andrew Christianson >Assignee: Andrew Christianson >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (MINIFICPP-484) Update sol2 to 2.20.0
[ https://issues.apache.org/jira/browse/MINIFICPP-484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Christianson resolved MINIFICPP-484. --- Resolution: Fixed > Update sol2 to 2.20.0 > - > > Key: MINIFICPP-484 > URL: https://issues.apache.org/jira/browse/MINIFICPP-484 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Andrew Christianson >Assignee: Andrew Christianson >Priority: Major > > The sol2-2.17.5 library is outdated and should be updated to the latest, > 2.20.0. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (MINIFICPP-475) Allow preferred image dimensions/mode to be set in GetUSBCamera properties
[ https://issues.apache.org/jira/browse/MINIFICPP-475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Christianson resolved MINIFICPP-475. --- Resolution: Fixed > Allow preferred image dimensions/mode to be set in GetUSBCamera properties > -- > > Key: MINIFICPP-475 > URL: https://issues.apache.org/jira/browse/MINIFICPP-475 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Andrew Christianson >Assignee: Andrew Christianson >Priority: Major > > GetUSBCamera currently selects the highest-quality image format for a given > FPS. This optimizes for image quality, but can be suboptimal for performance > on embedded devices where users may need to have low FPS and low/small image > quality. > Add additional optional properties to GetUSBCamera to allow specification of > preferred image dimensions/quality, and have this override automatic > selection if the properties are set. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
