[jira] [Resolved] (NIFI-11227) Parameter context description not saved
[ https://issues.apache.org/jira/browse/NIFI-11227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael W Moser resolved NIFI-11227. Resolution: Duplicate > Parameter context description not saved > --- > > Key: NIFI-11227 > URL: https://issues.apache.org/jira/browse/NIFI-11227 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.20.0 >Reporter: Julien G. >Priority: Major > Labels: beginner, easy-fix, newbie > > When restarting NiFi, the description of all parameter contexts is deleted. > This should not be the case. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (NIFI-13547) UpdateAttribute Advanced Configuration does not recognize Parameters
[
https://issues.apache.org/jira/browse/NIFI-13547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17866104#comment-17866104
]
Michael W Moser edited comment on NIFI-13547 at 7/15/24 3:47 PM:
-
Hi [~christof] I see your example uses #replacement_parameter, but did you try
#\{replacement_parameter}?
was (Author: mosermw):
Hi [~christof] I see your example uses #replacement_parameter, but did you try
#\{replacement_parameter}? You still use Expression Language to access
parameters.
> UpdateAttribute Advanced Configuration does not recognize Parameters
>
>
> Key: NIFI-13547
> URL: https://issues.apache.org/jira/browse/NIFI-13547
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.21.0, 1.23.2
>Reporter: Christof Dilcher
>Priority: Minor
> Attachments: Parameter_Test.json.zip,
> direct_use_of_parameter_in_UpdateAttribute.jpg,
> parameter_used_in_expression_language_in_UpdateAttribute.jpg
>
>
> When using UpdateAttribute Advanced Configuration and configuring a rule, a
> parameter value (from the parameter context) is not directly accessible. When
> referencing a parameter in the "Value" section of the "Action" part of the
> "Rule" setup (see screenshot direct_use_of_parameter_in_UpdateAttribute.jpg),
> it is instead interpreted literally and not replaced with the parameter value
> as would be expected.
> The value can be accessed however when the parameter is referenced within an
> expression language term (see screenshot
> parameter_used_in_expression_language_in_UpdateAttribute.jpg).
> IMHO the parameter value shall be directly accessible within this
> configuration. If this is intended behavior, this shall be mentioned in the
> documentation as it is not obvious to the user that in this place the
> expression language has to be used in order to access the parameter value.
> I have attached the flow definition to this ticket (see
> Parameter_Test.json.zip) for reference. The corresponding parameter context
> just contains one parameter named "replacement_parameter" with its value
> 'some replacement value'.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-13547) UpdateAttribute Advanced Configuration does not recognize Parameters
[
https://issues.apache.org/jira/browse/NIFI-13547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17866104#comment-17866104
]
Michael W Moser commented on NIFI-13547:
Hi [~christof] I see your example uses #replacement_parameter, but did you try
#\{replacement_parameter}? You still use Expression Language to access
parameters.
> UpdateAttribute Advanced Configuration does not recognize Parameters
>
>
> Key: NIFI-13547
> URL: https://issues.apache.org/jira/browse/NIFI-13547
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.21.0, 1.23.2
>Reporter: Christof Dilcher
>Priority: Minor
> Attachments: Parameter_Test.json.zip,
> direct_use_of_parameter_in_UpdateAttribute.jpg,
> parameter_used_in_expression_language_in_UpdateAttribute.jpg
>
>
> When using UpdateAttribute Advanced Configuration and configuring a rule, a
> parameter value (from the parameter context) is not directly accessible. When
> referencing a parameter in the "Value" section of the "Action" part of the
> "Rule" setup (see screenshot direct_use_of_parameter_in_UpdateAttribute.jpg),
> it is instead interpreted literally and not replaced with the parameter value
> as would be expected.
> The value can be accessed however when the parameter is referenced within an
> expression language term (see screenshot
> parameter_used_in_expression_language_in_UpdateAttribute.jpg).
> IMHO the parameter value shall be directly accessible within this
> configuration. If this is intended behavior, this shall be mentioned in the
> documentation as it is not obvious to the user that in this place the
> expression language has to be used in order to access the parameter value.
> I have attached the flow definition to this ticket (see
> Parameter_Test.json.zip) for reference. The corresponding parameter context
> just contains one parameter named "replacement_parameter" with its value
> 'some replacement value'.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-12762) ListenHTTP request headers limited to 8192 bytes
[
https://issues.apache.org/jira/browse/NIFI-12762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17865596#comment-17865596
]
Michael W Moser commented on NIFI-12762:
Thanks for looking at this [~ssujith]! The problem is that ListenHTTP uses the
default 8192 byte setting that is built into Jetty. It doesn't use the
nifi.properties nifi.web.max.header.size.
If we can setup a unit test that tries to send an HTTP header > 8192 bytes in
size, then we should be able to see the behavior and verify any changes we make.
> ListenHTTP request headers limited to 8192 bytes
>
>
> Key: NIFI-12762
> URL: https://issues.apache.org/jira/browse/NIFI-12762
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.25.0, 2.0.0-M2
>Reporter: Michael W Moser
>Assignee: Sash Sujith
>Priority: Minor
> Attachments: Screenshot from 2024-07-12 19-52-19-1.png
>
>
> ListenHTTP will parse HTTP request headers if the property "HTTP Headers to
> receive as Attributes" is set. An HTTP client can sent request headers that
> are larger than the default allowed 8192 bytes, and we get the log message
> {noformat}
> 2024-02-08 20:22:16,548 WARN [ListenHTTP
> (8a290f74-018d-1000-a9d2-c06fdb10e3f3) Web Server-188]
> org.eclipse.jetty.http.HttpParser Header is too large 8193>8192{noformat}
> and ListenHTTP responds with HTTP ERROR 431 Request Header Fields Too Large
> The NiFi UI and REST API sets the max header size to 16384 and it is
> configurable in nifi.properties "nifi.web.max.header.size", in the file
> FrameworkServerConnectorFactory.java.
> We should probably allow ListenHTTP to use the same max header size setting.
> Should we also look into this for HandleHttpRequest?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (NIFI-13532) Flow Configuration History loses its ability to sort after filter is applied
Michael W Moser created NIFI-13532: -- Summary: Flow Configuration History loses its ability to sort after filter is applied Key: NIFI-13532 URL: https://issues.apache.org/jira/browse/NIFI-13532 Project: Apache NiFi Issue Type: Improvement Components: Core UI Affects Versions: 2.0.0-M4 Reporter: Michael W Moser Bring up the Flow Configuration History page and observe that each column can be sorted ascending and descending. Apply any filter that results in multiple rows. The columns can no longer be sorted. Noticeably, the sort is by Date/Time ascending, which doesn't match the default Date/Time descending. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-13159) Change how PutSFTP handles the REPLACE Conflict Resolution strategy
[ https://issues.apache.org/jira/browse/NIFI-13159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17863936#comment-17863936 ] Michael W Moser commented on NIFI-13159: Small discussion on this happened on Slack https://apachenifi.slack.com/archives/C0L9S92JY/p1713799005100369 > Change how PutSFTP handles the REPLACE Conflict Resolution strategy > --- > > Key: NIFI-13159 > URL: https://issues.apache.org/jira/browse/NIFI-13159 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Michael W Moser >Assignee: Michael W Moser >Priority: Minor > Time Spent: 20m > Remaining Estimate: 0h > > In the PutSFTP processor, in the specific case when "Dot Rename = true" and > "Conflict Resolution = replace", PutSFTP currently behaves like this: > * if remote filename.txt exists, delete it > * send file to remote as .filename.txt > * rename .filename.txt to filename.txt > This causes filename.txt to not exist on the remote machine while the SFTP > transfer is taking place. This is different than the PutFile processor, > which will wait to replace the file until it has written the "dot file". Is > it possible for PutSFTP to behave like this: > * send file to remote as .filename.txt > * if remote filename.txt exists, delete it > * rename .filename.txt to filename.txt > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (NIFI-13383) Excessive log messages validating XML files
[ https://issues.apache.org/jira/browse/NIFI-13383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael W Moser reassigned NIFI-13383: -- Assignee: Michael W Moser > Excessive log messages validating XML files > --- > > Key: NIFI-13383 > URL: https://issues.apache.org/jira/browse/NIFI-13383 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 2.0.0-M3 >Reporter: Michael W Moser >Assignee: Michael W Moser >Priority: Trivial > > If I configure an XMLFileLookupService with an XML file, then I get an INFO > log every 5 seconds. I think we can change this to DEBUG. If there is a > validation problem, there are log messages at the WARN and ERROR level which > should be enough. > 2024-06-10 20:40:40,659 INFO [Validate Components Thread-1] > o.a.n.lookup.configuration2.XXEValidator Validating /path/to/filename.xml for > XXE attack > 2024-06-10 20:40:45,673 INFO [Validate Components Thread-1] > o.a.n.lookup.configuration2.XXEValidator Validating /path/to/filename.xml for > XXE attack > 2024-06-10 20:40:50,687 INFO [Validate Components Thread-2] > o.a.n.lookup.configuration2.XXEValidator Validating /path/to/filename.xml for > XXE attack > 2024-06-10 20:40:55,698 INFO [Validate Components Thread-1] > o.a.n.lookup.configuration2.XXEValidator Validating /path/to/filename.xml for > XXE attack > 2024-06-10 20:41:00,709 INFO [Validate Components Thread-3] > o.a.n.lookup.configuration2.XXEValidator Validating /path/to/filename.xml for > XXE attack > 2024-06-10 20:41:05,721 INFO [Validate Components Thread-3] > o.a.n.lookup.configuration2.XXEValidator Validating /path/to/filename.xml for > XXE attack > 2024-06-10 20:41:10,734 INFO [Validate Components Thread-3] > o.a.n.lookup.configuration2.XXEValidator Validating /path/to/filename.xml for > XXE attack > 2024-06-10 20:41:15,745 INFO [Validate Components Thread-3] > o.a.n.lookup.configuration2.XXEValidator Validating /path/to/filename.xml for > XXE attack -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-13383) Excessive log messages validating XML files
Michael W Moser created NIFI-13383: -- Summary: Excessive log messages validating XML files Key: NIFI-13383 URL: https://issues.apache.org/jira/browse/NIFI-13383 Project: Apache NiFi Issue Type: Improvement Components: Extensions Affects Versions: 2.0.0-M3 Reporter: Michael W Moser If I configure an XMLFileLookupService with an XML file, then I get an INFO log every 5 seconds. I think we can change this to DEBUG. If there is a validation problem, there are log messages at the WARN and ERROR level which should be enough. 2024-06-10 20:40:40,659 INFO [Validate Components Thread-1] o.a.n.lookup.configuration2.XXEValidator Validating /path/to/filename.xml for XXE attack 2024-06-10 20:40:45,673 INFO [Validate Components Thread-1] o.a.n.lookup.configuration2.XXEValidator Validating /path/to/filename.xml for XXE attack 2024-06-10 20:40:50,687 INFO [Validate Components Thread-2] o.a.n.lookup.configuration2.XXEValidator Validating /path/to/filename.xml for XXE attack 2024-06-10 20:40:55,698 INFO [Validate Components Thread-1] o.a.n.lookup.configuration2.XXEValidator Validating /path/to/filename.xml for XXE attack 2024-06-10 20:41:00,709 INFO [Validate Components Thread-3] o.a.n.lookup.configuration2.XXEValidator Validating /path/to/filename.xml for XXE attack 2024-06-10 20:41:05,721 INFO [Validate Components Thread-3] o.a.n.lookup.configuration2.XXEValidator Validating /path/to/filename.xml for XXE attack 2024-06-10 20:41:10,734 INFO [Validate Components Thread-3] o.a.n.lookup.configuration2.XXEValidator Validating /path/to/filename.xml for XXE attack 2024-06-10 20:41:15,745 INFO [Validate Components Thread-3] o.a.n.lookup.configuration2.XXEValidator Validating /path/to/filename.xml for XXE attack -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13381) Use AllowableValues for PutSFTP/PutFTP Conflict Resolution
[ https://issues.apache.org/jira/browse/NIFI-13381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael W Moser updated NIFI-13381: --- Status: Patch Available (was: In Progress) > Use AllowableValues for PutSFTP/PutFTP Conflict Resolution > -- > > Key: NIFI-13381 > URL: https://issues.apache.org/jira/browse/NIFI-13381 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Michael W Moser >Assignee: Michael W Moser >Priority: Trivial > Time Spent: 10m > Remaining Estimate: 0h > > PutSFTP/PutFTP Conflict Resolution choices could use some more documentation. > I couldn't find anywhere that explained what each choice does. We can > improve this using AllowableValues in the Conflicy Resolution property > descriptor. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-13382) UI - Make it more obvious that NiFi is not responding
Michael W Moser created NIFI-13382: -- Summary: UI - Make it more obvious that NiFi is not responding Key: NIFI-13382 URL: https://issues.apache.org/jira/browse/NIFI-13382 Project: Apache NiFi Issue Type: Improvement Components: Core UI Affects Versions: 2.0.0-M3 Reporter: Michael W Moser If I am navigating the new UI and NiFi is stopped, I get a small popup that says "Failed to load cluster summary - [An unspecified error occurred.] Dismiss" If I dismiss this popup, then I am left with a UI that appears to be unresponsive. The old UI used to change the whole page to an "Unable to communicate with NiFi" page. Is there anything we can do to make it more obvious in the new UI that the NiFi server isn't responding? -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-13367) Page title should be Root PG Name
[ https://issues.apache.org/jira/browse/NIFI-13367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17853810#comment-17853810 ] Michael W Moser commented on NIFI-13367: [~mcgilman] [~rfellows] As I have used the new NiFi UI, I find myself using the browser Back button very often. Then I naturally hold the Back button with the intent of going back 5-6 pages. Since the title is always the root PG name, I can't tell where that will take me. Do you have any thoughts if we can (or should) add a little more info to the Title? Things I'm thinking: * name of selected component * name of PG that I'm currently in * if you've selected something in the main hamburger menu, show "Provenance" or "Bulletion Board" or "Summary" > Page title should be Root PG Name > - > > Key: NIFI-13367 > URL: https://issues.apache.org/jira/browse/NIFI-13367 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Major > Fix For: 2.0.0-M4 > > Time Spent: 0.5h > Remaining Estimate: 0h > > The page title for the canvas should be the root Process Group name. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (NIFI-13381) Use AllowableValues for PutSFTP/PutFTP Conflict Resolution
[ https://issues.apache.org/jira/browse/NIFI-13381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael W Moser reassigned NIFI-13381: -- Assignee: Michael W Moser > Use AllowableValues for PutSFTP/PutFTP Conflict Resolution > -- > > Key: NIFI-13381 > URL: https://issues.apache.org/jira/browse/NIFI-13381 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Michael W Moser >Assignee: Michael W Moser >Priority: Trivial > > PutSFTP/PutFTP Conflict Resolution choices could use some more documentation. > I couldn't find anywhere that explained what each choice does. We can > improve this using AllowableValues in the Conflicy Resolution property > descriptor. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-13381) Use AllowableValues for PutSFTP/PutFTP Conflict Resolution
Michael W Moser created NIFI-13381: -- Summary: Use AllowableValues for PutSFTP/PutFTP Conflict Resolution Key: NIFI-13381 URL: https://issues.apache.org/jira/browse/NIFI-13381 Project: Apache NiFi Issue Type: Improvement Components: Extensions Reporter: Michael W Moser PutSFTP/PutFTP Conflict Resolution choices could use some more documentation. I couldn't find anywhere that explained what each choice does. We can improve this using AllowableValues in the Conflicy Resolution property descriptor. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13159) Change how PutSFTP handles the REPLACE Conflict Resolution strategy
[ https://issues.apache.org/jira/browse/NIFI-13159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael W Moser updated NIFI-13159: --- Status: Patch Available (was: In Progress) > Change how PutSFTP handles the REPLACE Conflict Resolution strategy > --- > > Key: NIFI-13159 > URL: https://issues.apache.org/jira/browse/NIFI-13159 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Michael W Moser >Assignee: Michael W Moser >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > In the PutSFTP processor, in the specific case when "Dot Rename = true" and > "Conflict Resolution = replace", PutSFTP currently behaves like this: > * if remote filename.txt exists, delete it > * send file to remote as .filename.txt > * rename .filename.txt to filename.txt > This causes filename.txt to not exist on the remote machine while the SFTP > transfer is taking place. This is different than the PutFile processor, > which will wait to replace the file until it has written the "dot file". Is > it possible for PutSFTP to behave like this: > * send file to remote as .filename.txt > * if remote filename.txt exists, delete it > * rename .filename.txt to filename.txt > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13375) Add missing logging parameter in SplitRecord
[
https://issues.apache.org/jira/browse/NIFI-13375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael W Moser updated NIFI-13375:
---
Fix Version/s: 2.0.0-M4
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Add missing logging parameter in SplitRecord
>
>
> Key: NIFI-13375
> URL: https://issues.apache.org/jira/browse/NIFI-13375
> Project: Apache NiFi
> Issue Type: Improvement
>Reporter: Daniel Stieglitz
>Assignee: Daniel Stieglitz
>Priority: Minor
> Fix For: 2.0.0-M4
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> In SplitRecord on line 201
> {code:java}
> getLogger().error("Failed to split", original, pe);
> {code}
> there is no logging parameter in the message. The code should look like
>
> {code:java}
> getLogger().error("Failed to split {}", original, pe);
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (NIFI-13159) Change how PutSFTP handles the REPLACE Conflict Resolution strategy
[ https://issues.apache.org/jira/browse/NIFI-13159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael W Moser reassigned NIFI-13159: -- Assignee: Michael W Moser > Change how PutSFTP handles the REPLACE Conflict Resolution strategy > --- > > Key: NIFI-13159 > URL: https://issues.apache.org/jira/browse/NIFI-13159 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Michael W Moser >Assignee: Michael W Moser >Priority: Minor > > In the PutSFTP processor, in the specific case when "Dot Rename = true" and > "Conflict Resolution = replace", PutSFTP currently behaves like this: > * if remote filename.txt exists, delete it > * send file to remote as .filename.txt > * rename .filename.txt to filename.txt > This causes filename.txt to not exist on the remote machine while the SFTP > transfer is taking place. This is different than the PutFile processor, > which will wait to replace the file until it has written the "dot file". Is > it possible for PutSFTP to behave like this: > * send file to remote as .filename.txt > * if remote filename.txt exists, delete it > * rename .filename.txt to filename.txt > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-13200) Framework should not allow removal of 'filename' or 'path' attributes
[ https://issues.apache.org/jira/browse/NIFI-13200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17845132#comment-17845132 ] Michael W Moser commented on NIFI-13200: [~markap14] Would this also resolve NIFI-4298? > Framework should not allow removal of 'filename' or 'path' attributes > - > > Key: NIFI-13200 > URL: https://issues.apache.org/jira/browse/NIFI-13200 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Currently, the framework prevents processors from removing the 'uuid' > attribute. However, it does allow any other attribute to be removed. However, > there are many processors that assume that the 'filename' and 'path' > attributes exist, and they are intended always to exist - they are even > assigned values when the FlowFile is created. We should ensure that these > attributes cannot be removed. > Otherwise, configuring UpdateAttribute to remove these attributes can cause > follow-on processors to fail with unexpected NullPointerExceptions -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-13159) Change how PutSFTP handles the REPLACE Conflict Resolution strategy
Michael W Moser created NIFI-13159: -- Summary: Change how PutSFTP handles the REPLACE Conflict Resolution strategy Key: NIFI-13159 URL: https://issues.apache.org/jira/browse/NIFI-13159 Project: Apache NiFi Issue Type: Improvement Components: Extensions Reporter: Michael W Moser In the PutSFTP processor, in the specific case when "Dot Rename = true" and "Conflict Resolution = replace", PutSFTP currently behaves like this: * if remote filename.txt exists, delete it * send file to remote as .filename.txt * rename .filename.txt to filename.txt This causes filename.txt to not exist on the remote machine while the SFTP transfer is taking place. This is different than the PutFile processor, which will wait to replace the file until it has written the "dot file". Is it possible for PutSFTP to behave like this: * send file to remote as .filename.txt * if remote filename.txt exists, delete it * rename .filename.txt to filename.txt -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-6461) FileTransfer accepts remote_path property as FlowFileAttribute but documentation only sais it accepts variable registry
[ https://issues.apache.org/jira/browse/NIFI-6461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael W Moser resolved NIFI-6461. --- Resolution: Duplicate This issue was resolved by NIFI-10559 > FileTransfer accepts remote_path property as FlowFileAttribute but > documentation only sais it accepts variable registry > --- > > Key: NIFI-6461 > URL: https://issues.apache.org/jira/browse/NIFI-6461 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 1.9.2 >Reporter: Alessandro D'Armiento >Priority: Minor > > h2. Current Situation > FileTransfer based processors (ex: PutSFTP) documentation refers they accept > remote_path property only from the variable registry. > Actually, SendFTP and SendFSTP will accept file-attribute based expressions, > while ListFTP and ListSFTP don't. > h2. Improvement Proposal > Fix documentation -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-11423) Ability to Export Data Provenance Results as CSV
[ https://issues.apache.org/jira/browse/NIFI-11423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael W Moser resolved NIFI-11423. Resolution: Won't Do > Ability to Export Data Provenance Results as CSV > > > Key: NIFI-11423 > URL: https://issues.apache.org/jira/browse/NIFI-11423 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework, Core UI >Reporter: Anson Yu >Priority: Minor > > Wanting to add a feature that allows a user to download the provenance as a > CSV for easier parsing. Currently only 1000 max is viewable for return, > looking to see if I can return a larger amount (within reason) and put into > CSV form. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11423) Ability to Export Data Provenance Results as CSV
[ https://issues.apache.org/jira/browse/NIFI-11423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17842434#comment-17842434 ] Michael W Moser commented on NIFI-11423: >From within NiFi, InvokeHTTP can hit the REST nifi-api and retrieve provenance >events as JSON, then use a ConvertRecord to transform it to CSV. There doesn't appear to be a need to implement this in the NiFi UI. > Ability to Export Data Provenance Results as CSV > > > Key: NIFI-11423 > URL: https://issues.apache.org/jira/browse/NIFI-11423 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework, Core UI >Reporter: Anson Yu >Priority: Minor > > Wanting to add a feature that allows a user to download the provenance as a > CSV for easier parsing. Currently only 1000 max is viewable for return, > looking to see if I can return a larger amount (within reason) and put into > CSV form. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (NIFI-11423) Ability to Export Data Provenance Results as CSV
[ https://issues.apache.org/jira/browse/NIFI-11423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael W Moser reassigned NIFI-11423: -- Assignee: (was: Anson Yu) > Ability to Export Data Provenance Results as CSV > > > Key: NIFI-11423 > URL: https://issues.apache.org/jira/browse/NIFI-11423 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework, Core UI >Reporter: Anson Yu >Priority: Minor > > Wanting to add a feature that allows a user to download the provenance as a > CSV for easier parsing. Currently only 1000 max is viewable for return, > looking to see if I can return a larger amount (within reason) and put into > CSV form. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13052) CRON Driven components should be searchable
[ https://issues.apache.org/jira/browse/NIFI-13052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael W Moser updated NIFI-13052: --- Status: Patch Available (was: Open) > CRON Driven components should be searchable > --- > > Key: NIFI-13052 > URL: https://issues.apache.org/jira/browse/NIFI-13052 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Michael W Moser >Assignee: Michael W Moser >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > By entering the keyword "timer" in the Search Bar, you can locate all > components with "Scheduling Strategy: Timer driven". > We should also be able to locate all components that have "Scheduling > Strategy: CRON driven". > This feature is documented in the User Guide under the "Search Components in > DataFlow" section. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (NIFI-13052) CRON Driven components should be searchable
[ https://issues.apache.org/jira/browse/NIFI-13052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael W Moser reassigned NIFI-13052: -- Assignee: Michael W Moser > CRON Driven components should be searchable > --- > > Key: NIFI-13052 > URL: https://issues.apache.org/jira/browse/NIFI-13052 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Michael W Moser >Assignee: Michael W Moser >Priority: Minor > > By entering the keyword "timer" in the Search Bar, you can locate all > components with "Scheduling Strategy: Timer driven". > We should also be able to locate all components that have "Scheduling > Strategy: CRON driven". > This feature is documented in the User Guide under the "Search Components in > DataFlow" section. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-13052) CRON Driven components should be searchable
Michael W Moser created NIFI-13052: -- Summary: CRON Driven components should be searchable Key: NIFI-13052 URL: https://issues.apache.org/jira/browse/NIFI-13052 Project: Apache NiFi Issue Type: Improvement Components: Core Framework Reporter: Michael W Moser By entering the keyword "timer" in the Search Bar, you can locate all components with "Scheduling Strategy: Timer driven". We should also be able to locate all components that have "Scheduling Strategy: CRON driven". This feature is documented in the User Guide under the "Search Components in DataFlow" section. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-12853) Refactor FlowFilePrioritizer using updated Java APIs
[ https://issues.apache.org/jira/browse/NIFI-12853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael W Moser resolved NIFI-12853. Fix Version/s: 2.0.0-M3 Resolution: Fixed > Refactor FlowFilePrioritizer using updated Java APIs > > > Key: NIFI-12853 > URL: https://issues.apache.org/jira/browse/NIFI-12853 > Project: Apache NiFi > Issue Type: Task >Reporter: endzeit >Assignee: endzeit >Priority: Major > Fix For: 2.0.0-M3 > > Time Spent: 0.5h > Remaining Estimate: 0h > > The FlowFilePrioritizer implementations and test contain some boilerplate / > duplication that can be reduced by using updated Java APIs. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12987) Controller Service type should be searchable
[ https://issues.apache.org/jira/browse/NIFI-12987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17833212#comment-17833212 ] Michael W Moser commented on NIFI-12987: [~pvillard] On the support/nifi-1.x branch, I noticed an error in the unit test ControllerSearchServiceRegressionTest.testSearchControllerServices. I ran out of time yesterday, but I will work on fixing that today. > Controller Service type should be searchable > > > Key: NIFI-12987 > URL: https://issues.apache.org/jira/browse/NIFI-12987 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 1.25.0 >Reporter: Michael W Moser >Assignee: Michael W Moser >Priority: Minor > Labels: backport-needed > Fix For: 2.0.0-M3, 1.26.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Currently controller service name, uuid, property name, property value and > comments are all searchable. However you cannot search by controller service > type (class name) like you can with other components. Allow the search to > find controller services by type. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (NIFI-12987) Controller Service type should be searchable
[ https://issues.apache.org/jira/browse/NIFI-12987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17833212#comment-17833212 ] Michael W Moser edited comment on NIFI-12987 at 4/2/24 2:08 PM: [~pvillard] On the support/nifi-1.x branch, I noticed an error in the unit test ControllerSearchServiceRegressionTest.testSearchControllerServices (this test doesn't exist on the main branch). I ran out of time yesterday, but I will work on fixing that today. was (Author: mosermw): [~pvillard] On the support/nifi-1.x branch, I noticed an error in the unit test ControllerSearchServiceRegressionTest.testSearchControllerServices. I ran out of time yesterday, but I will work on fixing that today. > Controller Service type should be searchable > > > Key: NIFI-12987 > URL: https://issues.apache.org/jira/browse/NIFI-12987 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 1.25.0 >Reporter: Michael W Moser >Assignee: Michael W Moser >Priority: Minor > Labels: backport-needed > Fix For: 2.0.0-M3, 1.26.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Currently controller service name, uuid, property name, property value and > comments are all searchable. However you cannot search by controller service > type (class name) like you can with other components. Allow the search to > find controller services by type. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12987) Controller Service type should be searchable
[ https://issues.apache.org/jira/browse/NIFI-12987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael W Moser updated NIFI-12987: --- Status: Patch Available (was: Open) > Controller Service type should be searchable > > > Key: NIFI-12987 > URL: https://issues.apache.org/jira/browse/NIFI-12987 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 1.25.0 >Reporter: Michael W Moser >Assignee: Michael W Moser >Priority: Minor > Labels: backport-needed > > Currently controller service name, uuid, property name, property value and > comments are all searchable. However you cannot search by controller service > type (class name) like you can with other components. Allow the search to > find controller services by type. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (NIFI-12987) Controller Service type should be searchable
[ https://issues.apache.org/jira/browse/NIFI-12987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael W Moser reassigned NIFI-12987: -- Assignee: Michael W Moser > Controller Service type should be searchable > > > Key: NIFI-12987 > URL: https://issues.apache.org/jira/browse/NIFI-12987 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 1.25.0 >Reporter: Michael W Moser >Assignee: Michael W Moser >Priority: Minor > Labels: backport-needed > > Currently controller service name, uuid, property name, property value and > comments are all searchable. However you cannot search by controller service > type (class name) like you can with other components. Allow the search to > find controller services by type. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-12987) Controller Service type should be searchable
Michael W Moser created NIFI-12987: -- Summary: Controller Service type should be searchable Key: NIFI-12987 URL: https://issues.apache.org/jira/browse/NIFI-12987 Project: Apache NiFi Issue Type: Improvement Components: Core Framework Affects Versions: 1.25.0 Reporter: Michael W Moser Currently controller service name, uuid, property name, property value and comments are all searchable. However you cannot search by controller service type (class name) like you can with other components. Allow the search to find controller services by type. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-5125) Unable to select Controller Service UUID from Bulletin Board
[ https://issues.apache.org/jira/browse/NIFI-5125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17832902#comment-17832902 ] Michael W Moser commented on NIFI-5125: --- I found this ticket, and tested again. The "Error: Unable to find specified component" still happens. However, after NIFI-7188, controller service UUID values are searchable from the toolbar. So the search capability is unrelated to the error. > Unable to select Controller Service UUID from Bulletin Board > > > Key: NIFI-5125 > URL: https://issues.apache.org/jira/browse/NIFI-5125 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.6.0 >Reporter: Mark Bean >Priority: Major > > When Selecting the linked UUID value of a Controller Service from the > Bulletin Board, the result is "Error: Unable to find the specified > component." Expected behavior would be to open the appropriate Controller > Services tab (either Component or Reporting Tasks) > Perhaps related is the fact that Controller Service UUID values are not > searchable from the toolbar. This ticket may need to add this capability as > well. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-7085) Consume JMS is very slow
[ https://issues.apache.org/jira/browse/NIFI-7085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael W Moser updated NIFI-7085: -- Status: Patch Available (was: Open) > Consume JMS is very slow > > > Key: NIFI-7085 > URL: https://issues.apache.org/jira/browse/NIFI-7085 > Project: Apache NiFi > Issue Type: Improvement > Environment: 1.8.0 nifi >Reporter: naveen kumar saharan >Assignee: Michael W Moser >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > ConsumeJMS as primary node consuming only 20 messages per second. > If i do all nodes will it read duplicate? > If i run multiple thread will it read as duplicate? > > We want to read 2000 messages per sec from Tibco jms. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-8960) Create ability for EvaluateJsonPath processor to match any or all defined json paths.
[ https://issues.apache.org/jira/browse/NIFI-8960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael W Moser updated NIFI-8960: -- Fix Version/s: (was: 1.14.0) Resolution: Won't Do Status: Resolved (was: Patch Available) Expression Language jsonPath() was built for this use case. > Create ability for EvaluateJsonPath processor to match any or all defined > json paths. > -- > > Key: NIFI-8960 > URL: https://issues.apache.org/jira/browse/NIFI-8960 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Tim Smith >Assignee: Tim Smith >Priority: Minor > Time Spent: 0.5h > Remaining Estimate: 0h > > Add the capability to set "Match Criteria" to "all" or "any", where all json > paths must match on json objects. Default behavior being "any". Also add the > ability to apply to either attributes matching regex or content, with > default being "match on content". -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (NIFI-8960) Create ability for EvaluateJsonPath processor to match any or all defined json paths.
[ https://issues.apache.org/jira/browse/NIFI-8960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831448#comment-17831448 ] Michael W Moser edited comment on NIFI-8960 at 3/27/24 4:29 PM: Expression Language jsonPath() was built for this use case. NIFI-1660 was (Author: mosermw): Expression Language jsonPath() was built for this use case. > Create ability for EvaluateJsonPath processor to match any or all defined > json paths. > -- > > Key: NIFI-8960 > URL: https://issues.apache.org/jira/browse/NIFI-8960 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Tim Smith >Assignee: Tim Smith >Priority: Minor > Time Spent: 0.5h > Remaining Estimate: 0h > > Add the capability to set "Match Criteria" to "all" or "any", where all json > paths must match on json objects. Default behavior being "any". Also add the > ability to apply to either attributes matching regex or content, with > default being "match on content". -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-8930) Add ability for ScanAttribute processor to match on attribute values containing a delimited list of values.
[ https://issues.apache.org/jira/browse/NIFI-8930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael W Moser updated NIFI-8930: -- Resolution: Won't Do Status: Resolved (was: Patch Available) > Add ability for ScanAttribute processor to match on attribute values > containing a delimited list of values. > --- > > Key: NIFI-8930 > URL: https://issues.apache.org/jira/browse/NIFI-8930 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.14.0 >Reporter: Tim Smith >Assignee: Tim Smith >Priority: Minor > Time Spent: 40m > Remaining Estimate: 0h > > The ScanAttribute processor is limited in that an exact match is required > when applied for attribute value. There are many times an attribute contains > a delimited list of values. It would be useful to specify a delimiter and > have the match criteria applied to each delimited value in the attribute. Two > additional property descriptors need created, 'Delimiter' and 'Delimited > Match Criteria'. 'Delimiter' sets the delimiter to apply to the attribute(s). > 'Delimited Match Criteria' specifies how the delimited attributes should > match the dictionary. If 'All Must Match' is selected, all delimited values > must match a dictionary term/pattern for attribute to be matched. For 'At > Least 1 Must Match' , if any one delimited value matches,then the attribute > matches. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12925) ListenHTTP should disable unused HTTP methods
[ https://issues.apache.org/jira/browse/NIFI-12925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael W Moser updated NIFI-12925: --- Description: For [security reasons|https://owasp.org/www-project-web-security-testing-guide/stable/4-Web_Application_Security_Testing/02-Configuration_and_Deployment_Management_Testing/06-Test_HTTP_Methods], ListenHTTP should reply with 405 Method Not Allowed for HTTP methods OPTIONS and TRACE. PUT already returns 405. GET, POST, DELETE, HEAD are used by the processor. was: For [security reasons|https://owasp.org/www-project-web-security-testing-guide/stable/4-Web_Application_Security_Testing/02-Configuration_and_Deployment_Management_Testing/06-Test_HTTP_Methods], ListenHTTP should reply with 405 Method Not Allowed for HTTP methods OPTIONS and TRACE. PUT and DELETE already return 405. A GET request returns 404 Not Found. > ListenHTTP should disable unused HTTP methods > - > > Key: NIFI-12925 > URL: https://issues.apache.org/jira/browse/NIFI-12925 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Michael W Moser >Priority: Major > > For [security > reasons|https://owasp.org/www-project-web-security-testing-guide/stable/4-Web_Application_Security_Testing/02-Configuration_and_Deployment_Management_Testing/06-Test_HTTP_Methods], > ListenHTTP should reply with 405 Method Not Allowed for HTTP methods OPTIONS > and TRACE. > PUT already returns 405. > GET, POST, DELETE, HEAD are used by the processor. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-12925) ListenHTTP should disable unused HTTP methods
Michael W Moser created NIFI-12925: -- Summary: ListenHTTP should disable unused HTTP methods Key: NIFI-12925 URL: https://issues.apache.org/jira/browse/NIFI-12925 Project: Apache NiFi Issue Type: Improvement Components: Extensions Reporter: Michael W Moser For [security reasons|https://owasp.org/www-project-web-security-testing-guide/stable/4-Web_Application_Security_Testing/02-Configuration_and_Deployment_Management_Testing/06-Test_HTTP_Methods], ListenHTTP should reply with 405 Method Not Allowed for HTTP methods OPTIONS and TRACE. PUT and DELETE already return 405. A GET request returns 404 Not Found. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-7085) Consume JMS is very slow
[ https://issues.apache.org/jira/browse/NIFI-7085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17825374#comment-17825374 ] Michael W Moser commented on NIFI-7085: --- When consuming a JMS queue, you can add more than 1 concurrent task to ConsumeJMS to get the performance you need. When consuming a JMS topic, if you add more than 1 concurrent task on a non-durable, non-shared topic, then you receive duplicate messages and performance does not improve. In this case, you cannot use more than 1 task. I think I can improve ConsumeJMS by adding a Message Batch Size property, which will allow the ability to consume more than 1 message per onTrigger(). As a note to anyone who reads this, recommend always setting ConsumeJMS Run Schedule to 0 sec. If no message is available, then the Timeout property controls how long to keep the timer-driven thread while waiting for a message from the broker. > Consume JMS is very slow > > > Key: NIFI-7085 > URL: https://issues.apache.org/jira/browse/NIFI-7085 > Project: Apache NiFi > Issue Type: Improvement > Environment: 1.8.0 nifi >Reporter: naveen kumar saharan >Priority: Minor > > ConsumeJMS as primary node consuming only 20 messages per second. > If i do all nodes will it read duplicate? > If i run multiple thread will it read as duplicate? > > We want to read 2000 messages per sec from Tibco jms. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (NIFI-7085) Consume JMS is very slow
[ https://issues.apache.org/jira/browse/NIFI-7085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael W Moser reassigned NIFI-7085: - Assignee: Michael W Moser > Consume JMS is very slow > > > Key: NIFI-7085 > URL: https://issues.apache.org/jira/browse/NIFI-7085 > Project: Apache NiFi > Issue Type: Improvement > Environment: 1.8.0 nifi >Reporter: naveen kumar saharan >Assignee: Michael W Moser >Priority: Minor > > ConsumeJMS as primary node consuming only 20 messages per second. > If i do all nodes will it read duplicate? > If i run multiple thread will it read as duplicate? > > We want to read 2000 messages per sec from Tibco jms. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12871) Upgrade Commons Compress to 1.26.1
[ https://issues.apache.org/jira/browse/NIFI-12871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17824441#comment-17824441 ] Michael W Moser commented on NIFI-12871: Haha, thanks [~exceptionfactory], I encountered NiFi build errors with 1.26.0 and that led me to the Jira ticket that you submitted to the Commons Compress team. Thanks for being on top of this! I will remove myself as Assignee while we wait for 1.26.1. > Upgrade Commons Compress to 1.26.1 > -- > > Key: NIFI-12871 > URL: https://issues.apache.org/jira/browse/NIFI-12871 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Extensions >Reporter: Michael W Moser >Assignee: Michael W Moser >Priority: Minor > Labels: backport-needed > > Apache Commons Compress has released version > [1.26.0|https://commons.apache.org/proper/commons-compress/changes-report.html#a1.26.0] > which is a minor feature and maintenance release. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (NIFI-12871) Upgrade Commons Compress to 1.26.1
[ https://issues.apache.org/jira/browse/NIFI-12871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael W Moser reassigned NIFI-12871: -- Assignee: (was: Michael W Moser) > Upgrade Commons Compress to 1.26.1 > -- > > Key: NIFI-12871 > URL: https://issues.apache.org/jira/browse/NIFI-12871 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Extensions >Reporter: Michael W Moser >Priority: Minor > Labels: backport-needed > > Apache Commons Compress has released version > [1.26.0|https://commons.apache.org/proper/commons-compress/changes-report.html#a1.26.0] > which is a minor feature and maintenance release. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (NIFI-12871) Upgrade Commons Compress to 1.26.0
[ https://issues.apache.org/jira/browse/NIFI-12871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael W Moser reassigned NIFI-12871: -- Assignee: Michael W Moser > Upgrade Commons Compress to 1.26.0 > -- > > Key: NIFI-12871 > URL: https://issues.apache.org/jira/browse/NIFI-12871 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Extensions >Reporter: Michael W Moser >Assignee: Michael W Moser >Priority: Minor > Labels: backport-needed > > Apache Commons Compress has released version > [1.26.0|https://commons.apache.org/proper/commons-compress/changes-report.html#a1.26.0] > which is a minor feature and maintenance release. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12871) Upgrade Commons Compress to 1.26.0
[ https://issues.apache.org/jira/browse/NIFI-12871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael W Moser updated NIFI-12871: --- Labels: backport-needed (was: ) > Upgrade Commons Compress to 1.26.0 > -- > > Key: NIFI-12871 > URL: https://issues.apache.org/jira/browse/NIFI-12871 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Extensions >Reporter: Michael W Moser >Priority: Minor > Labels: backport-needed > > Apache Commons Compress has released version > [1.26.0|https://commons.apache.org/proper/commons-compress/changes-report.html#a1.26.0] > which is a minor feature and maintenance release. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12871) Upgrade Commons Compress to 1.26.0
[ https://issues.apache.org/jira/browse/NIFI-12871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael W Moser updated NIFI-12871: --- Priority: Minor (was: Major) > Upgrade Commons Compress to 1.26.0 > -- > > Key: NIFI-12871 > URL: https://issues.apache.org/jira/browse/NIFI-12871 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Extensions >Reporter: Michael W Moser >Priority: Minor > > Apache Commons Compress has released version > [1.26.0|https://commons.apache.org/proper/commons-compress/changes-report.html#a1.26.0] > which is a minor feature and maintenance release. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-12871) Upgrade Commons Compress to 1.26.0
Michael W Moser created NIFI-12871: -- Summary: Upgrade Commons Compress to 1.26.0 Key: NIFI-12871 URL: https://issues.apache.org/jira/browse/NIFI-12871 Project: Apache NiFi Issue Type: Improvement Components: Core Framework, Extensions Reporter: Michael W Moser Apache Commons Compress has released version [1.26.0|https://commons.apache.org/proper/commons-compress/changes-report.html#a1.26.0] which is a minor feature and maintenance release. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12842) InvokeHTTP version wrong encoding of % in URL
[ https://issues.apache.org/jira/browse/NIFI-12842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael W Moser updated NIFI-12842: --- Affects Version/s: 2.0.0-M2 > InvokeHTTP version wrong encoding of % in URL > - > > Key: NIFI-12842 > URL: https://issues.apache.org/jira/browse/NIFI-12842 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.25.0, 2.0.0-M2 > Environment: RHEL 7 >Reporter: WojciechWitos >Assignee: Daniel Stieglitz >Priority: Major > Attachments: image-2024-02-26-08-10-12-657.png, > image-2024-02-26-08-11-25-213.png, image-2024-02-26-08-13-14-199.png, > image-2024-02-26-08-16-36-309.png, image-2024-02-27-12-09-31-831.png, > image-2024-02-27-12-10-33-542.png, image-2024-02-27-12-10-59-337.png, > image-2024-02-27-12-11-39-163.png, image-2024-02-27-12-12-00-043.png, > image-2024-02-27-12-12-06-720.png, image-2024-02-27-12-13-10-173.png > > > Hi! > I've encountered on the version 1.25 issue with encoding of % in invokehttp > processor. > It changes every % into %25, which causes error 403 and most of the flows > don't work. > On the version 1.24 everything works properly with this processor. > Here are the screenshots of 1.25: > !image-2024-02-26-08-10-12-657.png! > And request: > !image-2024-02-26-08-11-25-213.png! > Where on version 1.24 this issue doesn't persist: > !image-2024-02-26-08-13-14-199.png! > !image-2024-02-26-08-16-36-309.png! > Please investigate this issue, it is the blocker of upgrading the environment > to this version. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12785) InvokeHTTP handler should not urlencode HTTP URL
[ https://issues.apache.org/jira/browse/NIFI-12785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael W Moser updated NIFI-12785: --- Affects Version/s: 1.25.0 > InvokeHTTP handler should not urlencode HTTP URL > > > Key: NIFI-12785 > URL: https://issues.apache.org/jira/browse/NIFI-12785 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.25.0, 2.0.0-M2 > Environment: AlmaLinux 8.9 Kernel 4.18.0-513.5.1.el8_9.x86_64 > Apache NiFi 2.0.0-M2 >Reporter: macdoor615 >Priority: Major > Attachments: M1-output.png, M2-output.png > > > InvokeHTTP processor call HTTP URL > [http://hb3-ifz-gitlab-000:8100/gitlab/api/v4/projects/318/repository/files/ftp%2Fstage%2F15m%2Fheshangwuzhibo.yaml/raw?ref=main] > output attribute > invokehttp.request.url: > [http://hb3-ifz-gitlab-000:8100/gitlab/api/v4/projects/318/repository/files/ftp%252Fstage%252F15m%252Fheshangwuzhibo.yaml/raw?ref=main] > > invokehttp.status.code: 404 > > The situation is different for version 2.0.0-M1, output attribute > invokehttp.request.url: > [http://hb3-ifz-gitlab-000:8100/gitlab/api/v4/projects/318/repository/files/ftp%2Fstage%2F15m%2Fheshangwuzhibo.yaml/raw?ref=main] > > invokehttp.status.code: 200 > > I found that in the M2 version % symbol was urlencoded to %25, M1 version. > The M1 version does not urlencode > > pls refer to the uploaded pictures -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-6959) JMS Durable subscription does not support filters
[ https://issues.apache.org/jira/browse/NIFI-6959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael W Moser resolved NIFI-6959. --- Resolution: Fixed > JMS Durable subscription does not support filters > - > > Key: NIFI-6959 > URL: https://issues.apache.org/jira/browse/NIFI-6959 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.6.0, 1.7.0, 1.8.0, 1.9.0, 1.10.0, 1.9.1, 1.9.2 > Environment: All >Reporter: john uiterwyk >Priority: Major > > The ConsumeJMS processor does not support passing filters when creating a > topic subscription. > This can be seen in line 72 of JMSConsumer.java, where null is passed for the > filter when calling : > |return session.createDurableConsumer((Topic) destination, subscriberName, > null, JMSConsumer.this.jmsTemplate.isPubSubDomain());| > | | | > Some JMS implementations require a filter when creating a durable > subscription for performance reasons. > This can be resolved by adding a property 'filter' of type string to the > ConsumeJMS processor, and then passing that filter string to the JMSConsumer > method createMessageConsumer and then passing it to the createDurableConsumer > method. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-6391) ConsumeJMS/PublishJMS - Excessive connections created when get/publish operations continually fail
[ https://issues.apache.org/jira/browse/NIFI-6391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael W Moser resolved NIFI-6391. --- Resolution: Done > ConsumeJMS/PublishJMS - Excessive connections created when get/publish > operations continually fail > -- > > Key: NIFI-6391 > URL: https://issues.apache.org/jira/browse/NIFI-6391 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.8.0 >Reporter: Steven Youtsey >Assignee: Steven Youtsey >Priority: Major > Time Spent: 1h 10m > Remaining Estimate: 0h > > PublishJMS repeatedly threw "ResourceAllocationException: too many open > connections" after successive publish failures due to issues on the remote > JMS Broker. The cause of this is the specification and implementation of the > JMSConnectionFactoryProviderDefinition.resetConnectionFactory(ConnectionFactory > cf) method. Setting the ConnectionFactory to null may well indeed close > connections upon 'destruction', but it may take awhile for the GC to run; in > the meantime, more connections are opened. The connections need to be > manually closed rather than waiting for a GC. > From AbstractJMSProcessor.onTrigger(), need to call worker.shutdown() prior > to resetConnectionFactory(). > Also, noticed some problems iwth the ConnectionFactoryProviderDefinition > implementatoins wrt the resetConnectionFactory methods. The factory is nulled > but never re-initialized but for onEnabled(); which will lead to a NPE at > some point. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-12762) ListenHTTP request headers limited to 8192 bytes
Michael W Moser created NIFI-12762:
--
Summary: ListenHTTP request headers limited to 8192 bytes
Key: NIFI-12762
URL: https://issues.apache.org/jira/browse/NIFI-12762
Project: Apache NiFi
Issue Type: Bug
Affects Versions: 2.0.0-M2, 1.25.0
Reporter: Michael W Moser
ListenHTTP will parse HTTP request headers if the property "HTTP Headers to
receive as Attributes" is set. An HTTP client can sent request headers that
are larger than the default allowed 8192 bytes, and we get the log message
{noformat}
2024-02-08 20:22:16,548 WARN [ListenHTTP (8a290f74-018d-1000-a9d2-c06fdb10e3f3)
Web Server-188] org.eclipse.jetty.http.HttpParser Header is too large
8193>8192{noformat}
and ListenHTTP responds with HTTP ERROR 431 Request Header Fields Too Large
The NiFi UI and REST API sets the max header size to 16384 and it is
configurable in nifi.properties "nifi.web.max.header.size", in the file
FrameworkServerConnectorFactory.java.
We should probably allow ListenHTTP to use the same max header size setting.
Should we also look into this for HandleHttpRequest?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (NIFI-12641) Add local file upload option in PutS3 processor
[ https://issues.apache.org/jira/browse/NIFI-12641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael W Moser reassigned NIFI-12641: -- Assignee: (was: Peter Turcsanyi) > Add local file upload option in PutS3 processor > --- > > Key: NIFI-12641 > URL: https://issues.apache.org/jira/browse/NIFI-12641 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Balázs Gerner >Priority: Major > > There are cases when the files to be uploaded to Azure Storage are available > on the local filesystem where NiFi is running. That is, the flow could read > and upload the files directly from the filesystem without adding it in NiFi's > content repo which is an overhead in this case (can be relevant for huge > files). > Add "Data to Upload" property with options "FlowFile's Content" (default, > current behaviour) and "Local File". Using the latter, the user can by-pass > the content repo and upload the file from the local filesystem to Azure > Storage directly. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-12641) Add local file upload option in PutS3 processor
[ https://issues.apache.org/jira/browse/NIFI-12641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael W Moser resolved NIFI-12641. Resolution: Duplicate > Add local file upload option in PutS3 processor > --- > > Key: NIFI-12641 > URL: https://issues.apache.org/jira/browse/NIFI-12641 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Balázs Gerner >Priority: Major > > There are cases when the files to be uploaded to Azure Storage are available > on the local filesystem where NiFi is running. That is, the flow could read > and upload the files directly from the filesystem without adding it in NiFi's > content repo which is an overhead in this case (can be relevant for huge > files). > Add "Data to Upload" property with options "FlowFile's Content" (default, > current behaviour) and "Local File". Using the latter, the user can by-pass > the content repo and upload the file from the local filesystem to Azure > Storage directly. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12641) Add local file upload option in PutS3 processor
[ https://issues.apache.org/jira/browse/NIFI-12641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael W Moser updated NIFI-12641: --- Fix Version/s: (was: 2.0.0-M1) (was: 1.23.0) > Add local file upload option in PutS3 processor > --- > > Key: NIFI-12641 > URL: https://issues.apache.org/jira/browse/NIFI-12641 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Balázs Gerner >Assignee: Peter Turcsanyi >Priority: Major > > There are cases when the files to be uploaded to Azure Storage are available > on the local filesystem where NiFi is running. That is, the flow could read > and upload the files directly from the filesystem without adding it in NiFi's > content repo which is an overhead in this case (can be relevant for huge > files). > Add "Data to Upload" property with options "FlowFile's Content" (default, > current behaviour) and "Local File". Using the latter, the user can by-pass > the content repo and upload the file from the local filesystem to Azure > Storage directly. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12649) Explain why run-nifi.bat closes
[ https://issues.apache.org/jira/browse/NIFI-12649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17812362#comment-17812362 ] Michael W Moser commented on NIFI-12649: Hello [~simon.aubert] . Apache NiFi 2.0.0 requires a Java 21 runtime environment or higher. The error you attached shows that you are using Java 17. This will not work. If you open a command window first and then type the run-nifi.bat command, then the window should not close. > Explain why run-nifi.bat closes > --- > > Key: NIFI-12649 > URL: https://issues.apache.org/jira/browse/NIFI-12649 > Project: Apache NiFi > Issue Type: Improvement > Components: Tools and Build >Affects Versions: 2.0.0-M1 > Environment: All >Reporter: Simon AUBERT >Priority: Major > Attachments: image-2024-01-21-10-03-26-177.png > > > Hello all, > Here a frustrating issue that can be solved very easily : I follow the > instructions here [https://nifi.apache.org/documentation/v2/] in order to use > nifi on windows. > So I launched the run-nifi.bat and it closes the windows almost immediatly, > before I can read the message and apparently without logging anything. I lost > a lost of time to find the issue... (a not up to date jdk). How did I do ? By > using Greenshot to make a screenshot and luckily, I got the exact message. > !image-2024-01-21-10-03-26-177.png! > So I would like a way to know why this window closes. I guess logging the > issue to a file can be doable. > Best regards, > Simon -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-8968) Improve throughput performance for InvokeHTTP
[ https://issues.apache.org/jira/browse/NIFI-8968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17809153#comment-17809153 ] Michael W Moser commented on NIFI-8968: --- Yes, only 1 InvokeHTTP processor. I'm a bit surprised that we got different results. Perhaps our environments are different enough to explain it. > Improve throughput performance for InvokeHTTP > - > > Key: NIFI-8968 > URL: https://issues.apache.org/jira/browse/NIFI-8968 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.14.0 >Reporter: Mark Bean >Priority: Major > Attachments: Compare_Invoke_PostHTTP.json, > ListenHTTP-BytesOut-sizeFilter.PNG, ListenHTTP-BytesOut.PNG, > ListenHTTP-FFOut-sizeFilter.PNG, ListenHTTP-FFOut.PNG, > PostHTTP_vs_InvokeHTTP.json, PostHTTP_vs_InvokeHTTP.xml > > > InvokeHTTP is the preferred processor to use over the deprecated PostHTTP. > However, PostHTTP outperforms InvokeHTTP (at least in POST mode). A template > and a JSON file have been attached to this ticket for benchmarking the two > processors. Using this flow, PostHTTP was observed to have a throughput of > approximately 5 times greater than InvokeHTTP. > In addition, it was noted that InvokeHTTP had approximately 5 times as many > tasks and 5 times the task duration for a given 5 minute stats window. And, > the statistics of Bytes Read and Bytes Transferred remain at zero for > InvokeHTTP; this area accurate statistics also needs to be addressed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (NIFI-8968) Improve throughput performance for InvokeHTTP
[
https://issues.apache.org/jira/browse/NIFI-8968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17808964#comment-17808964
]
Michael W Moser edited comment on NIFI-8968 at 1/20/24 10:20 PM:
-
{{Thanks for trying this out again [~markbean] . I think your results are very
close to my results on row 3,2,1 of my table. Do you have time to try a
configuration that matches row 4 of my table? Essentially this:}}
{{GenerateFlowFile(s) -> RouteOnAttribute}}
{{ | (fileSize < 1_000_000) -> PackageFlowFile (batch
size 500) |}}
{{ | (fileSize >= 1_000_000) -> PackageFlowFile (batch
size 1) -> InvokeHTTP}}
was (Author: mosermw):
{{Thanks for trying this out again [~markbean] . I think your results are very
close to my results on row 3,2,1 of my table. Do you have time to try a
configuration that matches row 4 of my table? Essentially this:}}
{{GenerateFlowFile(s) -> RouteOnAttribute}}
{{ | }}{{(fileSize < 1_000_000) -> PackageFlowFile
(batch size 500) |}}
{{ | (fileSize >= 1_000_000) -> PackageFlowFile (batch
size 1) -> InvokeHTTP}}
> Improve throughput performance for InvokeHTTP
> -
>
> Key: NIFI-8968
> URL: https://issues.apache.org/jira/browse/NIFI-8968
> Project: Apache NiFi
> Issue Type: Improvement
>Affects Versions: 1.14.0
>Reporter: Mark Bean
>Priority: Major
> Attachments: ListenHTTP-BytesOut.PNG, ListenHTTP-FFOut.PNG,
> PostHTTP_vs_InvokeHTTP.json, PostHTTP_vs_InvokeHTTP.xml
>
>
> InvokeHTTP is the preferred processor to use over the deprecated PostHTTP.
> However, PostHTTP outperforms InvokeHTTP (at least in POST mode). A template
> and a JSON file have been attached to this ticket for benchmarking the two
> processors. Using this flow, PostHTTP was observed to have a throughput of
> approximately 5 times greater than InvokeHTTP.
> In addition, it was noted that InvokeHTTP had approximately 5 times as many
> tasks and 5 times the task duration for a given 5 minute stats window. And,
> the statistics of Bytes Read and Bytes Transferred remain at zero for
> InvokeHTTP; this area accurate statistics also needs to be addressed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-8968) Improve throughput performance for InvokeHTTP
[
https://issues.apache.org/jira/browse/NIFI-8968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17808964#comment-17808964
]
Michael W Moser commented on NIFI-8968:
---
{{Thanks for trying this out again [~markbean] . I think your results are very
close to my results on row 3,2,1 of my table. Do you have time to try a
configuration that matches row 4 of my table? Essentially this:}}
{{GenerateFlowFile(s) -> RouteOnAttribute}}
{{ | }}{{(fileSize < 1_000_000) -> PackageFlowFile
(batch size 500) |}}
{{ | (fileSize >= 1_000_000) -> PackageFlowFile (batch
size 1) -> InvokeHTTP}}
> Improve throughput performance for InvokeHTTP
> -
>
> Key: NIFI-8968
> URL: https://issues.apache.org/jira/browse/NIFI-8968
> Project: Apache NiFi
> Issue Type: Improvement
>Affects Versions: 1.14.0
>Reporter: Mark Bean
>Priority: Major
> Attachments: ListenHTTP-BytesOut.PNG, ListenHTTP-FFOut.PNG,
> PostHTTP_vs_InvokeHTTP.json, PostHTTP_vs_InvokeHTTP.xml
>
>
> InvokeHTTP is the preferred processor to use over the deprecated PostHTTP.
> However, PostHTTP outperforms InvokeHTTP (at least in POST mode). A template
> and a JSON file have been attached to this ticket for benchmarking the two
> processors. Using this flow, PostHTTP was observed to have a throughput of
> approximately 5 times greater than InvokeHTTP.
> In addition, it was noted that InvokeHTTP had approximately 5 times as many
> tasks and 5 times the task duration for a given 5 minute stats window. And,
> the statistics of Bytes Read and Bytes Transferred remain at zero for
> InvokeHTTP; this area accurate statistics also needs to be addressed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (NIFI-8968) Improve throughput performance for InvokeHTTP
[ https://issues.apache.org/jira/browse/NIFI-8968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17808848#comment-17808848 ] Michael W Moser edited comment on NIFI-8968 at 1/20/24 1:16 AM: With the addition of the PackageFlowFile processor, I decided to compare performance of PostHTTP and InvokeHTTP again. PackageFlowFile can be used to bundle flowfiles together and set the mime.type to application/flowfile-v3. When used with InvokeHTTP, this effectively is the same as PostHTTP's "send as flowfile" capability. PostHTTP will send many flowfiles in one batch (Max Batch Size = 100 MB), whereas InvokeHTTP will only send 1. This is why we see PostHTTP use a lot fewer Tasks compared to InvokeHTTP (because it uses 1 Task per batch). Also, with the flow definition attached to this ticket, both processors were provided 4 input queues for files of different sizes. Because of PostHTTP batching, the small files make it through the processor much quicker than InvokeHTTP. Here are my results (values are from 5-minute Status History): ||Scenario||Bytes In||Files In||Tasks||Total Task Duration||Files Out of ListenHTTP|| |PostHTTP|24.22 GB|1.44 million|524|5:00 minutes|1.44 million| |InvokeHTTP|16.97 GB|6320|6320|5:00 minutes|6320| |PackageFlowFile + InvokeHTTP|25.36 GB|76|76|4:11 minutes|28,200| |PackageFlowFile 500 files < 1 MB + PackageFlowFile 1 file >= 1 MB + InvokeHTTP|26.47 GB|7000|7000|5.00 minutes|2.3 million| Conclusions: * PostHTTP and InvokeHTTP can get the same bytes-per-second throughput when very small files are not involved. * The scenario of packaging 100 byte and 10 MB files together in a flow may not be real world, but can definitely cause unsatisfactory results. * *Configured correctly, PackageFlowFile + InvokeHTTP can actually perform better than PostHTTP.* was (Author: mosermw): With the addition of the PackageFlowFile processor, I decided to compare performance of PostHTTP and InvokeHTTP again. PackageFlowFile can be used to bundle flowfiles together and set the mime.type to application/flowfile-v3. When used with InvokeHTTP, this effectively is the same as PostHTTP's "send as flowfile" capability. PostHTTP will send many flowfiles in one batch (Max Batch Size = 100 MB), whereas InvokeHTTP will only send 1. This is why we see PostHTTP use a lot fewer Tasks compared to InvokeHTTP (because it uses 1 Task per batch). Also, with the flow definition attached to this ticket, both processors were provided 4 input queues for files of different sizes. Because of PostHTTP batching, the small files make it through the processor much quicker than InvokeHTTP. Here are my results: ||Scenario||Bytes In||Files In||Tasks||Total Task Duration||Files Out of ListenHTTP|| |PostHTTP|24.22 GB|1.44 million|524|5:00 minutes|1.44 million| |InvokeHTTP|16.97 GB|6320|6320|5:00 minutes|6320| |PackageFlowFile + InvokeHTTP|25.36 GB|76|76|4:11 minutes|28,200| |PackageFlowFile 500 files < 1 MB + PackageFlowFile 1 file >= 1 MB + InvokeHTTP|26.47 GB|7000|7000|5.00 minutes|2.3 million| Conclusions: * PostHTTP and InvokeHTTP can get the same bytes-per-second throughput when very small files are not involved. * The scenario of packaging 100 byte and 10 MB files together in a flow may not be real world, but can definitely cause unsatisfactory results. * *Configured correctly, PackageFlowFile + InvokeHTTP can actually perform better than PostHTTP.* > Improve throughput performance for InvokeHTTP > - > > Key: NIFI-8968 > URL: https://issues.apache.org/jira/browse/NIFI-8968 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.14.0 >Reporter: Mark Bean >Priority: Major > Attachments: PostHTTP_vs_InvokeHTTP.json, PostHTTP_vs_InvokeHTTP.xml > > > InvokeHTTP is the preferred processor to use over the deprecated PostHTTP. > However, PostHTTP outperforms InvokeHTTP (at least in POST mode). A template > and a JSON file have been attached to this ticket for benchmarking the two > processors. Using this flow, PostHTTP was observed to have a throughput of > approximately 5 times greater than InvokeHTTP. > In addition, it was noted that InvokeHTTP had approximately 5 times as many > tasks and 5 times the task duration for a given 5 minute stats window. And, > the statistics of Bytes Read and Bytes Transferred remain at zero for > InvokeHTTP; this area accurate statistics also needs to be addressed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-8968) Improve throughput performance for InvokeHTTP
[ https://issues.apache.org/jira/browse/NIFI-8968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17808848#comment-17808848 ] Michael W Moser commented on NIFI-8968: --- With the addition of the PackageFlowFile processor, I decided to compare performance of PostHTTP and InvokeHTTP again. PackageFlowFile can be used to bundle flowfiles together and set the mime.type to application/flowfile-v3. When used with InvokeHTTP, this effectively is the same as PostHTTP's "send as flowfile" capability. PostHTTP will send many flowfiles in one batch (Max Batch Size = 100 MB), whereas InvokeHTTP will only send 1. This is why we see PostHTTP use a lot fewer Tasks compared to InvokeHTTP (because it uses 1 Task per batch). Also, with the flow definition attached to this ticket, both processors were provided 4 input queues for files of different sizes. Because of PostHTTP batching, the small files make it through the processor much quicker than InvokeHTTP. Here are my results: ||Scenario||Bytes In||Files In||Tasks||Total Task Duration||Files Out of ListenHTTP|| |PostHTTP|24.22 GB|1.44 million|524|5:00 minutes|1.44 million| |InvokeHTTP|16.97 GB|6320|6320|5:00 minutes|6320| |PackageFlowFile + InvokeHTTP|25.36 GB|76|76|4:11 minutes|28,200| |PackageFlowFile 500 files < 1 MB + PackageFlowFile 1 file >= 1 MB + InvokeHTTP|26.47 GB|7000|7000|5.00 minutes|2.3 million| Conclusions: * PostHTTP and InvokeHTTP can get the same bytes-per-second throughput when very small files are not involved. * The scenario of packaging 100 byte and 10 MB files together in a flow may not be real world, but can definitely cause unsatisfactory results. * *Configured correctly, PackageFlowFile + InvokeHTTP can actually perform better than PostHTTP.* > Improve throughput performance for InvokeHTTP > - > > Key: NIFI-8968 > URL: https://issues.apache.org/jira/browse/NIFI-8968 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.14.0 >Reporter: Mark Bean >Priority: Major > Attachments: PostHTTP_vs_InvokeHTTP.json, PostHTTP_vs_InvokeHTTP.xml > > > InvokeHTTP is the preferred processor to use over the deprecated PostHTTP. > However, PostHTTP outperforms InvokeHTTP (at least in POST mode). A template > and a JSON file have been attached to this ticket for benchmarking the two > processors. Using this flow, PostHTTP was observed to have a throughput of > approximately 5 times greater than InvokeHTTP. > In addition, it was noted that InvokeHTTP had approximately 5 times as many > tasks and 5 times the task duration for a given 5 minute stats window. And, > the statistics of Bytes Read and Bytes Transferred remain at zero for > InvokeHTTP; this area accurate statistics also needs to be addressed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-8606) Add Stop & Configure button to Controller Service Details dialog
[ https://issues.apache.org/jira/browse/NIFI-8606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael W Moser updated NIFI-8606: -- Fix Version/s: 1.25.0 2.0.0-M2 Resolution: Fixed Status: Resolved (was: Patch Available) > Add Stop & Configure button to Controller Service Details dialog > > > Key: NIFI-8606 > URL: https://issues.apache.org/jira/browse/NIFI-8606 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Affects Versions: 1.13.2 >Reporter: Mark Bean >Assignee: Reynaldo Rea >Priority: Minor > Fix For: 1.25.0, 2.0.0-M2 > > Time Spent: 1h 50m > Remaining Estimate: 0h > > This ticket is similar to NIFI-5986 which added the 'stop and configure' > option to a "Configure Processor" details dialog, i.e configure a processor. > The same feature should be available when in the "Controller Service > Details" dialog of a controller service. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12394) when importing versioned flow with component that migrates properties, controller service reference is invalid
[
https://issues.apache.org/jira/browse/NIFI-12394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael W Moser updated NIFI-12394:
---
Status: Patch Available (was: Open)
> when importing versioned flow with component that migrates properties,
> controller service reference is invalid
> --
>
> Key: NIFI-12394
> URL: https://issues.apache.org/jira/browse/NIFI-12394
> Project: Apache NiFi
> Issue Type: Bug
> Components: Flow Versioning
>Reporter: Michael W Moser
>Assignee: Michael W Moser
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> I built a Process Group containing one StandardRestrictedSSLContextService
> that is referenced by one InvokeHTTP processor. I downloaded that Process
> Group as a flow definition {*}with external services{*}. I also versioned
> that Process Group in NiFi Registry.
> Inside the flow definition file, I see the
> StandardRestrictedSSLContextService with
> "identifier":"d7d70b6c-abe4-3564-a219-b289cb7f25d2" and InvokeHTTP references
> that UUID.
> When I create a new Process Group using either the downloaded flow definition
> or the NiFi Registry flow, a new StandardRestrictedSSLContextService is
> created and it has a new UUID as expected. The InvokeHTTP processor is
> invalid because it references the proposed
> StandardRestrictedSSLContextService UUID d7d70b6c-abe4-3564-a219-b289cb7f25d2
> which does not exist.
> The service and processor are created and references are updated, but when
> migrating processor properties and any change occurs, the service reference
> is reverted back to what was in proposedProperties.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (NIFI-12394) when importing versioned flow with component that migrates properties, controller service reference is invalid
[
https://issues.apache.org/jira/browse/NIFI-12394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael W Moser reassigned NIFI-12394:
--
Assignee: Michael W Moser
> when importing versioned flow with component that migrates properties,
> controller service reference is invalid
> --
>
> Key: NIFI-12394
> URL: https://issues.apache.org/jira/browse/NIFI-12394
> Project: Apache NiFi
> Issue Type: Bug
> Components: Flow Versioning
>Reporter: Michael W Moser
>Assignee: Michael W Moser
>Priority: Major
>
> I built a Process Group containing one StandardRestrictedSSLContextService
> that is referenced by one InvokeHTTP processor. I downloaded that Process
> Group as a flow definition {*}with external services{*}. I also versioned
> that Process Group in NiFi Registry.
> Inside the flow definition file, I see the
> StandardRestrictedSSLContextService with
> "identifier":"d7d70b6c-abe4-3564-a219-b289cb7f25d2" and InvokeHTTP references
> that UUID.
> When I create a new Process Group using either the downloaded flow definition
> or the NiFi Registry flow, a new StandardRestrictedSSLContextService is
> created and it has a new UUID as expected. The InvokeHTTP processor is
> invalid because it references the proposed
> StandardRestrictedSSLContextService UUID d7d70b6c-abe4-3564-a219-b289cb7f25d2
> which does not exist.
> The service and processor are created and references are updated, but when
> migrating processor properties and any change occurs, the service reference
> is reverted back to what was in proposedProperties.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-12394) when importing versioned flow with component that migrates properties, controller service reference is invalid
[
https://issues.apache.org/jira/browse/NIFI-12394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17796885#comment-17796885
]
Michael W Moser commented on NIFI-12394:
Thanks [~markap14] I really appreciate your thoughts. I will work on a PR that
follows this guidance.
> when importing versioned flow with component that migrates properties,
> controller service reference is invalid
> --
>
> Key: NIFI-12394
> URL: https://issues.apache.org/jira/browse/NIFI-12394
> Project: Apache NiFi
> Issue Type: Bug
> Components: Flow Versioning
>Reporter: Michael W Moser
>Priority: Major
>
> I built a Process Group containing one StandardRestrictedSSLContextService
> that is referenced by one InvokeHTTP processor. I downloaded that Process
> Group as a flow definition {*}with external services{*}. I also versioned
> that Process Group in NiFi Registry.
> Inside the flow definition file, I see the
> StandardRestrictedSSLContextService with
> "identifier":"d7d70b6c-abe4-3564-a219-b289cb7f25d2" and InvokeHTTP references
> that UUID.
> When I create a new Process Group using either the downloaded flow definition
> or the NiFi Registry flow, a new StandardRestrictedSSLContextService is
> created and it has a new UUID as expected. The InvokeHTTP processor is
> invalid because it references the proposed
> StandardRestrictedSSLContextService UUID d7d70b6c-abe4-3564-a219-b289cb7f25d2
> which does not exist.
> The service and processor are created and references are updated, but when
> migrating processor properties and any change occurs, the service reference
> is reverted back to what was in proposedProperties.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12394) when importing versioned flow with component that migrates properties, controller service reference is invalid
[
https://issues.apache.org/jira/browse/NIFI-12394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael W Moser updated NIFI-12394:
---
Description:
I built a Process Group containing one StandardRestrictedSSLContextService that
is referenced by one InvokeHTTP processor. I downloaded that Process Group as
a flow definition {*}with external services{*}. I also versioned that Process
Group in NiFi Registry.
Inside the flow definition file, I see the StandardRestrictedSSLContextService
with "identifier":"d7d70b6c-abe4-3564-a219-b289cb7f25d2" and InvokeHTTP
references that UUID.
When I create a new Process Group using either the downloaded flow definition
or the NiFi Registry flow, a new StandardRestrictedSSLContextService is created
and it has a new UUID as expected. The InvokeHTTP processor is invalid because
it references the proposed StandardRestrictedSSLContextService UUID
d7d70b6c-abe4-3564-a219-b289cb7f25d2 which does not exist.
The service and processor are created and references are updated, but when
migrating processor properties and any change occurs, the service reference is
reverted back to what was in proposedProperties.
was:
I built a Process Group containing one StandardRestrictedSSLContextService that
is referenced by an InvokeHTTP processor. I downloaded that Process Group as a
flow definition {*}with external services{*}. I also versioned that Process
Group in NiFi Registry.
Inside the flow definition, I see the StandardRestrictedSSLContextService with
"identifier":"d7d70b6c-abe4-3564-a219-b289cb7f25d2" and InvokeHTTP references
that UUID.
When I create a new Process Group using either the downloaded flow definition
or the NiFi Registry flow, a new StandardRestrictedSSLContextService is created
and it has a new UUID as expected. The InvokeHTTP processor is invalid because
it references the old StandardRestrictedSSLContextService UUID
d7d70b6c-abe4-3564-a219-b289cb7f25d2 which does not exist.
It looks like when migrating processor properties and any change occurs, the
controller service reference is reverted.
> when importing versioned flow with component that migrates properties,
> controller service reference is invalid
> --
>
> Key: NIFI-12394
> URL: https://issues.apache.org/jira/browse/NIFI-12394
> Project: Apache NiFi
> Issue Type: Bug
> Components: Flow Versioning
>Reporter: Michael W Moser
>Priority: Major
>
> I built a Process Group containing one StandardRestrictedSSLContextService
> that is referenced by one InvokeHTTP processor. I downloaded that Process
> Group as a flow definition {*}with external services{*}. I also versioned
> that Process Group in NiFi Registry.
> Inside the flow definition file, I see the
> StandardRestrictedSSLContextService with
> "identifier":"d7d70b6c-abe4-3564-a219-b289cb7f25d2" and InvokeHTTP references
> that UUID.
> When I create a new Process Group using either the downloaded flow definition
> or the NiFi Registry flow, a new StandardRestrictedSSLContextService is
> created and it has a new UUID as expected. The InvokeHTTP processor is
> invalid because it references the proposed
> StandardRestrictedSSLContextService UUID d7d70b6c-abe4-3564-a219-b289cb7f25d2
> which does not exist.
> The service and processor are created and references are updated, but when
> migrating processor properties and any change occurs, the service reference
> is reverted back to what was in proposedProperties.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12394) when importing versioned flow with component that migrates properties, controller service reference is invalid
[
https://issues.apache.org/jira/browse/NIFI-12394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael W Moser updated NIFI-12394:
---
Description:
I built a Process Group containing one StandardRestrictedSSLContextService that
is referenced by an InvokeHTTP processor. I downloaded that Process Group as a
flow definition {*}with external services{*}. I also versioned that Process
Group in NiFi Registry.
Inside the flow definition, I see the StandardRestrictedSSLContextService with
"identifier":"d7d70b6c-abe4-3564-a219-b289cb7f25d2" and InvokeHTTP references
that UUID.
When I create a new Process Group using either the downloaded flow definition
or the NiFi Registry flow, a new StandardRestrictedSSLContextService is created
and it has a new UUID as expected. The InvokeHTTP processor is invalid because
it references the old StandardRestrictedSSLContextService UUID
d7d70b6c-abe4-3564-a219-b289cb7f25d2 which does not exist.
It looks like when migrating processor properties and any change occurs, the
controller service reference is reverted.
was:
I built a Process Group containing one StandardRestrictedSSLContextService that
is referenced by an InvokeHTTP processor. I downloaded that Process Group as a
flow definition {*}with external services{*}. I also versioned that Process
Group in NiFi Registry.
Inside the flow definition, I see the StandardRestrictedSSLContextService with
"identifier":"d7d70b6c-abe4-3564-a219-b289cb7f25d2" and InvokeHTTP references
that UUID.
When I create a new Process Group using either the downloaded flow definition
or the NiFi Registry flow, a new StandardRestrictedSSLContextService is created
and it has a new UUID as expected. The InvokeHTTP processor is invalid because
it references the old StandardRestrictedSSLContextService UUID
d7d70b6c-abe4-3564-a219-b289cb7f25d2 which does not exist.
> when importing versioned flow with component that migrates properties,
> controller service reference is invalid
> --
>
> Key: NIFI-12394
> URL: https://issues.apache.org/jira/browse/NIFI-12394
> Project: Apache NiFi
> Issue Type: Bug
> Components: Flow Versioning
>Reporter: Michael W Moser
>Priority: Major
>
> I built a Process Group containing one StandardRestrictedSSLContextService
> that is referenced by an InvokeHTTP processor. I downloaded that Process
> Group as a flow definition {*}with external services{*}. I also versioned
> that Process Group in NiFi Registry.
> Inside the flow definition, I see the StandardRestrictedSSLContextService
> with "identifier":"d7d70b6c-abe4-3564-a219-b289cb7f25d2" and InvokeHTTP
> references that UUID.
> When I create a new Process Group using either the downloaded flow definition
> or the NiFi Registry flow, a new StandardRestrictedSSLContextService is
> created and it has a new UUID as expected. The InvokeHTTP processor is
> invalid because it references the old StandardRestrictedSSLContextService
> UUID d7d70b6c-abe4-3564-a219-b289cb7f25d2 which does not exist.
> It looks like when migrating processor properties and any change occurs, the
> controller service reference is reverted.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12394) when importing versioned flow with component that migrates properties, controller service reference is invalid
[
https://issues.apache.org/jira/browse/NIFI-12394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael W Moser updated NIFI-12394:
---
Summary: when importing versioned flow with component that migrates
properties, controller service reference is invalid (was: when importing
versioned flow with InvokeHTTP, SSLContextService property is invalid)
> when importing versioned flow with component that migrates properties,
> controller service reference is invalid
> --
>
> Key: NIFI-12394
> URL: https://issues.apache.org/jira/browse/NIFI-12394
> Project: Apache NiFi
> Issue Type: Bug
> Components: Flow Versioning
>Reporter: Michael W Moser
>Priority: Major
>
> I built a Process Group containing one StandardRestrictedSSLContextService
> that is referenced by an InvokeHTTP processor. I downloaded that Process
> Group as a flow definition {*}with external services{*}. I also versioned
> that Process Group in NiFi Registry.
> Inside the flow definition, I see the StandardRestrictedSSLContextService
> with "identifier":"d7d70b6c-abe4-3564-a219-b289cb7f25d2" and InvokeHTTP
> references that UUID.
> When I create a new Process Group using either the downloaded flow definition
> or the NiFi Registry flow, a new StandardRestrictedSSLContextService is
> created and it has a new UUID as expected. The InvokeHTTP processor is
> invalid because it references the old StandardRestrictedSSLContextService
> UUID d7d70b6c-abe4-3564-a219-b289cb7f25d2 which does not exist.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12394) when importing versioned flow with InvokeHTTP, SSLContextService property is invalid
[
https://issues.apache.org/jira/browse/NIFI-12394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael W Moser updated NIFI-12394:
---
Description:
I built a Process Group containing one StandardRestrictedSSLContextService that
is referenced by an InvokeHTTP processor. I downloaded that Process Group as a
flow definition {*}with external services{*}. I also versioned that Process
Group in NiFi Registry.
Inside the flow definition, I see the StandardRestrictedSSLContextService with
"identifier":"d7d70b6c-abe4-3564-a219-b289cb7f25d2" and InvokeHTTP references
that UUID.
When I create a new Process Group using either the downloaded flow definition
or the NiFi Registry flow, a new StandardRestrictedSSLContextService is created
and it has a new UUID as expected. The InvokeHTTP processor is invalid because
it references the old StandardRestrictedSSLContextService UUID
d7d70b6c-abe4-3564-a219-b289cb7f25d2 which does not exist.
was:
I built a Process Group containing one StandardRestrictedSSLContextService that
is referenced by an InvokeHTTP and a ListenHTTP processor. I downloaded that
Process Group as a flow definition with external services. I also versioned
that Process Group in NiFi Registry.
Inside the flow definition, I see the StandardRestrictedSSLContextService with
"identifier":"d7d70b6c-abe4-3564-a219-b289cb7f25d2" and both InvokeHTTP and
ListenHTTP referencing that same UUID.
When I create a new Process Group using either the downloaded flow definition
or the NiFi Registry flow, the StandardRestrictedSSLContextService is invalid
as expected and it has a new UUID as expected. I enter the keystore and
truststore passwords and enable the service. Only one processor references the
new StandardRestrictedSSLContextService. The other processor is still invalid
because it references the old UUID d7d70b6c-abe4-3564-a219-b289cb7f25d2 which
does not exist.
edit: I tried this using a controller service that does not have sensitive
properties (XMLFileLookupService), and did not have this problem, so it could
be related to sensitive properties.
> when importing versioned flow with InvokeHTTP, SSLContextService property is
> invalid
>
>
> Key: NIFI-12394
> URL: https://issues.apache.org/jira/browse/NIFI-12394
> Project: Apache NiFi
> Issue Type: Bug
> Components: Flow Versioning
>Reporter: Michael W Moser
>Priority: Major
>
> I built a Process Group containing one StandardRestrictedSSLContextService
> that is referenced by an InvokeHTTP processor. I downloaded that Process
> Group as a flow definition {*}with external services{*}. I also versioned
> that Process Group in NiFi Registry.
> Inside the flow definition, I see the StandardRestrictedSSLContextService
> with "identifier":"d7d70b6c-abe4-3564-a219-b289cb7f25d2" and InvokeHTTP
> references that UUID.
> When I create a new Process Group using either the downloaded flow definition
> or the NiFi Registry flow, a new StandardRestrictedSSLContextService is
> created and it has a new UUID as expected. The InvokeHTTP processor is
> invalid because it references the old StandardRestrictedSSLContextService
> UUID d7d70b6c-abe4-3564-a219-b289cb7f25d2 which does not exist.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12394) when importing versioned flow with InvokeHTTP, SSLContextService property is invalid
[ https://issues.apache.org/jira/browse/NIFI-12394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael W Moser updated NIFI-12394: --- Summary: when importing versioned flow with InvokeHTTP, SSLContextService property is invalid (was: Versioned flow with multiple components referencing same controller service will only have 1 valid component after import) > when importing versioned flow with InvokeHTTP, SSLContextService property is > invalid > > > Key: NIFI-12394 > URL: https://issues.apache.org/jira/browse/NIFI-12394 > Project: Apache NiFi > Issue Type: Bug > Components: Flow Versioning >Reporter: Michael W Moser >Priority: Major > > I built a Process Group containing one StandardRestrictedSSLContextService > that is referenced by an InvokeHTTP and a ListenHTTP processor. I downloaded > that Process Group as a flow definition with external services. I also > versioned that Process Group in NiFi Registry. > Inside the flow definition, I see the StandardRestrictedSSLContextService > with "identifier":"d7d70b6c-abe4-3564-a219-b289cb7f25d2" and both InvokeHTTP > and ListenHTTP referencing that same UUID. > When I create a new Process Group using either the downloaded flow definition > or the NiFi Registry flow, the StandardRestrictedSSLContextService is invalid > as expected and it has a new UUID as expected. I enter the keystore and > truststore passwords and enable the service. Only one processor references > the new StandardRestrictedSSLContextService. The other processor is still > invalid because it references the old UUID > d7d70b6c-abe4-3564-a219-b289cb7f25d2 which does not exist. > edit: I tried this using a controller service that does not have sensitive > properties (XMLFileLookupService), and did not have this problem, so it could > be related to sensitive properties. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12468) Xodus Flow Configuration History does not recover from exception
[
https://issues.apache.org/jira/browse/NIFI-12468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17793011#comment-17793011
]
Michael W Moser commented on NIFI-12468:
Hi [~exceptionfactory] I was able to duplicate this with a simpler scenario.
Connect a debugger to NiFi and set a breakpoint in the UpdateAttribute
onPropertyModified method. Drop an UpdateAttribute on the graph and modify the
"Delete Attributes Expression" property. The breakpoint should be hit, and I
waited more than 5 minutes. Then I released the breakpoint and got the error.
I admit this is a contrived example, but exceptions happen in strange ways in
real world scenarios too. I thought it was best to report this, and if the
appropriate action is to restart NiFi, then at least this won't surprise us in
the future.
> Xodus Flow Configuration History does not recover from exception
>
>
> Key: NIFI-12468
> URL: https://issues.apache.org/jira/browse/NIFI-12468
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 2.0.0-M1
>Reporter: Michael W Moser
>Priority: Major
>
> I caused Xodus to timeout writing a transation to flow configuration history.
> Thereafter, every operation that would update flow configuration history, or
> trying to view flow configuration history, continues to throw an exception.
> The UI shows "An unexpected error has occurred. Please check the logs for
> additional details" based on a 500 response from the REST API
> /nifi-api/flow/history
> I had to restart NiFi to recover. Hopefully, this code can be improved to
> detect exceptions and recover on its own.
> I caused the initial problem by setting a breakpoint while a debugger was
> attached to NiFi. I dragged a new Process Group onto the graph from a flow
> definition file and let it sit at the breakpoint for a while.
> {noformat}
> 2023-12-04 16:32:00,448 ERROR [NiFi Web Server-466]
> jetbrains.exodus.env.EnvironmentImpl Failed to flush transaction
> java.io.IOException: Stream Closed
> at java.base/java.io.RandomAccessFile.length0(Native Method)
> at java.base/java.io.RandomAccessFile.length(RandomAccessFile.java:653)
> at jetbrains.exodus.io.FileDataWriter.ensureFile(FileDataWriter.kt:172)
> at jetbrains.exodus.io.FileDataWriter.write(FileDataWriter.kt:59)
> at
> jetbrains.exodus.log.BufferedDataWriter.writePage(BufferedDataWriter.java:264)
> at
> jetbrains.exodus.log.BufferedDataWriter.flush(BufferedDataWriter.java:176)
> at jetbrains.exodus.log.Log.flush(Log.kt:631)
> at jetbrains.exodus.log.Log.flush$default(Log.kt:630)
> at jetbrains.exodus.log.Log.flush(Log.kt)
> at
> jetbrains.exodus.env.EnvironmentImpl.flushTransaction(EnvironmentImpl.java:731)
> at
> jetbrains.exodus.env.EnvironmentImpl.commitTransaction(EnvironmentImpl.java:684)
> at
> jetbrains.exodus.env.ReadWriteTransaction.commit(ReadWriteTransaction.java:96)
> at
> jetbrains.exodus.entitystore.PersistentStoreTransaction.doCommit(PersistentStoreTransaction.java:192)
> at
> jetbrains.exodus.entitystore.PersistentStoreTransaction.commit(PersistentStoreTransaction.java:180)
> at
> jetbrains.exodus.entitystore.PersistentEntityStoreImpl.executeInExclusiveTransaction(PersistentEntityStoreImpl.java:720)
> at
> org.apache.nifi.admin.service.EntityStoreAuditService.addActions(EntityStoreAuditService.java:116)
> at org.apache.nifi.audit.NiFiAuditor.saveActions(NiFiAuditor.java:66)
> at org.apache.nifi.audit.NiFiAuditor.saveAction(NiFiAuditor.java:53)
> at
> org.apache.nifi.audit.ProcessGroupAuditor.saveUpdateAction(ProcessGroupAuditor.java:359)
> at
> org.apache.nifi.audit.ProcessGroupAuditor.updateProcessGroupFlowAdvice(ProcessGroupAuditor.java:305)
> at
> java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
> {noformat}
> Afterwards, any attempt to read or write from Xodus produces an exception
> similar to this:
> {noformat}
> 2023-12-04 16:32:53,111 WARN [NiFi Web Server-371]
> o.apache.nifi.audit.ProcessGroupAuditor Unable to record actions:
> jetbrains.exodus.ExodusException: Environment is inoperative
> at
> jetbrains.exodus.ExodusException.toExodusException(ExodusException.java:49)
> at
> jetbrains.exodus.env.EnvironmentImpl.checkIsOperative(EnvironmentImpl.java:1069)
> at
> jetbrains.exodus.env.EnvironmentImpl.beginTransaction(EnvironmentImpl.java:613)
> at
> jetbrains.exodus.env.EnvironmentImpl.beginExclusiveTransaction(EnvironmentImpl.java:264)
> at
> jetbrains.exodus.entitystore.PersistentStoreTransaction.(PersistentStoreTransaction.java:130)
> at
> jetbrains.exodus.entitystore.PersistentEntityStoreImpl.beginExclusiveTransaction(PersistentEntityStoreImpl.java:539)
> at
>
[jira] [Created] (NIFI-12468) Xodus Flow Configuration History does not recover from exception
Michael W Moser created NIFI-12468:
--
Summary: Xodus Flow Configuration History does not recover from
exception
Key: NIFI-12468
URL: https://issues.apache.org/jira/browse/NIFI-12468
Project: Apache NiFi
Issue Type: Bug
Affects Versions: 2.0.0-M1
Reporter: Michael W Moser
I caused Xodus to timeout writing a transation to flow configuration history.
Thereafter, every operation that would update flow configuration history, or
trying to view flow configuration history, continues to throw an exception.
The UI shows "An unexpected error has occurred. Please check the logs for
additional details" based on a 500 response from the REST API
/nifi-api/flow/history
I had to restart NiFi to recover. Hopefully, this code can be improved to
detect exceptions and recover on its own.
I caused the initial problem by setting a breakpoint while a debugger was
attached to NiFi. I dragged a new Process Group onto the graph from a flow
definition file and let it sit at the breakpoint for a while.
{noformat}
2023-12-04 16:32:00,448 ERROR [NiFi Web Server-466]
jetbrains.exodus.env.EnvironmentImpl Failed to flush transaction
java.io.IOException: Stream Closed
at java.base/java.io.RandomAccessFile.length0(Native Method)
at java.base/java.io.RandomAccessFile.length(RandomAccessFile.java:653)
at jetbrains.exodus.io.FileDataWriter.ensureFile(FileDataWriter.kt:172)
at jetbrains.exodus.io.FileDataWriter.write(FileDataWriter.kt:59)
at
jetbrains.exodus.log.BufferedDataWriter.writePage(BufferedDataWriter.java:264)
at
jetbrains.exodus.log.BufferedDataWriter.flush(BufferedDataWriter.java:176)
at jetbrains.exodus.log.Log.flush(Log.kt:631)
at jetbrains.exodus.log.Log.flush$default(Log.kt:630)
at jetbrains.exodus.log.Log.flush(Log.kt)
at
jetbrains.exodus.env.EnvironmentImpl.flushTransaction(EnvironmentImpl.java:731)
at
jetbrains.exodus.env.EnvironmentImpl.commitTransaction(EnvironmentImpl.java:684)
at
jetbrains.exodus.env.ReadWriteTransaction.commit(ReadWriteTransaction.java:96)
at
jetbrains.exodus.entitystore.PersistentStoreTransaction.doCommit(PersistentStoreTransaction.java:192)
at
jetbrains.exodus.entitystore.PersistentStoreTransaction.commit(PersistentStoreTransaction.java:180)
at
jetbrains.exodus.entitystore.PersistentEntityStoreImpl.executeInExclusiveTransaction(PersistentEntityStoreImpl.java:720)
at
org.apache.nifi.admin.service.EntityStoreAuditService.addActions(EntityStoreAuditService.java:116)
at org.apache.nifi.audit.NiFiAuditor.saveActions(NiFiAuditor.java:66)
at org.apache.nifi.audit.NiFiAuditor.saveAction(NiFiAuditor.java:53)
at
org.apache.nifi.audit.ProcessGroupAuditor.saveUpdateAction(ProcessGroupAuditor.java:359)
at
org.apache.nifi.audit.ProcessGroupAuditor.updateProcessGroupFlowAdvice(ProcessGroupAuditor.java:305)
at
java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
{noformat}
Afterwards, any attempt to read or write from Xodus produces an exception
similar to this:
{noformat}
2023-12-04 16:32:53,111 WARN [NiFi Web Server-371]
o.apache.nifi.audit.ProcessGroupAuditor Unable to record actions:
jetbrains.exodus.ExodusException: Environment is inoperative
at
jetbrains.exodus.ExodusException.toExodusException(ExodusException.java:49)
at
jetbrains.exodus.env.EnvironmentImpl.checkIsOperative(EnvironmentImpl.java:1069)
at
jetbrains.exodus.env.EnvironmentImpl.beginTransaction(EnvironmentImpl.java:613)
at
jetbrains.exodus.env.EnvironmentImpl.beginExclusiveTransaction(EnvironmentImpl.java:264)
at
jetbrains.exodus.entitystore.PersistentStoreTransaction.(PersistentStoreTransaction.java:130)
at
jetbrains.exodus.entitystore.PersistentEntityStoreImpl.beginExclusiveTransaction(PersistentEntityStoreImpl.java:539)
at
jetbrains.exodus.entitystore.PersistentEntityStoreImpl.executeInExclusiveTransaction(PersistentEntityStoreImpl.java:717)
at
org.apache.nifi.admin.service.EntityStoreAuditService.addActions(EntityStoreAuditService.java:116)
at org.apache.nifi.audit.NiFiAuditor.saveActions(NiFiAuditor.java:66)
at org.apache.nifi.audit.NiFiAuditor.saveAction(NiFiAuditor.java:53)
at
org.apache.nifi.audit.ProcessGroupAuditor.saveUpdateAction(ProcessGroupAuditor.java:359)
at
org.apache.nifi.audit.ProcessGroupAuditor.updateProcessGroupFlowAdvice(ProcessGroupAuditor.java:305)
at
java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
...
...
Caused by: java.io.IOException: Stream Closed
at java.base/java.io.RandomAccessFile.length0(Native Method)
at java.base/java.io.RandomAccessFile.length(RandomAccessFile.java:653)
at jetbrains.exodus.io.FileDataWriter.ensureFile(FileDataWriter.kt:172)
at
[jira] [Updated] (NIFI-12446) Align implementation of FilterAttribute with coding conventions
[
https://issues.apache.org/jira/browse/NIFI-12446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael W Moser updated NIFI-12446:
---
Labels: backport-needed (was: )
> Align implementation of FilterAttribute with coding conventions
> ---
>
> Key: NIFI-12446
> URL: https://issues.apache.org/jira/browse/NIFI-12446
> Project: Apache NiFi
> Issue Type: Improvement
>Reporter: endzeit
>Priority: Major
> Labels: backport-needed
> Attachments: image-2023-11-30-16-18-43-040.png
>
>
> The new processor {{FilterAttribute}} should be aligned with common coding
> conventions followed by other Processor components.
> This applies to the implementation both on the s{{{}upport/nifi-1.x{}}} and
> {{main}} branch, unless mentioned otherwise.
> Desired changes:
> * rename property "Filter mode" ("FILTER_MODE") to "Filter Mode" ("Filter
> Mode")
> * rename property "Attribute matching strategy" ("MATCHING_STRATEGY") to
> "Attribute Matching Strategy" ("Attribute Matching Strategy")
> * rename property "Set of attributes to filter" ("ATTRIBUTE_SET") to
> "Filtered Attributes" ("Filtered Attributes")
> * rename property "Regular expression to filter attributes"
> ("ATTRIBUTE_REGEX") to "Filtered Attributes Pattern" ("Filtered Attributes
> Pattern")
> * rename MatchingStrategy.REGEX to MatchingStrategy.PATTERN
> * use toString() on Enum entry instead of raw String, when declaring
> AllowableValue for Filter Mode
> * move field cachedMatchingPredicate before the method declarations
> * remove exclamation mark from IllegalArgumentExceptiion; put actual
> strategy value in [] (support/nifi-1.x only)
> * use Collections.unmodifiableList(Arrays.asList(...)) instead of
> _propertyDescriptors (support/nifi-1.x only)
> * replace !attributeName.trim().isEmpty() with
> attributeName.trim().isEmpty().not()
> * align AllowableValue value and displayName fields to the same string as
> "Title Case"
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (NIFI-12386) Add a FilterAttribute processor
[
https://issues.apache.org/jira/browse/NIFI-12386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael W Moser resolved NIFI-12386.
Fix Version/s: 1.25.0
2.0.0
Resolution: Implemented
> Add a FilterAttribute processor
> ---
>
> Key: NIFI-12386
> URL: https://issues.apache.org/jira/browse/NIFI-12386
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Core Framework
>Reporter: endzeit
>Assignee: endzeit
>Priority: Major
> Fix For: 1.25.0, 2.0.0
>
> Time Spent: 15h
> Remaining Estimate: 0h
>
> Flows in Apache NiFi can get quite sophisticated, consisting of a long chains
> of both {{ProcessGroup}} and {{Processor}} components.
> Oftentimes {{Processor}} components, including those in the NiFi standard
> bundle, enrich an incoming {{FlowFile}} with additional FlowFile attributes.
> This can lead to a fair amount of different FlowFile attributes accumulating
> over the FlowFile's lifecycle.
> In order to prevent subsequent {{ProcessGroup}} / {{Processor}} components to
> accidentally rely on implementation details of preceding components, a good
> practice is to:
> # define which FlowFile attributes should exist at selected points in the
> {{Flow}}
> # reduce the attributes of the FlowFile at the selected point to those
> defined
> This can be achieved by using the
> [UpdateAttribute|https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-update-attribute-nar/1.23.2/org.apache.nifi.processors.attributes.UpdateAttribute/index.html]
> processor of the standard processor bundle.
> However, the {{UpdateAttribute}} processor allows only for a regular
> expression to define a set of attributes to remove. Instead, the outlined
> practice above desires to explicitly state a set of attributes to keep. One
> can do so with a regular expression as well, but writing the reverse lookup
> to achieve this is not the easiest endeavor to put it mildly.
> This issue proposes a new processor {{FilterAttribute}} to be added to the
> library of {{{}nifi-standard-processors{}}}, which can be configured with a
> set of attributes and removes all attributes of an incoming FlowFile other
> than the ones configured.
> The processor should
> * have a required, non-blank property "Attributes to keep", which takes a
> list of attribute names separated by delimiter, e.g. comma (,).
> ** trailing whitespace around attribute names should be ignored
> ** leading or trailing delimiters should be ignored
> * have a required, non-blank property "Delimiter", which is used to delimit
> the individual attribute names, with a default value of "," (comma).
> * have a single relationship "success" to which all FlowFiles are routed,
> similar to {{UpdateAttribute}}
> * have an
> [InputRequirement|https://www.javadoc.io/doc/org.apache.nifi/nifi-api/latest/org/apache/nifi/annotation/behavior/InputRequirement.html]
> of
> [INPUT_REQUIRED|https://www.javadoc.io/doc/org.apache.nifi/nifi-api/latest/org/apache/nifi/annotation/behavior/InputRequirement.Requirement.html]
> *
> [@SupportsBatching|https://www.javadoc.io/doc/org.apache.nifi/nifi-api/latest/org/apache/nifi/annotation/behavior/SupportsBatching.html]
> * be
> [@SideEffectFree|https://www.javadoc.io/doc/org.apache.nifi/nifi-api/latest/org/apache/nifi/annotation/behavior/SideEffectFree.html]
> Some possible extension might be:
> * have a required property "Core attributes", with allowable values of "Keep
> UUID only", "Keep all", with a default of "Keep UUID only"
> ** an additional allowable value e.g. "Specify behaviour" may be added,
> which allows for more customization
> * have a required property "Mode", with allowable values of "Retain" and
> "Remove", with a default of "Retain"
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-12446) Align implementation of FilterAttribute with coding conventions
[
https://issues.apache.org/jira/browse/NIFI-12446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17791794#comment-17791794
]
Michael W Moser commented on NIFI-12446:
!image-2023-11-30-16-18-43-040.png!
Added to description: make AllowableValue value and displayName fields the
same, else we get what is shown in the image where "ENUMERATION" does not match
"Enumerate attributes" ... it's probably close enough but we have the chance to
make them the same.
> Align implementation of FilterAttribute with coding conventions
> ---
>
> Key: NIFI-12446
> URL: https://issues.apache.org/jira/browse/NIFI-12446
> Project: Apache NiFi
> Issue Type: Improvement
>Reporter: endzeit
>Priority: Major
> Attachments: image-2023-11-30-16-18-43-040.png
>
>
> The new processor {{FilterAttribute}} should be aligned with common coding
> conventions followed by other Processor components.
> This applies to the implementation both on the s{{{}upport/nifi-1.x{}}} and
> {{main}} branch, unless mentioned otherwise.
> Desired changes:
> * rename property "Filter mode" ("FILTER_MODE") to "Filter Mode" ("Filter
> Mode")
> * rename property "Attribute matching strategy" ("MATCHING_STRATEGY") to
> "Attribute Matching Strategy" ("Attribute Matching Strategy")
> * rename property "Set of attributes to filter" ("ATTRIBUTE_SET") to
> "Filtered Attributes" ("Filtered Attributes")
> * rename property "Regular expression to filter attributes"
> ("ATTRIBUTE_REGEX") to "Filtered Attributes Pattern" ("Filtered Attributes
> Pattern")
> * rename MatchingStrategy.REGEX to MatchingStrategy.PATTERN
> * use toString() on Enum entry instead of raw String, when declaring
> AllowableValue for Filter Mode
> * move field cachedMatchingPredicate before the method declarations
> * remove exclamation mark from IllegalArgumentExceptiion; put actual
> strategy value in [] (support/nifi-1.x only)
> * use Collections.unmodifiableList(Arrays.asList(...)) instead of
> _propertyDescriptors (support/nifi-1.x only)
> * replace !attributeName.trim().isEmpty() with
> attributeName.trim().isEmpty().not()
> * align AllowableValue value and displayName fields to the same string as
> "Title Case"
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12446) Align implementation of FilterAttribute with coding conventions
[
https://issues.apache.org/jira/browse/NIFI-12446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael W Moser updated NIFI-12446:
---
Attachment: image-2023-11-30-16-18-43-040.png
> Align implementation of FilterAttribute with coding conventions
> ---
>
> Key: NIFI-12446
> URL: https://issues.apache.org/jira/browse/NIFI-12446
> Project: Apache NiFi
> Issue Type: Improvement
>Reporter: endzeit
>Priority: Major
> Attachments: image-2023-11-30-16-18-43-040.png
>
>
> The new processor {{FilterAttribute}} should be aligned with common coding
> conventions followed by other Processor components.
> This applies to the implementation both on the s{{{}upport/nifi-1.x{}}} and
> {{main}} branch, unless mentioned otherwise.
> Desired changes:
> * rename property "Filter mode" ("FILTER_MODE") to "Filter Mode" ("Filter
> Mode")
> * rename property "Attribute matching strategy" ("MATCHING_STRATEGY") to
> "Attribute Matching Strategy" ("Attribute Matching Strategy")
> * rename property "Set of attributes to filter" ("ATTRIBUTE_SET") to
> "Filtered Attributes" ("Filtered Attributes")
> * rename property "Regular expression to filter attributes"
> ("ATTRIBUTE_REGEX") to "Filtered Attributes Pattern" ("Filtered Attributes
> Pattern")
> * rename MatchingStrategy.REGEX to MatchingStrategy.PATTERN
> * use toString() on Enum entry instead of raw String, when declaring
> AllowableValue for Filter Mode
> * move field cachedMatchingPredicate before the method declarations
> * remove exclamation mark from IllegalArgumentExceptiion; put actual
> strategy value in [] (support/nifi-1.x only)
> * use Collections.unmodifiableList(Arrays.asList(...)) instead of
> _propertyDescriptors (support/nifi-1.x only)
> * replace !attributeName.trim().isEmpty() with
> attributeName.trim().isEmpty().not()
> * align AllowableValue value and displayName fields to the same string as
> "Title Case"
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12446) Align implementation of FilterAttribute with coding conventions
[
https://issues.apache.org/jira/browse/NIFI-12446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael W Moser updated NIFI-12446:
---
Description:
The new processor {{FilterAttribute}} should be aligned with common coding
conventions followed by other Processor components.
This applies to the implementation both on the s{{{}upport/nifi-1.x{}}} and
{{main}} branch, unless mentioned otherwise.
Desired changes:
* rename property "Filter mode" ("FILTER_MODE") to "Filter Mode" ("Filter
Mode")
* rename property "Attribute matching strategy" ("MATCHING_STRATEGY") to
"Attribute Matching Strategy" ("Attribute Matching Strategy")
* rename property "Set of attributes to filter" ("ATTRIBUTE_SET") to "Filtered
Attributes" ("Filtered Attributes")
* rename property "Regular expression to filter attributes"
("ATTRIBUTE_REGEX") to "Filtered Attributes Pattern" ("Filtered Attributes
Pattern")
* rename MatchingStrategy.REGEX to MatchingStrategy.PATTERN
* use toString() on Enum entry instead of raw String, when declaring
AllowableValue for Filter Mode
* move field cachedMatchingPredicate before the method declarations
* remove exclamation mark from IllegalArgumentExceptiion; put actual strategy
value in [] (support/nifi-1.x only)
* use Collections.unmodifiableList(Arrays.asList(...)) instead of
_propertyDescriptors (support/nifi-1.x only)
* replace !attributeName.trim().isEmpty() with
attributeName.trim().isEmpty().not()
* align AllowableValue value and displayName fields to the same string as
"Title Case"
was:
The new processor {{FilterAttribute}} should be aligned with common coding
conventions followed by other Processor components.
This applies to the implementation both on the s{{{}upport/nifi-1.x{}}} and
{{main}} branch, unless mentioned otherwise.
Desired changes:
* rename property "Filter mode" ("FILTER_MODE") to "Filter Mode" ("Filter
Mode")
* rename property "Attribute matching strategy" ("MATCHING_STRATEGY") to
"Attribute Matching Strategy" ("Attribute Matching Strategy")
* rename property "Set of attributes to filter" ("ATTRIBUTE_SET") to "Filtered
Attributes" ("Filtered Attributes")
* rename property "Regular expression to filter attributes"
("ATTRIBUTE_REGEX") to "Filtered Attributes Pattern" ("Filtered Attributes
Pattern")
* rename MatchingStrategy.REGEX to MatchingStrategy.PATTERN
* use toString() on Enum entry instead of raw String, when declaring
AllowableValue for Filter Mode
* move field cachedMatchingPredicate before the method declarations
* remove exclamation mark from IllegalArgumentExceptiion; put actual strategy
value in [] (support/nifi-1.x only)
* use Collections.unmodifiableList(Arrays.asList(...)) instead of
_propertyDescriptors (support/nifi-1.x only)
* replace !attributeName.trim().isEmpty() with
attributeName.trim().isEmpty().not()
> Align implementation of FilterAttribute with coding conventions
> ---
>
> Key: NIFI-12446
> URL: https://issues.apache.org/jira/browse/NIFI-12446
> Project: Apache NiFi
> Issue Type: Improvement
>Reporter: endzeit
>Priority: Major
> Attachments: image-2023-11-30-16-18-43-040.png
>
>
> The new processor {{FilterAttribute}} should be aligned with common coding
> conventions followed by other Processor components.
> This applies to the implementation both on the s{{{}upport/nifi-1.x{}}} and
> {{main}} branch, unless mentioned otherwise.
> Desired changes:
> * rename property "Filter mode" ("FILTER_MODE") to "Filter Mode" ("Filter
> Mode")
> * rename property "Attribute matching strategy" ("MATCHING_STRATEGY") to
> "Attribute Matching Strategy" ("Attribute Matching Strategy")
> * rename property "Set of attributes to filter" ("ATTRIBUTE_SET") to
> "Filtered Attributes" ("Filtered Attributes")
> * rename property "Regular expression to filter attributes"
> ("ATTRIBUTE_REGEX") to "Filtered Attributes Pattern" ("Filtered Attributes
> Pattern")
> * rename MatchingStrategy.REGEX to MatchingStrategy.PATTERN
> * use toString() on Enum entry instead of raw String, when declaring
> AllowableValue for Filter Mode
> * move field cachedMatchingPredicate before the method declarations
> * remove exclamation mark from IllegalArgumentExceptiion; put actual
> strategy value in [] (support/nifi-1.x only)
> * use Collections.unmodifiableList(Arrays.asList(...)) instead of
> _propertyDescriptors (support/nifi-1.x only)
> * replace !attributeName.trim().isEmpty() with
> attributeName.trim().isEmpty().not()
> * align AllowableValue value and displayName fields to the same string as
> "Title Case"
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12394) Versioned flow with multiple components referencing same controller service will only have 1 valid component after import
[ https://issues.apache.org/jira/browse/NIFI-12394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael W Moser updated NIFI-12394: --- Description: I built a Process Group containing one StandardRestrictedSSLContextService that is referenced by an InvokeHTTP and a ListenHTTP processor. I downloaded that Process Group as a flow definition with external services. I also versioned that Process Group in NiFi Registry. Inside the flow definition, I see the StandardRestrictedSSLContextService with "identifier":"d7d70b6c-abe4-3564-a219-b289cb7f25d2" and both InvokeHTTP and ListenHTTP referencing that same UUID. When I create a new Process Group using either the downloaded flow definition or the NiFi Registry flow, the StandardRestrictedSSLContextService is invalid as expected and it has a new UUID as expected. I enter the keystore and truststore passwords and enable the service. Only one processor references the new StandardRestrictedSSLContextService. The other processor is still invalid because it references the old UUID d7d70b6c-abe4-3564-a219-b289cb7f25d2 which does not exist. edit: I tried this using a controller service that does not have sensitive properties (XMLFileLookupService), and did not have this problem, so it could be related to sensitive properties. was: I built a Process Group containing one StandardRestrictedSSLContextService that is referenced by an InvokeHTTP and a ListenHTTP processor. I downloaded that Process Group as a flow definition with external services. I also versioned that Process Group in NiFi Registry. Inside the flow definition, I see the StandardRestrictedSSLContextService with "identifier":"d7d70b6c-abe4-3564-a219-b289cb7f25d2" and both InvokeHTTP and ListenHTTP referencing that same UUID. When I create a new Process Group using either the downloaded flow definition or the NiFi Registry flow, the StandardRestrictedSSLContextService is invalid as expected and it has a new UUID as expected. I enter the keystore and truststore passwords and enable the service. Only one processor references the new StandardRestrictedSSLContextService. The other processor is still invalid because it references the old UUID d7d70b6c-abe4-3564-a219-b289cb7f25d2 which does not exist. edit: I tried this using a controller service that does not have sensitive properties, and did not have this problem, so it could be related to sensitive properties. > Versioned flow with multiple components referencing same controller service > will only have 1 valid component after import > - > > Key: NIFI-12394 > URL: https://issues.apache.org/jira/browse/NIFI-12394 > Project: Apache NiFi > Issue Type: Bug > Components: Flow Versioning >Reporter: Michael W Moser >Priority: Major > > I built a Process Group containing one StandardRestrictedSSLContextService > that is referenced by an InvokeHTTP and a ListenHTTP processor. I downloaded > that Process Group as a flow definition with external services. I also > versioned that Process Group in NiFi Registry. > Inside the flow definition, I see the StandardRestrictedSSLContextService > with "identifier":"d7d70b6c-abe4-3564-a219-b289cb7f25d2" and both InvokeHTTP > and ListenHTTP referencing that same UUID. > When I create a new Process Group using either the downloaded flow definition > or the NiFi Registry flow, the StandardRestrictedSSLContextService is invalid > as expected and it has a new UUID as expected. I enter the keystore and > truststore passwords and enable the service. Only one processor references > the new StandardRestrictedSSLContextService. The other processor is still > invalid because it references the old UUID > d7d70b6c-abe4-3564-a219-b289cb7f25d2 which does not exist. > edit: I tried this using a controller service that does not have sensitive > properties (XMLFileLookupService), and did not have this problem, so it could > be related to sensitive properties. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12394) Versioned flow with multiple components referencing same controller service will only have 1 valid component after import
[ https://issues.apache.org/jira/browse/NIFI-12394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael W Moser updated NIFI-12394: --- Description: I built a Process Group containing one StandardRestrictedSSLContextService that is referenced by an InvokeHTTP and a ListenHTTP processor. I downloaded that Process Group as a flow definition with external services. I also versioned that Process Group in NiFi Registry. Inside the flow definition, I see the StandardRestrictedSSLContextService with "identifier":"d7d70b6c-abe4-3564-a219-b289cb7f25d2" and both InvokeHTTP and ListenHTTP referencing that same UUID. When I create a new Process Group using either the downloaded flow definition or the NiFi Registry flow, the StandardRestrictedSSLContextService is invalid as expected and it has a new UUID as expected. I enter the keystore and truststore passwords and enable the service. Only one processor references the new StandardRestrictedSSLContextService. The other processor is still invalid because it references the old UUID d7d70b6c-abe4-3564-a219-b289cb7f25d2 which does not exist. edit: I tried this using a controller service that does not have sensitive properties, and did not have this problem, so it could be related to sensitive properties. was: I built a Process Group containing one StandardRestrictedSSLContextService that is referenced by an InvokeHTTP and a ListenHTTP processor. I downloaded that Process Group as a flow definition with external services. I also versioned that Process Group in NiFi Registry. Inside the flow definition, I see the StandardRestrictedSSLContextService with "identifier":"d7d70b6c-abe4-3564-a219-b289cb7f25d2" and both InvokeHTTP and ListenHTTP referencing that same UUID. When I create a new Process Group using either the downloaded flow definition or the NiFi Registry flow, the StandardRestrictedSSLContextService is invalid as expected and it has a new UUID as expected. I enter the keystore and truststore passwords and enable the service. Only one processor references the new StandardRestrictedSSLContextService. The other processor is still invalid because it references the old UUID d7d70b6c-abe4-3564-a219-b289cb7f25d2 which does not exist. > Versioned flow with multiple components referencing same controller service > will only have 1 valid component after import > - > > Key: NIFI-12394 > URL: https://issues.apache.org/jira/browse/NIFI-12394 > Project: Apache NiFi > Issue Type: Bug > Components: Flow Versioning >Reporter: Michael W Moser >Priority: Major > > I built a Process Group containing one StandardRestrictedSSLContextService > that is referenced by an InvokeHTTP and a ListenHTTP processor. I downloaded > that Process Group as a flow definition with external services. I also > versioned that Process Group in NiFi Registry. > Inside the flow definition, I see the StandardRestrictedSSLContextService > with "identifier":"d7d70b6c-abe4-3564-a219-b289cb7f25d2" and both InvokeHTTP > and ListenHTTP referencing that same UUID. > When I create a new Process Group using either the downloaded flow definition > or the NiFi Registry flow, the StandardRestrictedSSLContextService is invalid > as expected and it has a new UUID as expected. I enter the keystore and > truststore passwords and enable the service. Only one processor references > the new StandardRestrictedSSLContextService. The other processor is still > invalid because it references the old UUID > d7d70b6c-abe4-3564-a219-b289cb7f25d2 which does not exist. > edit: I tried this using a controller service that does not have sensitive > properties, and did not have this problem, so it could be related to > sensitive properties. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-12394) Versioned flow with multiple components referencing same controller service will only have 1 valid component after import
Michael W Moser created NIFI-12394: -- Summary: Versioned flow with multiple components referencing same controller service will only have 1 valid component after import Key: NIFI-12394 URL: https://issues.apache.org/jira/browse/NIFI-12394 Project: Apache NiFi Issue Type: Bug Components: Flow Versioning Reporter: Michael W Moser I built a Process Group containing one StandardRestrictedSSLContextService that is referenced by an InvokeHTTP and a ListenHTTP processor. I downloaded that Process Group as a flow definition with external services. I also versioned that Process Group in NiFi Registry. Inside the flow definition, I see the StandardRestrictedSSLContextService with "identifier":"d7d70b6c-abe4-3564-a219-b289cb7f25d2" and both InvokeHTTP and ListenHTTP referencing that same UUID. When I create a new Process Group using either the downloaded flow definition or the NiFi Registry flow, the StandardRestrictedSSLContextService is invalid as expected and it has a new UUID as expected. I enter the keystore and truststore passwords and enable the service. Only one processor references the new StandardRestrictedSSLContextService. The other processor is still invalid because it references the old UUID d7d70b6c-abe4-3564-a219-b289cb7f25d2 which does not exist. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12290) Move from Quartz to Spring Cron Expression
[
https://issues.apache.org/jira/browse/NIFI-12290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael W Moser updated NIFI-12290:
---
Fix Version/s: 2.0.0
(was: 2.latest)
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Move from Quartz to Spring Cron Expression
> --
>
> Key: NIFI-12290
> URL: https://issues.apache.org/jira/browse/NIFI-12290
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Major
> Fix For: 2.0.0
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> The Quartz library supports [cron|https://en.wikipedia.org/wiki/Cron]
> expression parsing to enable framework component scheduling using the Cron
> strategy.
> The Quartz library includes several legacy dependencies on HikariCP and C3P0
> which are not necessary for framework operations. The Quartz library also
> relies on {{java.util.Date}} for calculation, as opposed to {{java.time}}
> components.
> Spring Framework 5.3 introduced new [Cron Expression
> capabilities|https://spring.io/blog/2020/11/10/new-in-spring-5-3-improved-cron-expressions]
> that provide standard compatibility with [Cron scheduling
> expressions|https://docs.spring.io/spring-framework/docs/current/javadoc-api/org/springframework/scheduling/support/CronExpression.html].
> The Spring CronExpression supports standard {{java.time}} classes.
> The Spring
> [CronExpression|https://docs.spring.io/spring-framework/docs/current/javadoc-api/org/springframework/scheduling/support/CronExpression.html]
> has two differences from the Quartz implementation:
> 1. The Year field is not supported
> 2. The numeric day of the week representation begins with 0 instead of 1
> These differences follow the capabilities of the standard Unix crontab, and
> are not commonly used. However, these differences could invalidate custom
> scheduling configurations that used numeric day of the week or year values.
> Based on existing NiFI framework use of Spring, Quartz should be replaced
> with Spring Cron Expression parsing.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-2517) Apply ordering to DNs from certificates
[
https://issues.apache.org/jira/browse/NIFI-2517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17779636#comment-17779636
]
Michael W Moser commented on NIFI-2517:
---
One use case in favor of this is to input an RFC-2253 formatted DN into
authorizers.xml Initial Admin Identity only to have it not match the
X500Principal name of the actual certificate because NiFi only accepts RFC-1779
formatting.
> Apply ordering to DNs from certificates
> ---
>
> Key: NIFI-2517
> URL: https://issues.apache.org/jira/browse/NIFI-2517
> Project: Apache NiFi
> Issue Type: Improvement
>Affects Versions: 1.0.0, 0.7.0
>Reporter: Bryan Bende
>Priority: Minor
>
> Currently when a user authenticates to NiFi with a certificate, the DN is
> extracted with the following code from SubjectDnX509PrincipalExtractor:
> {code}
> public Object extractPrincipal(X509Certificate cert) {
> return cert.getSubjectDN().getName().trim();
> }
> {code}
> This string ends up being the user identity that needs to line up with
> policies.
> It is not guaranteed that the subject DN from a certificate will always be in
> a known format. For example, one cert can put the CN before the OU, and
> another can put the OU before the CN. Different tools can also display the
> same DN in different orders, such as openssl vs keytool.
> NiFi should be able to apply a re-ordering of the DNs so that after passing
> through the X509 authentication code, the app can then assume the DN is in a
> known order. We should also consider how this interacts with the identity
> mapping concept introduced in 1.0.0.
> In addition we are currently using getSubjectDN() from X509 certificate and
> the Java Doc says:
> {code}
> * Denigrated, replaced by {@linkplain
> * #getSubjectX500Principal()}. This method returns the {@code subject}
> * as an implementation specific Principal object, which should not be
> * relied upon by portable code.
> {code}
> So we may want to look at moving away from that method.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-12272) Add Standard Certificate Principal Formatting
[
https://issues.apache.org/jira/browse/NIFI-12272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17779633#comment-17779633
]
Michael W Moser commented on NIFI-12272:
I have known users that input an RFC-2253 formatted DN into authorizers.xml
Initial Admin Identity only to be frustrated that NiFi only accepts RFC-1779
formatting. I'll make this note on that Jira ticket as well.
> Add Standard Certificate Principal Formatting
> -
>
> Key: NIFI-12272
> URL: https://issues.apache.org/jira/browse/NIFI-12272
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework, Extensions
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Major
> Fix For: 2.0.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> The {{X509Certificate.getSubjectDN()}} is denigrated in favor of
> {{getSubjectX500Principal()}} and recent improvements to the main branch
> replaced legacy usage.
> The getSubjectDN method returns Distinguished Names formatted according to
> RFC 1779, which includes separating Relative Distinguished Name elements
> using spaces. This impacts configuration properties such as Identity Mapping,
> and also impacts flow designs that evaluate Certificate Subject and Issuer
> attributes.
> The default behavior of {{X500Principal.getName()}} formats Distinguished
> Names according to RFC 2253, which does not use space separators. Passing
> {{X500Principal.RFC1779}} to the {{getName()}} method follows the same
> formatting approach as {{{}getSubjectDN().getName(){}}}.
> Existing behavior should be maintained to avoid unexpected changes when
> upgrading NiFi versions, and a new utility should be introduced to provide
> standardized formatting.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-12272) Add Standard Certificate Principal Formatting
[
https://issues.apache.org/jira/browse/NIFI-12272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17779586#comment-17779586
]
Michael W Moser commented on NIFI-12272:
Thanks for finding and resolving this, [~exceptionfactory]. It certainly would
have caused complications upgrading to 2.0.0
I still would like to see NIFI-2517 implemented (I'll put it on my TODO list)
to relax the matching of certificate DNs in the various places in NiFi that
user identities can be configured.
> Add Standard Certificate Principal Formatting
> -
>
> Key: NIFI-12272
> URL: https://issues.apache.org/jira/browse/NIFI-12272
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework, Extensions
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Major
> Fix For: 2.0.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The {{X509Certificate.getSubjectDN()}} is denigrated in favor of
> {{getSubjectX500Principal()}} and recent improvements to the main branch
> replaced legacy usage.
> The getSubjectDN method returns Distinguished Names formatted according to
> RFC 1779, which includes separating Relative Distinguished Name elements
> using spaces. This impacts configuration properties such as Identity Mapping,
> and also impacts flow designs that evaluate Certificate Subject and Issuer
> attributes.
> The default behavior of {{X500Principal.getName()}} formats Distinguished
> Names according to RFC 2253, which does not use space separators. Passing
> {{X500Principal.RFC1779}} to the {{getName()}} method follows the same
> formatting approach as {{{}getSubjectDN().getName(){}}}.
> Existing behavior should be maintained to avoid unexpected changes when
> upgrading NiFi versions, and a new utility should be introduced to provide
> standardized formatting.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-12165) JoltTransformJSON Processor - Vanishing properties
[ https://issues.apache.org/jira/browse/NIFI-12165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17779270#comment-17779270 ] Michael W Moser commented on NIFI-12165: I resolved this ticket. Thanks for letting us know [~dstiegli1] > JoltTransformJSON Processor - Vanishing properties > -- > > Key: NIFI-12165 > URL: https://issues.apache.org/jira/browse/NIFI-12165 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.21.0 > Environment: Docker >Reporter: Benji Benning >Assignee: Daniel Stieglitz >Priority: Minor > Fix For: 2.0.0, 1.24.0 > > Attachments: JoltTransformJSON.mp4 > > Time Spent: 40m > Remaining Estimate: 0h > > When selecting any Property to edit (then either accepting or canceling the > change) the "Custom Transformation Class Name" and the "Custom Module > Directory" properties vanish. > Note that the change does seem to get applied. > See attached video -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12165) JoltTransformJSON Processor - Vanishing properties
[ https://issues.apache.org/jira/browse/NIFI-12165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael W Moser updated NIFI-12165: --- Fix Version/s: 2.0.0 1.24.0 Resolution: Fixed Status: Resolved (was: Patch Available) > JoltTransformJSON Processor - Vanishing properties > -- > > Key: NIFI-12165 > URL: https://issues.apache.org/jira/browse/NIFI-12165 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.21.0 > Environment: Docker >Reporter: Benji Benning >Assignee: Daniel Stieglitz >Priority: Minor > Fix For: 2.0.0, 1.24.0 > > Attachments: JoltTransformJSON.mp4 > > Time Spent: 40m > Remaining Estimate: 0h > > When selecting any Property to edit (then either accepting or canceling the > change) the "Custom Transformation Class Name" and the "Custom Module > Directory" properties vanish. > Note that the change does seem to get applied. > See attached video -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12121) Update JSLTTransformJSON to output null-value JSON objects
[
https://issues.apache.org/jira/browse/NIFI-12121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17778833#comment-17778833
]
Michael W Moser commented on NIFI-12121:
[~nickdos] I marked this PR for the main branch (2.x). Please let us know if
this new feature is desired for a 1.x release. Thanks.
> Update JSLTTransformJSON to output null-value JSON objects
> --
>
> Key: NIFI-12121
> URL: https://issues.apache.org/jira/browse/NIFI-12121
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Affects Versions: 1.23.2
>Reporter: Nick dos Remedios
>Assignee: Michael W Moser
>Priority: Minor
> Fix For: 2.latest
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The current implementation of the JSLTTransformJSON processor is configured
> using the default {{com.schibsted.spt.data.jslt.Parser}} "filter"
> configuration that results in the removal of any JSON objects where the value
> is `null`.
> When processing data in Nifi with the "infer schema" option for a
> reader/writer controller service, this results in some {*}data not conforming
> to the "original" schema{*}, once it passed through the JSLTTransformJSON
> processor. As any JSON objects with a null value are removed from the final
> JSON output. See [this issue|https://github.com/schibsted/jslt/issues/308]
> that illustrates this problem.
> The [JSLT API allows for the ability to
> configure|https://github.com/schibsted/jslt/blob/master/docs/api.md#object-key-filter]
> whether null-value objects are retained or remove. I've checked the Nifi
> JSLTTransformJSON source and it does not currently set
> {{.withObjectFilter(filter)}} on the {{Parse}} object.
> I'd like to see the JSLTTransformJSON processor be updated with a
> user-selectable option to include null-value objects in the output (or not).
> Which would optionally set {{{}Parser.withObjectFilter(true){}}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12121) Update JSLTTransformJSON to output null-value JSON objects
[
https://issues.apache.org/jira/browse/NIFI-12121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael W Moser updated NIFI-12121:
---
Fix Version/s: 2.latest
Status: Patch Available (was: In Progress)
> Update JSLTTransformJSON to output null-value JSON objects
> --
>
> Key: NIFI-12121
> URL: https://issues.apache.org/jira/browse/NIFI-12121
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Affects Versions: 1.23.2
>Reporter: Nick dos Remedios
>Assignee: Michael W Moser
>Priority: Minor
> Fix For: 2.latest
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The current implementation of the JSLTTransformJSON processor is configured
> using the default {{com.schibsted.spt.data.jslt.Parser}} "filter"
> configuration that results in the removal of any JSON objects where the value
> is `null`.
> When processing data in Nifi with the "infer schema" option for a
> reader/writer controller service, this results in some {*}data not conforming
> to the "original" schema{*}, once it passed through the JSLTTransformJSON
> processor. As any JSON objects with a null value are removed from the final
> JSON output. See [this issue|https://github.com/schibsted/jslt/issues/308]
> that illustrates this problem.
> The [JSLT API allows for the ability to
> configure|https://github.com/schibsted/jslt/blob/master/docs/api.md#object-key-filter]
> whether null-value objects are retained or remove. I've checked the Nifi
> JSLTTransformJSON source and it does not currently set
> {{.withObjectFilter(filter)}} on the {{Parse}} object.
> I'd like to see the JSLTTransformJSON processor be updated with a
> user-selectable option to include null-value objects in the output (or not).
> Which would optionally set {{{}Parser.withObjectFilter(true){}}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (NIFI-12121) Update JSLTTransformJSON to output null-value JSON objects
[
https://issues.apache.org/jira/browse/NIFI-12121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael W Moser reassigned NIFI-12121:
--
Assignee: Michael W Moser
> Update JSLTTransformJSON to output null-value JSON objects
> --
>
> Key: NIFI-12121
> URL: https://issues.apache.org/jira/browse/NIFI-12121
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Affects Versions: 1.23.2
>Reporter: Nick dos Remedios
>Assignee: Michael W Moser
>Priority: Minor
>
> The current implementation of the JSLTTransformJSON processor is configured
> using the default {{com.schibsted.spt.data.jslt.Parser}} "filter"
> configuration that results in the removal of any JSON objects where the value
> is `null`.
> When processing data in Nifi with the "infer schema" option for a
> reader/writer controller service, this results in some {*}data not conforming
> to the "original" schema{*}, once it passed through the JSLTTransformJSON
> processor. As any JSON objects with a null value are removed from the final
> JSON output. See [this issue|https://github.com/schibsted/jslt/issues/308]
> that illustrates this problem.
> The [JSLT API allows for the ability to
> configure|https://github.com/schibsted/jslt/blob/master/docs/api.md#object-key-filter]
> whether null-value objects are retained or remove. I've checked the Nifi
> JSLTTransformJSON source and it does not currently set
> {{.withObjectFilter(filter)}} on the {{Parse}} object.
> I'd like to see the JSLTTransformJSON processor be updated with a
> user-selectable option to include null-value objects in the output (or not).
> Which would optionally set {{{}Parser.withObjectFilter(true){}}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12242) Allow ControlRate to optionally route to a new 'rate exceeded' relationship instead of just slowing things down
[ https://issues.apache.org/jira/browse/NIFI-12242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael W Moser updated NIFI-12242: --- Fix Version/s: 2.0.0 (was: 2.latest) Resolution: Fixed Status: Resolved (was: Patch Available) > Allow ControlRate to optionally route to a new 'rate exceeded' relationship > instead of just slowing things down > --- > > Key: NIFI-12242 > URL: https://issues.apache.org/jira/browse/NIFI-12242 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 2.0.0 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > ControlRate provides the ability to slow down the rate of data through the > system. This is important in cases where we want to avoid overwhelming a > downstream system. However, there are times when we don't want to just slow > down the flow of data but instead reject it if the rate is too high. > Take, for example, the typical case with an HTTP Server that responds with a > 429: Too Many Requests status code. ControlRate should give us the option of > rejecting FlowFiles when they exceed the limit, not just slow everything down. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12237) Label size does not update when changing version
[ https://issues.apache.org/jira/browse/NIFI-12237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael W Moser updated NIFI-12237: --- Fix Version/s: 1.latest 2.latest > Label size does not update when changing version > > > Key: NIFI-12237 > URL: https://issues.apache.org/jira/browse/NIFI-12237 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.22.0, 1.23.2 >Reporter: Michael W Moser >Assignee: Michael W Moser >Priority: Minor > Fix For: 1.latest, 2.latest > > Time Spent: 10m > Remaining Estimate: 0h > > * in a versioned PG containing a label, resize a label width and height. > diff shows two "Position was changed" difference (one is width, the second is > height) > * move the label. diff shows a third "Position was changed" (the actual > position) > * commit local changes > * change version back one version. label moves but does not resize. > * PG still indicates local changes made. revert local changes does not work > * delete label. revert local changes now works. label now shows correct > position and size > * if you change version forward one version, same thing happens > I think this behavior is because label height and width is considered a > position, and NIFI-11473 changed that position differences are not included > in "updatedVersionedComponentIds". > [https://github.com/apache/nifi/blob/main/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-components/src/main/java/org/apache/nifi/flow/synchronization/StandardVersionedComponentSynchronizer.java#L216] > [https://github.com/apache/nifi/blob/main/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-components/src/main/java/org/apache/nifi/flow/synchronization/StandardVersionedComponentSynchronizer.java#L908] > I propose to make label width and height a SIZE rather than a POSITION change. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12237) Label size does not update when changing version
[ https://issues.apache.org/jira/browse/NIFI-12237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael W Moser updated NIFI-12237: --- Status: Patch Available (was: In Progress) > Label size does not update when changing version > > > Key: NIFI-12237 > URL: https://issues.apache.org/jira/browse/NIFI-12237 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.23.2, 1.22.0 >Reporter: Michael W Moser >Assignee: Michael W Moser >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > * in a versioned PG containing a label, resize a label width and height. > diff shows two "Position was changed" difference (one is width, the second is > height) > * move the label. diff shows a third "Position was changed" (the actual > position) > * commit local changes > * change version back one version. label moves but does not resize. > * PG still indicates local changes made. revert local changes does not work > * delete label. revert local changes now works. label now shows correct > position and size > * if you change version forward one version, same thing happens > I think this behavior is because label height and width is considered a > position, and NIFI-11473 changed that position differences are not included > in "updatedVersionedComponentIds". > [https://github.com/apache/nifi/blob/main/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-components/src/main/java/org/apache/nifi/flow/synchronization/StandardVersionedComponentSynchronizer.java#L216] > [https://github.com/apache/nifi/blob/main/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-components/src/main/java/org/apache/nifi/flow/synchronization/StandardVersionedComponentSynchronizer.java#L908] > I propose to make label width and height a SIZE rather than a POSITION change. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-12237) Label size does not update when changing version
Michael W Moser created NIFI-12237: -- Summary: Label size does not update when changing version Key: NIFI-12237 URL: https://issues.apache.org/jira/browse/NIFI-12237 Project: Apache NiFi Issue Type: Bug Affects Versions: 1.23.2, 1.22.0 Reporter: Michael W Moser Assignee: Michael W Moser * in a versioned PG containing a label, resize a label width and height. diff shows two "Position was changed" difference (one is width, the second is height) * move the label. diff shows a third "Position was changed" (the actual position) * commit local changes * change version back one version. label moves but does not resize. * PG still indicates local changes made. revert local changes does not work * delete label. revert local changes now works. label now shows correct position and size * if you change version forward one version, same thing happens I think this behavior is because label height and width is considered a position, and NIFI-11473 changed that position differences are not included in "updatedVersionedComponentIds". [https://github.com/apache/nifi/blob/main/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-components/src/main/java/org/apache/nifi/flow/synchronization/StandardVersionedComponentSynchronizer.java#L216] [https://github.com/apache/nifi/blob/main/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-components/src/main/java/org/apache/nifi/flow/synchronization/StandardVersionedComponentSynchronizer.java#L908] I propose to make label width and height a SIZE rather than a POSITION change. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12230) GeoEnrichIP shouldn't produce a message when an IP is not found
[ https://issues.apache.org/jira/browse/NIFI-12230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17775750#comment-17775750 ] Michael W Moser commented on NIFI-12230: Recommendation from Apache NiFi #general Slack channel is to change the log level from WARN to INFO for this message: [https://github.com/apache/nifi/blob/main/nifi-nar-bundles/nifi-enrich-bundle/nifi-enrich-processors/src/main/java/org/apache/nifi/processors/GeoEnrichIP.java#L139] "Failure while trying to find enrichment data" This will remove the bulletin in default configurations of GeoEnrichIP. > GeoEnrichIP shouldn't produce a message when an IP is not found > --- > > Key: NIFI-12230 > URL: https://issues.apache.org/jira/browse/NIFI-12230 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Didier B. >Priority: Major > > GeoEnrichIP is a processor that convert an IP (found in the flowfile > attribute) to a geolocation information that it put in attributes. To do so > it requires a database file. > In many environments, it's normal to have IP not found in the database > (internal IP, unknown IP, and so on). This is an information that concern the > rest of the workflow if needed. > The problem here is that for every IP not found, GeoEnrichIP make a log warn > call which imply that a bulletin is emitted. This bulletin will trigger the > red box (upper right corner of the processor) up to the root flowgraph, > warning everyone working on the flowgraph that there is a problem, which is > not and will also emit a log that will flood the log system. > *When an IP is not found, routing to failure is enough. No bulletin should be > emitted. Not finding an IP is normal and can happen several times per second.* -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12165) JoltTransformJSON Processor - Vanishing properties
[ https://issues.apache.org/jira/browse/NIFI-12165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774240#comment-17774240 ] Michael W Moser commented on NIFI-12165: Confirmed. The properties "Custom Transformation Class Name" and "Custom Module Directory" should depend on the "Jolt Transformation DSL = Custom". However, the dependency is incorrectly set to depend on "Jolt Specification" property. As an experiment, if you set "Jolt Specification = jolt-transform-custom", you will see the two custom properties come back. > JoltTransformJSON Processor - Vanishing properties > -- > > Key: NIFI-12165 > URL: https://issues.apache.org/jira/browse/NIFI-12165 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.21.0 > Environment: Docker >Reporter: Benji Benning >Priority: Minor > Attachments: JoltTransformJSON.mp4 > > > When selecting any Property to edit (then either accepting or canceling the > change) the "Custom Transformation Class Name" and the "Custom Module > Directory" properties vanish. > Note that the change does seem to get applied. > See attached video -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12213) Since nifi-bom was introduced, using LdapUserGroupProvider causes NiFi to fail to start
[
https://issues.apache.org/jira/browse/NIFI-12213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774235#comment-17774235
]
Michael W Moser commented on NIFI-12213:
I didn't find any other places where nifi-utils needed an explicit dependency.
I did find that nifi-twitter-processors and nifi-networking-processors have an
explicit dependency on nifi-utils that they probably don't need, but leaving
them alone doesn't harm anything (nifi-utils does not get included in their
nar).
> Since nifi-bom was introduced, using LdapUserGroupProvider causes NiFi to
> fail to start
> ---
>
> Key: NIFI-12213
> URL: https://issues.apache.org/jira/browse/NIFI-12213
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Michael W Moser
>Assignee: Michael W Moser
>Priority: Major
> Fix For: 2.latest
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> I built the main branch and tried to use the LdapUserGroupProvider in
> authorizers.xml. I get a ClassNotFoundException looking for FormatUtils in
> nifi-utils.jar. The nifi-bom marks nifi-utils as provided by
> nifi-standard-services-api-nar but nifi-ldap-iaa-providers-nar doesn't depend
> on nifi-standard-services-api-nar.
> {noformat}
> Caused by: java.lang.NoClassDefFoundError: org/apache/nifi/util/FormatUtils
> at
> org.apache.nifi.ldap.tenants.LdapUserGroupProvider.setTimeout(LdapUserGroupProvider.java:824)
> at
> org.apache.nifi.ldap.tenants.LdapUserGroupProvider.onConfigured(LdapUserGroupProvider.java:166)
> at
> java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
> at java.base/java.lang.reflect.Method.invoke(Method.java:580)
> at
> org.apache.nifi.authorization.UserGroupProviderInvocationHandler.invoke(UserGroupProviderInvocationHandler.java:38)
> at jdk.proxy5/jdk.proxy5.$Proxy59.onConfigured(Unknown Source)
> at
> org.apache.nifi.authorization.AuthorizerFactoryBean.loadProviderProperties(AuthorizerFactoryBean.java:198)
> at
> org.apache.nifi.authorization.AuthorizerFactoryBean.getObject(AuthorizerFactoryBean.java:167)
> at
> org.apache.nifi.authorization.AuthorizerFactoryBean.getObject(AuthorizerFactoryBean.java:71)
> at
> org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:169)
> ... 106 common frames omitted
> Caused by: java.lang.ClassNotFoundException: org.apache.nifi.util.FormatUtils
> at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:445)
> at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:593)
> at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:526)
> ... 116 common frames omitted
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12213) Since nifi-bom was introduced, using LdapUserGroupProvider causes NiFi to fail to start
[
https://issues.apache.org/jira/browse/NIFI-12213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael W Moser updated NIFI-12213:
---
Status: Patch Available (was: Open)
> Since nifi-bom was introduced, using LdapUserGroupProvider causes NiFi to
> fail to start
> ---
>
> Key: NIFI-12213
> URL: https://issues.apache.org/jira/browse/NIFI-12213
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Michael W Moser
>Assignee: Michael W Moser
>Priority: Major
> Fix For: 2.latest
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> I built the main branch and tried to use the LdapUserGroupProvider in
> authorizers.xml. I get a ClassNotFoundException looking for FormatUtils in
> nifi-utils.jar. The nifi-bom marks nifi-utils as provided by
> nifi-standard-services-api-nar but nifi-ldap-iaa-providers-nar doesn't depend
> on nifi-standard-services-api-nar.
> {noformat}
> Caused by: java.lang.NoClassDefFoundError: org/apache/nifi/util/FormatUtils
> at
> org.apache.nifi.ldap.tenants.LdapUserGroupProvider.setTimeout(LdapUserGroupProvider.java:824)
> at
> org.apache.nifi.ldap.tenants.LdapUserGroupProvider.onConfigured(LdapUserGroupProvider.java:166)
> at
> java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
> at java.base/java.lang.reflect.Method.invoke(Method.java:580)
> at
> org.apache.nifi.authorization.UserGroupProviderInvocationHandler.invoke(UserGroupProviderInvocationHandler.java:38)
> at jdk.proxy5/jdk.proxy5.$Proxy59.onConfigured(Unknown Source)
> at
> org.apache.nifi.authorization.AuthorizerFactoryBean.loadProviderProperties(AuthorizerFactoryBean.java:198)
> at
> org.apache.nifi.authorization.AuthorizerFactoryBean.getObject(AuthorizerFactoryBean.java:167)
> at
> org.apache.nifi.authorization.AuthorizerFactoryBean.getObject(AuthorizerFactoryBean.java:71)
> at
> org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:169)
> ... 106 common frames omitted
> Caused by: java.lang.ClassNotFoundException: org.apache.nifi.util.FormatUtils
> at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:445)
> at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:593)
> at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:526)
> ... 116 common frames omitted
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (NIFI-12213) Since nifi-bom was introduced, using LdapUserGroupProvider causes NiFi to fail to start
[
https://issues.apache.org/jira/browse/NIFI-12213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael W Moser reassigned NIFI-12213:
--
Assignee: Michael W Moser
> Since nifi-bom was introduced, using LdapUserGroupProvider causes NiFi to
> fail to start
> ---
>
> Key: NIFI-12213
> URL: https://issues.apache.org/jira/browse/NIFI-12213
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Michael W Moser
>Assignee: Michael W Moser
>Priority: Major
> Fix For: 2.latest
>
>
> I built the main branch and tried to use the LdapUserGroupProvider in
> authorizers.xml. I get a ClassNotFoundException looking for FormatUtils in
> nifi-utils.jar. The nifi-bom marks nifi-utils as provided by
> nifi-standard-services-api-nar but nifi-ldap-iaa-providers-nar doesn't depend
> on nifi-standard-services-api-nar.
> {noformat}
> Caused by: java.lang.NoClassDefFoundError: org/apache/nifi/util/FormatUtils
> at
> org.apache.nifi.ldap.tenants.LdapUserGroupProvider.setTimeout(LdapUserGroupProvider.java:824)
> at
> org.apache.nifi.ldap.tenants.LdapUserGroupProvider.onConfigured(LdapUserGroupProvider.java:166)
> at
> java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
> at java.base/java.lang.reflect.Method.invoke(Method.java:580)
> at
> org.apache.nifi.authorization.UserGroupProviderInvocationHandler.invoke(UserGroupProviderInvocationHandler.java:38)
> at jdk.proxy5/jdk.proxy5.$Proxy59.onConfigured(Unknown Source)
> at
> org.apache.nifi.authorization.AuthorizerFactoryBean.loadProviderProperties(AuthorizerFactoryBean.java:198)
> at
> org.apache.nifi.authorization.AuthorizerFactoryBean.getObject(AuthorizerFactoryBean.java:167)
> at
> org.apache.nifi.authorization.AuthorizerFactoryBean.getObject(AuthorizerFactoryBean.java:71)
> at
> org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:169)
> ... 106 common frames omitted
> Caused by: java.lang.ClassNotFoundException: org.apache.nifi.util.FormatUtils
> at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:445)
> at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:593)
> at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:526)
> ... 116 common frames omitted
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (NIFI-12213) Since nifi-bom was introduced, using LdapUserGroupProvider causes NiFi to fail to start
[
https://issues.apache.org/jira/browse/NIFI-12213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774205#comment-17774205
]
Michael W Moser edited comment on NIFI-12213 at 10/11/23 7:19 PM:
--
I added nifi-utils as a compile dependency in the nifi-ldap-iaa-providers-nar
pom.xml and this fixes it.
I wonder if any other nar is affected?
[edit] looks like nifi-kerberos-iaa-providers-nar needs this too
was (Author: mosermw):
I added nifi-utils as a compile dependency in the nifi-ldap-iaa-providers-nar
pom.xml and this fixes it.
I wonder if any other nar is affected?
> Since nifi-bom was introduced, using LdapUserGroupProvider causes NiFi to
> fail to start
> ---
>
> Key: NIFI-12213
> URL: https://issues.apache.org/jira/browse/NIFI-12213
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Michael W Moser
>Priority: Major
> Fix For: 2.latest
>
>
> I built the main branch and tried to use the LdapUserGroupProvider in
> authorizers.xml. I get a ClassNotFoundException looking for FormatUtils in
> nifi-utils.jar. The nifi-bom marks nifi-utils as provided by
> nifi-standard-services-api-nar but nifi-ldap-iaa-providers-nar doesn't depend
> on nifi-standard-services-api-nar.
> {noformat}
> Caused by: java.lang.NoClassDefFoundError: org/apache/nifi/util/FormatUtils
> at
> org.apache.nifi.ldap.tenants.LdapUserGroupProvider.setTimeout(LdapUserGroupProvider.java:824)
> at
> org.apache.nifi.ldap.tenants.LdapUserGroupProvider.onConfigured(LdapUserGroupProvider.java:166)
> at
> java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
> at java.base/java.lang.reflect.Method.invoke(Method.java:580)
> at
> org.apache.nifi.authorization.UserGroupProviderInvocationHandler.invoke(UserGroupProviderInvocationHandler.java:38)
> at jdk.proxy5/jdk.proxy5.$Proxy59.onConfigured(Unknown Source)
> at
> org.apache.nifi.authorization.AuthorizerFactoryBean.loadProviderProperties(AuthorizerFactoryBean.java:198)
> at
> org.apache.nifi.authorization.AuthorizerFactoryBean.getObject(AuthorizerFactoryBean.java:167)
> at
> org.apache.nifi.authorization.AuthorizerFactoryBean.getObject(AuthorizerFactoryBean.java:71)
> at
> org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:169)
> ... 106 common frames omitted
> Caused by: java.lang.ClassNotFoundException: org.apache.nifi.util.FormatUtils
> at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:445)
> at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:593)
> at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:526)
> ... 116 common frames omitted
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-12213) Since nifi-bom was introduced, using LdapUserGroupProvider causes NiFi to fail to start
[
https://issues.apache.org/jira/browse/NIFI-12213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774205#comment-17774205
]
Michael W Moser commented on NIFI-12213:
I added nifi-utils as a compile dependency in the nifi-ldap-iaa-providers-nar
pom.xml and this fixes it.
I wonder if any other nar is affected?
> Since nifi-bom was introduced, using LdapUserGroupProvider causes NiFi to
> fail to start
> ---
>
> Key: NIFI-12213
> URL: https://issues.apache.org/jira/browse/NIFI-12213
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Michael W Moser
>Priority: Major
> Fix For: 2.latest
>
>
> I built the main branch and tried to use the LdapUserGroupProvider in
> authorizers.xml. I get a ClassNotFoundException looking for FormatUtils in
> nifi-utils.jar. The nifi-bom marks nifi-utils as provided by
> nifi-standard-services-api-nar but nifi-ldap-iaa-providers-nar doesn't depend
> on nifi-standard-services-api-nar.
> {noformat}
> Caused by: java.lang.NoClassDefFoundError: org/apache/nifi/util/FormatUtils
> at
> org.apache.nifi.ldap.tenants.LdapUserGroupProvider.setTimeout(LdapUserGroupProvider.java:824)
> at
> org.apache.nifi.ldap.tenants.LdapUserGroupProvider.onConfigured(LdapUserGroupProvider.java:166)
> at
> java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
> at java.base/java.lang.reflect.Method.invoke(Method.java:580)
> at
> org.apache.nifi.authorization.UserGroupProviderInvocationHandler.invoke(UserGroupProviderInvocationHandler.java:38)
> at jdk.proxy5/jdk.proxy5.$Proxy59.onConfigured(Unknown Source)
> at
> org.apache.nifi.authorization.AuthorizerFactoryBean.loadProviderProperties(AuthorizerFactoryBean.java:198)
> at
> org.apache.nifi.authorization.AuthorizerFactoryBean.getObject(AuthorizerFactoryBean.java:167)
> at
> org.apache.nifi.authorization.AuthorizerFactoryBean.getObject(AuthorizerFactoryBean.java:71)
> at
> org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:169)
> ... 106 common frames omitted
> Caused by: java.lang.ClassNotFoundException: org.apache.nifi.util.FormatUtils
> at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:445)
> at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:593)
> at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:526)
> ... 116 common frames omitted
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (NIFI-12213) Since nifi-bom was introduced, using LdapUserGroupProvider causes NiFi to fail to start
Michael W Moser created NIFI-12213:
--
Summary: Since nifi-bom was introduced, using
LdapUserGroupProvider causes NiFi to fail to start
Key: NIFI-12213
URL: https://issues.apache.org/jira/browse/NIFI-12213
Project: Apache NiFi
Issue Type: Bug
Reporter: Michael W Moser
Fix For: 2.latest
I built the main branch and tried to use the LdapUserGroupProvider in
authorizers.xml. I get a ClassNotFoundException looking for FormatUtils in
nifi-utils.jar. The nifi-bom marks nifi-utils as provided by
nifi-standard-services-api-nar but nifi-ldap-iaa-providers-nar doesn't depend
on nifi-standard-services-api-nar.
{noformat}
Caused by: java.lang.NoClassDefFoundError: org/apache/nifi/util/FormatUtils
at
org.apache.nifi.ldap.tenants.LdapUserGroupProvider.setTimeout(LdapUserGroupProvider.java:824)
at
org.apache.nifi.ldap.tenants.LdapUserGroupProvider.onConfigured(LdapUserGroupProvider.java:166)
at
java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at
org.apache.nifi.authorization.UserGroupProviderInvocationHandler.invoke(UserGroupProviderInvocationHandler.java:38)
at jdk.proxy5/jdk.proxy5.$Proxy59.onConfigured(Unknown Source)
at
org.apache.nifi.authorization.AuthorizerFactoryBean.loadProviderProperties(AuthorizerFactoryBean.java:198)
at
org.apache.nifi.authorization.AuthorizerFactoryBean.getObject(AuthorizerFactoryBean.java:167)
at
org.apache.nifi.authorization.AuthorizerFactoryBean.getObject(AuthorizerFactoryBean.java:71)
at
org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:169)
... 106 common frames omitted
Caused by: java.lang.ClassNotFoundException: org.apache.nifi.util.FormatUtils
at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:445)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:593)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:526)
... 116 common frames omitted
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
