[jira] [Created] (MINIFICPP-2423) Fix RHEL8 bootstrap issues
Marton Szasz created MINIFICPP-2423: --- Summary: Fix RHEL8 bootstrap issues Key: MINIFICPP-2423 URL: https://issues.apache.org/jira/browse/MINIFICPP-2423 Project: Apache NiFi MiNiFi C++ Issue Type: Bug Reporter: Marton Szasz -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-13460) Conduct Apache NiFi 1.27.0 Release
[ https://issues.apache.org/jira/browse/NIFI-13460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17860645#comment-17860645 ] ASF subversion and git services commented on NIFI-13460: Commit bd86d1f07fe8d0f45b72e91e925ee85ffa7ed143 in nifi's branch refs/heads/NIFI-13460-RC1 from Pierre Villard [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=bd86d1f07f ] NIFI-13460-RC1 prepare release nifi-1.27.0-RC1 > Conduct Apache NiFi 1.27.0 Release > -- > > Key: NIFI-13460 > URL: https://issues.apache.org/jira/browse/NIFI-13460 > Project: Apache NiFi > Issue Type: Task >Affects Versions: 1.27.0 >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Major > Fix For: 1.27.0 > > > Conduct Apache NiFi 1.27.0 Release -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-13460) Conduct Apache NiFi 1.27.0 Release
[ https://issues.apache.org/jira/browse/NIFI-13460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17860646#comment-17860646 ] ASF subversion and git services commented on NIFI-13460: Commit 5962ce30e4f5e9d60799c43c2848200395c466f9 in nifi's branch refs/heads/NIFI-13460-RC1 from Pierre Villard [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=5962ce30e4 ] NIFI-13460-RC1 prepare for next development iteration > Conduct Apache NiFi 1.27.0 Release > -- > > Key: NIFI-13460 > URL: https://issues.apache.org/jira/browse/NIFI-13460 > Project: Apache NiFi > Issue Type: Task >Affects Versions: 1.27.0 >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Major > Fix For: 1.27.0 > > > Conduct Apache NiFi 1.27.0 Release -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-13462) Conduct Apache NiFi 2.0.0-M4 Release
David Handermann created NIFI-13462: --- Summary: Conduct Apache NiFi 2.0.0-M4 Release Key: NIFI-13462 URL: https://issues.apache.org/jira/browse/NIFI-13462 Project: Apache NiFi Issue Type: Task Reporter: David Handermann Assignee: David Handermann Fix For: 2.0.0-M4 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (NIFI-8916) Modify windows-service-assembly profile to be disabled by default
[ https://issues.apache.org/jira/browse/NIFI-8916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hunter Goller reassigned NIFI-8916: --- Assignee: Sash Sujith > Modify windows-service-assembly profile to be disabled by default > - > > Key: NIFI-8916 > URL: https://issues.apache.org/jira/browse/NIFI-8916 > Project: Apache NiFi > Issue Type: Improvement > Components: Tools and Build >Affects Versions: 1.14.0 >Reporter: Mark Bean >Assignee: Sash Sujith >Priority: Major > > The windows-service-assembly profile is in minifi/minifi-assembly/pom.xml. It > is enabled by default. Now that minifi source code has been migrated to the > nifi project as a module, there may be a majority of users buliding nifi for > non-Windows systems. > Recommend this profile be disabled by default. > This change should include updates to any relevant READMEs, Dev guides, etc. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13427) Extend the NiFi Python API to support Python-based source processors
[ https://issues.apache.org/jira/browse/NIFI-13427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-13427: Fix Version/s: 2.0.0-M4 Resolution: Fixed Status: Resolved (was: Patch Available) > Extend the NiFi Python API to support Python-based source processors > > > Key: NIFI-13427 > URL: https://issues.apache.org/jira/browse/NIFI-13427 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework, Extensions >Reporter: Peter Gyori >Assignee: Peter Gyori >Priority: Major > Fix For: 2.0.0-M4 > > > Extend the NiFi Python API so that one can write in Python a "source" > processor. A "source" processor is one that has no incoming connections and > the processor alone creates new flow files. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-13427) Extend the NiFi Python API to support Python-based source processors
[ https://issues.apache.org/jira/browse/NIFI-13427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17860610#comment-17860610 ] ASF subversion and git services commented on NIFI-13427: Commit 3efb0763da4b2cb123011c46d2dccb1b1786e8af in nifi's branch refs/heads/main from Peter Gyori [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=3efb0763da ] NIFI-13427 Added FlowFileSource interface for Python Processors This closes #9000 Signed-off-by: David Handermann > Extend the NiFi Python API to support Python-based source processors > > > Key: NIFI-13427 > URL: https://issues.apache.org/jira/browse/NIFI-13427 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework, Extensions >Reporter: Peter Gyori >Assignee: Peter Gyori >Priority: Major > > Extend the NiFi Python API so that one can write in Python a "source" > processor. A "source" processor is one that has no incoming connections and > the processor alone creates new flow files. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-13452) GetSlackReaction processor to fetch reactions
[ https://issues.apache.org/jira/browse/NIFI-13452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17860602#comment-17860602 ] Joe Witt commented on NIFI-13452: - https://github.com/apache/nifi/pull/9010 > GetSlackReaction processor to fetch reactions > - > > Key: NIFI-13452 > URL: https://issues.apache.org/jira/browse/NIFI-13452 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Zsihovszki Krisztina >Assignee: Zsihovszki Krisztina >Priority: Major > > The goal of this JIRA is to create a new processor so it'd possible to be > fetch the emojis added to a message in Slack. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-13452) GetSlackReaction processor to fetch reactions
[ https://issues.apache.org/jira/browse/NIFI-13452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17860601#comment-17860601 ] Joe Witt commented on NIFI-13452: - [~kzsihovszki] OK great. Then can you please describe the use case and then how it'd be setup in a nifi flow? For example is this what you mean to convey. ? Today NiFi allows users to create flows that listen to and consume slack messages which they can then optionally enrich, transform, and deliver to a downstream system or service. And they can also optionally consume reactions associated with messages but it currently would gather all reactions. There are times when a user might want to gather reactions only to a specific message like when it is talking about a certain topic or contains a certain tag like '#alert'. The emojis should be optionally captured and only for specific messages. ... a description like that then allows for discussion/consideration of implementation options. Right now we have an implementation option without knowing the why. Do I understand the use case intended here? If so I'd suggest we put that logic in consume slack and not a separate processor. If I dont can you please explain the use case. Thanks > GetSlackReaction processor to fetch reactions > - > > Key: NIFI-13452 > URL: https://issues.apache.org/jira/browse/NIFI-13452 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Zsihovszki Krisztina >Assignee: Zsihovszki Krisztina >Priority: Major > > The goal of this JIRA is to create a new processor so it'd possible to be > fetch the emojis added to a message in Slack. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13461) Add DeleteFile processor (backport 1.x)
[ https://issues.apache.org/jira/browse/NIFI-13461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] endzeit updated NIFI-13461: --- Description: Backport the introduction of `DeleteFile` in NIFI-12280 nto the 1.x branch. (was: Backport the introduction of `DeleteFile` in iNIFI-12280 nto the 1.x branch. ) > Add DeleteFile processor (backport 1.x) > --- > > Key: NIFI-13461 > URL: https://issues.apache.org/jira/browse/NIFI-13461 > Project: Apache NiFi > Issue Type: New Feature >Reporter: endzeit >Priority: Major > > Backport the introduction of `DeleteFile` in NIFI-12280 nto the 1.x branch. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13461) Add DeleteFile processor (backport 1.x)
[ https://issues.apache.org/jira/browse/NIFI-13461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] endzeit updated NIFI-13461: --- Description: Backport the introduction of `DeleteFile` in NIFI-12880 into the 1.x branch. (was: Backport the introduction of `DeleteFile` in NIFI-12280 nto the 1.x branch. ) > Add DeleteFile processor (backport 1.x) > --- > > Key: NIFI-13461 > URL: https://issues.apache.org/jira/browse/NIFI-13461 > Project: Apache NiFi > Issue Type: New Feature >Reporter: endzeit >Priority: Major > > Backport the introduction of `DeleteFile` in NIFI-12880 into the 1.x branch. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-13461) Add DeleteFile processor (backport 1.x)
endzeit created NIFI-13461: -- Summary: Add DeleteFile processor (backport 1.x) Key: NIFI-13461 URL: https://issues.apache.org/jira/browse/NIFI-13461 Project: Apache NiFi Issue Type: New Feature Reporter: endzeit Backport the introduction of `DeleteFile` in iNIFI-12280 nto the 1.x branch. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-13460) Conduct Apache NiFi 1.27.0 Release
Pierre Villard created NIFI-13460: - Summary: Conduct Apache NiFi 1.27.0 Release Key: NIFI-13460 URL: https://issues.apache.org/jira/browse/NIFI-13460 Project: Apache NiFi Issue Type: Task Affects Versions: 1.27.0 Reporter: Pierre Villard Assignee: Pierre Villard Fix For: 1.27.0 Conduct Apache NiFi 1.27.0 Release -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-13273) Release NiFi NAR Maven Plugin 2.0.0
[ https://issues.apache.org/jira/browse/NIFI-13273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard resolved NIFI-13273. --- Resolution: Fixed > Release NiFi NAR Maven Plugin 2.0.0 > --- > > Key: NIFI-13273 > URL: https://issues.apache.org/jira/browse/NIFI-13273 > Project: Apache NiFi > Issue Type: Task > Components: Tools and Build >Affects Versions: nifi-nar-maven-plugin-2.0.0 >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Major > Fix For: nifi-nar-maven-plugin-2.0.0 > > > Release NiFi NAR Maven Plugin 1.6.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13304) Add SplitExcel Processor
[ https://issues.apache.org/jira/browse/NIFI-13304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13304: -- Fix Version/s: 2.0.0-M4 > Add SplitExcel Processor > > > Key: NIFI-13304 > URL: https://issues.apache.org/jira/browse/NIFI-13304 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Philipp Korniets >Assignee: Daniel Stieglitz >Priority: Major > Fix For: 2.0.0-M4, 1.27.0 > > > Currently ConvertExcelToCSV processor, if processing excel with multiple > required sheets in it, produces flowfile per sheet. > New ExcelReader produces one flowfile with all data in it > > There should be a way to configure output to be sheet-per-flowfile > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12231) Add completion strategy to FetchSmb
[ https://issues.apache.org/jira/browse/NIFI-12231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-12231: -- Fix Version/s: 2.0.0-M3 > Add completion strategy to FetchSmb > --- > > Key: NIFI-12231 > URL: https://issues.apache.org/jira/browse/NIFI-12231 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Affects Versions: 1.23.2, 1.26.0 >Reporter: Jens M Kofoed >Assignee: Peter Turcsanyi >Priority: Major > Fix For: 2.0.0-M3, 1.27.0 > > Time Spent: 50m > Remaining Estimate: 0h > > The newly added ListSmb and FetchSmb has been very much appreciated. But > sadly the FetchSmb does not have the same options like other Fetch processors. > FetchFTP and FetchSFTP process has a Completion Strategy where you can choose > to Move or Delete the file after it has been fetched. > It would be very helpfull if FetchSmb gets the same possibilities -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12704) Record Path function escapeJson raises NPE when given root "/" as argument
[
https://issues.apache.org/jira/browse/NIFI-12704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-12704:
--
Fix Version/s: 2.0.0-M4
> Record Path function escapeJson raises NPE when given root "/" as argument
> --
>
> Key: NIFI-12704
> URL: https://issues.apache.org/jira/browse/NIFI-12704
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.24.0, 2.0.0-M2
> Environment: Docker, RedHat 8
>Reporter: Stephen Jeffrey Hindmarch
>Assignee: endzeit
>Priority: Minor
> Fix For: 2.0.0-M4, 1.27.0
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> My use case is I want to create a field called "original" which is the
> escaped string of my whole record. This will preserve the original contents
> of the message before I start transforming it.
> My expectation is that I should be able to use an UpdateRecord processor to
> create the field using RecordPath language.
> What actually happens is that when I use
> {code:java}
> escapeJson(/)
> {code}
> as the function the result is that the processor throws a Null Pointer
> Exception (NPE).
> Detail:
> For any input flow file containing JSON records, define an UpdateRecord
> processor with these settings.
> * Replacement Value Strategy = Record Path Value
> * /original = escapeJson(/)
> When the flow file is presented to the processor, the processor generates an
> NPE.
> For example, I present this record.
> {noformat}
> [{"hello":"world","record":{"key":"one","value":"hello","subrecord":{"key":"two","value":"bob"}},"array":[0,1,2]}]{noformat}
> If the escapeJson function is offered an existing field to escape then it
> works as expected. Stack trace as follows.
> {noformat}
> 2024-01-31 12:17:15 2024-01-31 12:17:14,741 ERROR [Timer-Driven Process
> Thread-8] o.a.n.processors.standard.UpdateRecord
> UpdateRecord[id=5f6b498d-018d-1000--323c6b89] Failed to process
> StandardFlowFileRecord[uuid=2d4a3eab-e752-4117-93d3-3e5515b6f3f8,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1706703050563-1, container=default,
> section=1], offset=114,
> length=114],offset=0,name=2d4a3eab-e752-4117-93d3-3e5515b6f3f8,size=114];
> will route to failure
> 2024-01-31 12:17:15
> org.apache.nifi.record.path.exception.RecordPathException:
> java.lang.NullPointerException
> 2024-01-31 12:17:15 at
> org.apache.nifi.record.path.RecordPath.compile(RecordPath.java:105)
> 2024-01-31 12:17:15 at
> com.github.benmanes.caffeine.cache.LocalLoadingCache.lambda$newMappingFunction$2(LocalLoadingCache.java:145)
> 2024-01-31 12:17:15 at
> com.github.benmanes.caffeine.cache.BoundedLocalCache.lambda$doComputeIfAbsent$14(BoundedLocalCache.java:2406)
> 2024-01-31 12:17:15 at
> java.base/java.util.concurrent.ConcurrentHashMap.compute(Unknown Source)
> 2024-01-31 12:17:15 at
> com.github.benmanes.caffeine.cache.BoundedLocalCache.doComputeIfAbsent(BoundedLocalCache.java:2404)
> 2024-01-31 12:17:15 at
> com.github.benmanes.caffeine.cache.BoundedLocalCache.computeIfAbsent(BoundedLocalCache.java:2387)
> 2024-01-31 12:17:15 at
> com.github.benmanes.caffeine.cache.LocalCache.computeIfAbsent(LocalCache.java:108)
> 2024-01-31 12:17:15 at
> com.github.benmanes.caffeine.cache.LocalLoadingCache.get(LocalLoadingCache.java:56)
> 2024-01-31 12:17:15 at
> org.apache.nifi.record.path.util.RecordPathCache.getCompiled(RecordPathCache.java:34)
> 2024-01-31 12:17:15 at
> org.apache.nifi.processors.standard.UpdateRecord.process(UpdateRecord.java:166)
> 2024-01-31 12:17:15 at
> org.apache.nifi.processors.standard.AbstractRecordProcessor$1.process(AbstractRecordProcessor.java:147)
> 2024-01-31 12:17:15 at
> org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:3432)
> 2024-01-31 12:17:15 at
> org.apache.nifi.processors.standard.AbstractRecordProcessor.onTrigger(AbstractRecordProcessor.java:122)
> 2024-01-31 12:17:15 at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> 2024-01-31 12:17:15 at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1361)
> 2024-01-31 12:17:15 at
> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:247)
> 2024-01-31 12:17:15 at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:102)
> 2024-01-31 12:17:15 at
> org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
> 2024-01-31 12:17:15 at
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
> 2024-01-31 12:17:15 at
>
[jira] [Updated] (NIFI-12896) Add Endpoint Override Property to PutSNS
[ https://issues.apache.org/jira/browse/NIFI-12896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-12896: -- Fix Version/s: 2.0.0-M4 > Add Endpoint Override Property to PutSNS > > > Key: NIFI-12896 > URL: https://issues.apache.org/jira/browse/NIFI-12896 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Sue Kim >Assignee: Sue Kim >Priority: Minor > Fix For: 2.0.0-M4, 1.27.0 > > Time Spent: 3h 10m > Remaining Estimate: 0h > > Request for PutSNS processor to have an endpoint override property similar to > Get/PutSQS processors. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12669) EvaluateXQuery processor incorrectly encodes result attributes
[
https://issues.apache.org/jira/browse/NIFI-12669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-12669:
--
Fix Version/s: 2.0.0-M4
> EvaluateXQuery processor incorrectly encodes result attributes
> --
>
> Key: NIFI-12669
> URL: https://issues.apache.org/jira/browse/NIFI-12669
> Project: Apache NiFi
> Issue Type: Bug
> Components: Configuration, Extensions
> Environment: JVM with non-UTF-8 default encoding (e.g. default
> Windows installation)
>Reporter: René Zeidler
>Assignee: Jim Steinebrey
>Priority: Major
> Labels: encoding, utf8, windows, xml
> Fix For: 2.0.0-M4, 1.27.0
>
> Attachments: EvaluateXQuery_Encoding_Bug.json,
> image-2024-01-25-10-24-17-005.png, image-2024-01-25-10-31-35-200.png
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> h2. Environment
> This issue affects environments where the JVM default encoding is not
> {{{}UTF-8{}}}. Standard Java installations on Windows are affected, as they
> usually use the default encoding {{{}windows-1252{}}}. To reproduce the issue
> on Linux, change the default encoding to {{windows-1252}} by adding the
> following line to your {{{}bootstrap.conf{}}}:
> {quote}{{java.arg.21=-Dfile.encoding=windows-1252}}
> {quote}
> h2. Summary
> The EvaluateXQuery incorrectly encodes result values when storing them in
> attributes. This causes non-ASCII characters to be garbled.
> Example:
> !image-2024-01-25-10-24-17-005.png!
> h2. Steps to reproduce
> # Make sure NiFi runs with a non-UTF-8 default encoding, see "Environment"
> # Create a GenerateFlowFile processor with the following content:
> {quote}{{}}
> {{}}
> {{ This text contains non-ASCII characters: ÄÖÜäöüßéèóò}}
> {{}}
> {quote}
> # Connect the processor to an EvaluateXQuery processor.
> Set the {{Destination}} to {{{}flowfile-attribute{}}}.
> Create a custom property {{myData}} with value {{{}string(/myRoot/myData){}}}.
> # Connect the outputs of the EvaluateXQuery processor to funnels to be able
> to observe the result in the queue.
> # Start the EvaluateXQuery processor and run the GenerateFlowFile processor
> once.
> The flow should look similar to this:
> !image-2024-01-25-10-31-35-200.png!
> I also attached a JSON export of the example flow.
> # Observe the attributes of the resulting FlowFile in the queue.
> h3. Expected Result
> The FlowFile should contain an attribute {{myData}} with the value {{{}"This
> text contains non-ASCII characters: ÄÖÜäöüßéèóò"{}}}.
> h3. Actual Result
> The attribute has the value "This text contains non-ASCII characters:
> ÄÖÜäöüßéèóò".
> h2. Root Cause Analysis
> EvaluateXQuery uses the method
> [{{formatItem}}|https://github.com/apache/nifi/blob/2e3f83eb54cbc040b5a1da5bce9a74a558f08ea4/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EvaluateXQuery.java#L368-L372]
> to write the query result to an attribute. This method calls
> {{{}ByteArrayOutputStream{}}}'s
> [toString|https://docs.oracle.com/en/java/javase/21/docs/api/java.base/java/io/ByteArrayOutputStream.html#toString()]
> method without an encoding argument, which then defaults to the default
> charset of the environment. Bytes are always written to this output stream
> using UTF-8
> ([.getBytes(StandardCharsets.UTF8)|https://github.com/apache/nifi/blob/2e3f83eb54cbc040b5a1da5bce9a74a558f08ea4/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EvaluateXQuery.java#L397]).
> When the default charset is not UTF-8, this results in UTF-8 bytes to be
> interpreted in a different encoding when converting to a string, resulting in
> garbled text (see above).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-13156) Replace use of com.fasterxml.jackson.core.JsonParser deprecated getCurrentName() with currentName() in JsonParser
[ https://issues.apache.org/jira/browse/NIFI-13156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13156: -- Fix Version/s: 2.0.0-M3 > Replace use of com.fasterxml.jackson.core.JsonParser deprecated > getCurrentName() with currentName() in JsonParser > - > > Key: NIFI-13156 > URL: https://issues.apache.org/jira/browse/NIFI-13156 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Daniel Stieglitz >Assignee: Daniel Stieglitz >Priority: Minor > Fix For: 2.0.0-M3, 1.27.0 > > Time Spent: 20m > Remaining Estimate: 0h > > The getCurrentName() of com.fasterxml.jackson.core.JsonParser has been > deprecated and the suggested replacement is to use method currentName(). > This should be replaced in the following places > # > nifi-extension-bundles/nifi-extension-utils/nifi-record-utils/nifi-json-record-utils/src/main/j > ava/org/apache/nifi/json/AbstractJsonRowRecordReader.java > # > nifi-extension-bundles/nifi-hubspot-bundle/nifi-hubspot-processors/src/main/java/org/apache/nifi/processors/hubspot/GetHubSpot.java > # > nifi-extension-bundles/nifi-salesforce-bundle/nifi-salesforce-processors/src/main/java/org/apac > he/nifi/processors/salesforce/QuerySalesforceObject.java > # > nifi-extension-bundles/nifi-shopify-bundle/nifi-shopify-processors/src/main/java/org/apache/nifi/processors/shopify/GetShopify.java > # > nifi-extension-bundles/nifi-zendesk-bundle/nifi-zendesk-processors/src/main/java/org/apache/nifi/processors/zendesk/GetZendesk.java -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13340) Flowfiles stopped to be ingested before a processor group
[
https://issues.apache.org/jira/browse/NIFI-13340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-13340:
--
Fix Version/s: 2.0.0-M4
> Flowfiles stopped to be ingested before a processor group
> -
>
> Key: NIFI-13340
> URL: https://issues.apache.org/jira/browse/NIFI-13340
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.25.0
> Environment: Host OS :ubuntu 20.04 in wsl
> CPU: Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
> RAM: 128GB
> ZFS ARC cache was configured to be maximum 4 GB
>Reporter: Yuanhao Zhu
>Assignee: Mark Payne
>Priority: Major
> Labels: data-consistency, statestore
> Fix For: 2.0.0-M4, 1.27.0
>
> Attachments: image-2024-06-03-14-42-37-280.png,
> image-2024-06-03-14-44-15-590.png
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> *Background:* We run nifi as a standalone instance in a docker container in
> on all our stages, the statemanagement provider used by our instance is the
> local WAL backed provider.
> {*}Description{*}: We observed that the flowfiles in front of one of our
> processor groups stopped to be ingested once in a while and it happens on all
> our stages without noticeable pattern. The flowfile concurrency policy of the
> processor group that stopped ingesting data is set to SINGLE_BATCH_PER_NODE
> and the outbound policy is set to BATCH_OUTPUT. The ingestion should continue
> since the processor group had already send out every flowfile in it, but it
> stopped.
>
> There is only one brutal solution from our side(We need to delete the
> processor group and restore it from nifi registry again) and the occurrence
> of this issue had impacted our data ingestion.
> !image-2024-06-03-14-42-37-280.png!
> !image-2024-06-03-14-44-15-590.png!
> As you can see in the screenshot that the processor group 'Delete before
> Insert' has no more flowfile to output but still it does not ingest the data
> queued in the input port
>
> {{In the log file I found the following:}}
>
> {code:java}
> 2024-06-03 11:34:01,772 TRACE [Timer-Driven Process Thread-15]
> o.apache.nifi.groups.StandardDataValve Will not allow data to flow into
> StandardProcessGroup[identifier=5eb0ad69-e8ed-3ba0-52da-af94fb9836cd,name=Delete
> before Insert] because Outbound Policy is Batch Output and valve is already
> open to allow data to flow out of group{code}
>
> {{ }}
> {{Also in the diagnostics, I found the following for the 'Delete before
> Insert' processor group:}}
> {{ }}
> {code:java}
> Process Group e6510a87-aa78-3268-1b11-3c310f0ad144, Name = Search and Delete
> existing reports(This is the parent processor group of the Delete before
> Insert)
> Currently Have Data Flowing In: []
> Currently Have Data Flowing Out:
> [StandardProcessGroup[identifier=5eb0ad69-e8ed-3ba0-52da-af94fb9836cd,name=Delete
> before Insert]]
> Reason for Not allowing data to flow in:
> Data Valve is already allowing data to flow out of group:
>
> StandardProcessGroup[identifier=5eb0ad69-e8ed-3ba0-52da-af94fb9836cd,name=Delete
> before Insert]
> Reason for Not allowing data to flow out:
> Output is Allowed:
>
> StandardProcessGroup[identifier=5eb0ad69-e8ed-3ba0-52da-af94fb9836cd,name=Delete
> before Insert] {code}
> {{ }}
> {{Which is clearly {*}not correct{*}. since there are currently not data
> flowing out from the 'Delete before Insert' processor group.}}
> {{ }}
> {{We dig through the source code of StandardDataValve.java and found that
> that data valve's states are stored every time the data valve is opened and
> close so the potential reason causing this issue is that the processor group
> id was put into the statemap when data flowed in but somehow the removal of
> the entry was not successful. We are aware that if the statemap is not stored
> before the nifi restarts, it could lead to such circumstances, but in the
> recent occurrences of this issue, there were not nifi restart recorded or
> observed at the time when all those flowfiles started to queue in front of
> the processor group}}
> {{ }}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12750) ExecuteStreamCommand incorrectly decodes error stream
[
https://issues.apache.org/jira/browse/NIFI-12750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-12750:
--
Fix Version/s: 2.0.0-M4
> ExecuteStreamCommand incorrectly decodes error stream
> -
>
> Key: NIFI-12750
> URL: https://issues.apache.org/jira/browse/NIFI-12750
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.25.0, 2.0.0-M2
> Environment: any
>Reporter: René Zeidler
>Assignee: Jim Steinebrey
>Priority: Major
> Labels: ExecuteStreamCommand, encoding, iso-8859-1, utf-8
> Fix For: 2.0.0-M4, 1.27.0
>
> Attachments: ExecuteStreamCommand_Encoding_Bug.json, encodingTest.sh,
> image-2024-02-07-15-14-08-518.png, image-2024-02-07-15-14-54-841.png,
> image-2024-02-07-15-20-11-684.png
>
>
> h1. Summary
> The ExecuteStreamCommand processor stores everything the invoked command
> writes to the error stream (stderr) into the FlowFile attribute
> {{{}execution.error{}}}.
> When converting the bytes from the stream to a String, it interprets each
> individual byte as a Unicode codepoint. When reading only single bytes this
> effectively results in ISO-8859-1 (Latin-1).
> Instead, it should use the system default encoding (like it already does for
> writing stdout if Output Destination Attribute is set) or use a configurable
> encoding (for both stdout and stderr).
> h1. Details
> When reading/writing FlowFiles, NiFi always uses raw bytes, so encoding
> issues are the responsibility of the flow designer, and NiFi has the
> ConvertCharacterSet processor to deal with those issues.
> When writing to attributes, the API uses Java String objects, which are
> encoding agnostic (they represent Unicode codepoints, not bytes). Therefore,
> processors receiving bytes have to interpret them using an encoding.
> The ExecuteStreamCommand processor writes the output of the command (stdout)
> to the Output Destination Attribute (if set). To do that, it convertes bytes
> into a String using the system default encoding* by calling {{new String}}
> without an encoding argument:
> [https://github.com/apache/nifi/blob/72f6d8a6800c643d5f283ae9bff6d7de25b503e9/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ExecuteStreamCommand.java#L499]
> When converting stderr to a String to write into the {{execution.error}}
> attribute, it uses this weird algorithm:
> [https://github.com/apache/nifi/blob/72f6d8a6800c643d5f283ae9bff6d7de25b503e9/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ExecuteStreamCommand.java#L507-L517]
> It reads individual bytes from the error stream (as {{{}int{}}}s) and casts
> them to {{{}char{}}}s. What Java does in this case is interpret the integer
> as a Unicode code point. For single bytes, this matches the ISO-8859-1
> encoding. Instead, it should use the same decoding method as for stdout.
> h1. Reproduction steps
> These steps are for a Linux environment, but can be adapted with a different
> executable for Windows.
> # Create the file /opt/nifi/data/encodingTest.sh (attached) with the
> following contents and make it executable:
> {quote}{{#/bin/bash}}
> {{echo "|out static: ÄÖÜäöüß"}}
> {{{}echo "|error static: ÄÖÜäöüß" >&2{}}}{{{}echo "|out arg: $1"{}}}
> {{{}echo "|error arg: $1" >&2{}}}{{{}echo "|out arg hexdump:"{}}}
> {{printf '%s' "$1" | od -A x -t x1z -v}}
> {{echo "|error arg hexdump:" >&2}}
> {{printf '%s' "$1" | od -A x -t x1z -v >&2}}{quote}The script writes
> identical data to both stdout and stderr. It contains non-ASCII characters to
> make the encoding issues visible.
> # Import the attached flow or create it manually:
> !image-2024-02-07-15-14-08-518.png|width=324,height=373!!image-2024-02-07-15-14-54-841.png|width=326,height=120!
> # Run the GenerateFlowFile processor once and observe the attributes of the
> FlowFile in the final queue:
> !image-2024-02-07-15-20-11-684.png|width=523,height=195!
> The output attribute (stdout) is correctly decoded. The execution.error
> attribute (stderr) contains garbled text (UTF-8 bytes interpreted as
> ISO-8859-1 and reencoded in UTF-8).
> h1. *On the system default encoding
> The system default encoding is a property of the JVM. It is UTF-8 on Linux,
> but Windows-1252 (or a different copepage depending on locale) in Windows
> environments. It can be overriden using the {{file.encoding}} JVM arg on
> startup.
> Relying on the system default encoding is dangerous and can lead to subtle
> bugs, like the ones I previously reported (NIFI-12669 and NIFI-12670).
> In this case, it might make sense to use the system default encoding, as it
> concerns data passed between NiFi and
[jira] [Updated] (NIFI-11658) Streamline using single parameter context for nested PGs
[ https://issues.apache.org/jira/browse/NIFI-11658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-11658: -- Fix Version/s: 2.0.0-M1 > Streamline using single parameter context for nested PGs > > > Key: NIFI-11658 > URL: https://issues.apache.org/jira/browse/NIFI-11658 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Timea Barna >Assignee: Timea Barna >Priority: Major > Fix For: 2.0.0-M1, 1.27.0 > > Time Spent: 2h 20m > Remaining Estimate: 0h > > 1. when adding a new process group to the canvas, in the create PG view where > one should give the name of the process group, add a drop down menu to select > the parameter context to associate with the process group. It would default > to the parameter context associated to the parent process group (if there is > one specified and if the user has the permissions) > 2. in the configuration view of a process group, add a dedicated tab for > parameter context to have dedicated handling. In addition to the dropdown > menu for selecting a parameter context, add a checkbox to recursively apply > the change. This checkbox would only be used when clicking "Apply", it would > not be persisted. If checked, it means that the parameter context will be > applied to the process group as well to all the embedded process groups > (recursively) if and only if the user has the proper permissions on all > process groups. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-10666) PrometheusReportingTask does not use UTF-8 encoding on /metrics/ endpoint
[
https://issues.apache.org/jira/browse/NIFI-10666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-10666:
--
Fix Version/s: 2.0.0-M4
> PrometheusReportingTask does not use UTF-8 encoding on /metrics/ endpoint
> -
>
> Key: NIFI-10666
> URL: https://issues.apache.org/jira/browse/NIFI-10666
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.17.0, 1.16.3, 1.18.0, 1.23.2, 1.25.0, 2.0.0-M2
> Environment: JVM with non-UTF-8 default encoding (e.g. default
> Windows installation)
>Reporter: René Zeidler
>Assignee: Jim Steinebrey
>Priority: Minor
> Labels: encoding, prometheus, utf-8
> Fix For: 2.0.0-M4, 1.27.0
>
> Attachments: missing-header.png
>
>
> We have created a default PrometheusReportingTask for our NiFi instance and
> tried to consume the metrics with Prometheus. However, Prometheus threw the
> following error:
> {code:java}
> ts=2022-10-19T12:25:18.110Z caller=scrape.go:1332 level=debug
> component="scrape manager" scrape_pool=nifi-cluster
> target=http://***nifi***:9092/metrics msg="Append failed" err="invalid UTF-8
> label value" {code}
> Upon further inspection, we noticed that the /metrics/ endpoint exposed by
> the reporting task does not use UTF-8 encoding, which is required by
> Prometheus (as documented here: [Exposition formats |
> Prometheus|https://prometheus.io/docs/instrumenting/exposition_formats/]).
> Our flow uses non-ASCII characters (in our case German umlauts like "ü"). As
> a workaround, removing those characters fixes the Prometheus error, but this
> is not practical for a large flow in German language.
> Opening the /metrics/ endpoint in a browser confirms that the encoding used
> is not UTF-8:
> {code:java}
> > document.characterSet
> 'windows-1252' {code}
>
> The responsible code might be here:
> [https://github.com/apache/nifi/blob/2be5c26f287469f4f19f0fa759d6c1b56dc0e348/nifi-nar-bundles/nifi-prometheus-bundle/nifi-prometheus-reporting-task/src/main/java/org/apache/nifi/reporting/prometheus/PrometheusServer.java#L67]
> The PrometheusServer used by the reporting task uses an OutputStreamWriter
> with the default encoding, instead of explicitly using UTF-8. The
> Content-Type header set in that function also does not get passed along (see
> screenshot).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-13008) CLI command to upgrade all instances of a versioned flow
[ https://issues.apache.org/jira/browse/NIFI-13008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13008: -- Fix Version/s: 2.0.0-M3 > CLI command to upgrade all instances of a versioned flow > > > Key: NIFI-13008 > URL: https://issues.apache.org/jira/browse/NIFI-13008 > Project: Apache NiFi > Issue Type: Improvement > Components: Tools and Build >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Major > Labels: backport-needed > Fix For: 2.0.0-M3, 1.27.0 > > Time Spent: 1h 20m > Remaining Estimate: 0h > > Add a CLI command allowing someone to upgrade all the instances of a given > versioned flow. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13400) SmbDfsIT failures
[ https://issues.apache.org/jira/browse/NIFI-13400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13400: -- Fix Version/s: 2.0.0-M4 > SmbDfsIT failures > -- > > Key: NIFI-13400 > URL: https://issues.apache.org/jira/browse/NIFI-13400 > Project: Apache NiFi > Issue Type: Task > Environment: Apache Maven 3.9.6 > (bc0240f3c744dd6b6ec2920b3cd08dcc295161ae) > Maven home: /development/tools/apache-maven-3.9.6 > Java version: 21.0.3, vendor: Azul Systems, Inc., runtime: > /usr/lib/jvm/zulu21.34.19-ca-jdk21.0.3-linux_x64 > Default locale: en_US, platform encoding: UTF-8 > OS name: "linux", version: "6.8.9-200.fc39.x86_64", arch: "amd64", family: > "unix" >Reporter: Joe Witt >Assignee: Peter Turcsanyi >Priority: Major > Fix For: 2.0.0-M4, 1.27.0 > > Attachments: TEST-org.apache.nifi.processors.smb.SmbDfsIT.xml > > > [ERROR] Failures: > [ERROR] SmbDfsIT.testFetchSmb:94 expected: <1> but was: <0> > [ERROR] SmbDfsIT.testGetSmbFile:185 expected: <1> but was: <0> > [ERROR] SmbDfsIT.testListSmbWithDfsLink:121->testListSmb:143 expected: <1> > but was: <0> > [ERROR] SmbDfsIT.testPutSmbFile:165 expected: <1> but was: <0> > NIFI-12837 introduced this IT. It does not appear to work successfully on > linux hosts. Unclear exactly. Does seem to work on osx. Perhaps the test > is setup to assume arm or something. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13298) Remove unused instantiated java.util.HashSet from RouteAttribute
[ https://issues.apache.org/jira/browse/NIFI-13298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13298: -- Fix Version/s: 2.0.0-M4 > Remove unused instantiated java.util.HashSet from RouteAttribute > > > Key: NIFI-13298 > URL: https://issues.apache.org/jira/browse/NIFI-13298 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Daniel Stieglitz >Assignee: Daniel Stieglitz >Priority: Minor > Fix For: 2.0.0-M4, 1.27.0 > > Time Spent: 20m > Remaining Estimate: 0h > > In RouteOnAttribute on line 352 a set is instantiated > final Set clones = new HashSet<>(); > and on line 358 its populated > clones.add(cloneFlowFile); > Yet after that nothing is done with it (per Intellij it is updated but > never queried). > This should removed as it is not serving any purpose. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13397) PutDatabaseRecord check for cause of ProcessException being a SQLTransientException
[ https://issues.apache.org/jira/browse/NIFI-13397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13397: -- Fix Version/s: 2.0.0-M4 > PutDatabaseRecord check for cause of ProcessException being a > SQLTransientException > --- > > Key: NIFI-13397 > URL: https://issues.apache.org/jira/browse/NIFI-13397 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Jim Steinebrey >Assignee: Jim Steinebrey >Priority: Minor > Fix For: 2.0.0-M4, 1.27.0 > > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13359) Tune ExecuteSQL/Record to create fewer transient flow files
[ https://issues.apache.org/jira/browse/NIFI-13359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13359: -- Fix Version/s: 2.0.0-M4 > Tune ExecuteSQL/Record to create fewer transient flow files > --- > > Key: NIFI-13359 > URL: https://issues.apache.org/jira/browse/NIFI-13359 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.26.0, 2.0.0-M3 >Reporter: Jim Steinebrey >Assignee: Jim Steinebrey >Priority: Major > Fix For: 2.0.0-M4, 1.27.0 > > Time Spent: 20m > Remaining Estimate: 0h > > AbstractExecuteSQL (which is extended by ExecuteSQL and ExceuteSQLRecord) > creates several unneeded temp flow files by using putAttribute instead of > putAllAttributes. Tune the code to create fewer intermediate flow files. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13356) Protobuf Reader fails on parsing certain repeated fields
[ https://issues.apache.org/jira/browse/NIFI-13356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13356: -- Fix Version/s: 2.0.0-M4 > Protobuf Reader fails on parsing certain repeated fields > > > Key: NIFI-13356 > URL: https://issues.apache.org/jira/browse/NIFI-13356 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.26.0, 2.0.0-M3 >Reporter: Mark Bathori >Assignee: Mark Bathori >Priority: Major > Fix For: 2.0.0-M4, 1.27.0 > > Time Spent: 1h 20m > Remaining Estimate: 0h > > The Protobuf Reader fails on parsing repeated fields where the field type is > not LengthDelimited. The UnknownFieldSet parser is unable to process these > repeated fields, instead it converts them to byte array so they need to be > handled separately. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13072) MonitorActivity processor generates a false positive on a node restart
[ https://issues.apache.org/jira/browse/NIFI-13072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13072: -- Fix Version/s: 2.0.0-M3 > MonitorActivity processor generates a false positive on a node restart > -- > > Key: NIFI-13072 > URL: https://issues.apache.org/jira/browse/NIFI-13072 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 2.0.0-M1, 1.18.0, 1.19.0, 1.20.0, 1.19.1, 1.21.0, > 1.22.0, 1.23.0, 1.24.0, 1.23.1, 1.23.2, 1.25.0, 2.0.0-M2 >Reporter: Rajmund Takacs >Assignee: Rajmund Takacs >Priority: Major > Fix For: 2.0.0-M3, 1.27.0 > > Attachments: MonitorActivity_Flow_Example_NiFi_1.18_OSS.json > > Time Spent: 1h > Remaining Estimate: 0h > > If the monitoring scope is configured to "cluster", then there are multiple > restart cases when markers are incorrectly generated: > h4. CASE 1 > * The flow is already inactive, and there is no traffic. The processor is > still running and scheduled on all nodes. > * We restart any of the NiFi nodes in the cluster. > * *All nodes, except the restarted one, generates activation marker.* (Reason > for this is, that the common timestamp is updated by the restarted node > during initialization.) > h4. CASE 2: > * The flow is already inactive, and there is no traffic. The processor is > still running and scheduled on all nodes. Reporting is done by all nodes. > * We temporarily stop any of the nodes. > * We start traffic on any other node, so that an activation marker is > generated. > * We start the previously stopped node again. > * *This node does not send activation marker.* (Reason for this is that the > initial state is always active.) > Expected behavior for stop/start cases would be: > * Single node restart alone should not affect the flow activity state. > * If a marker has already been sent before stopping the processor, it should > not be repeated on next start, unless asked for, or there is a change in > activity state. > * If the flow was active during a weekend shutdown, it should not begin with > an inactivity marker right after startup on monday. Although it should send > inactivity marker after the threshold has expired since startup. > * If the flow was inactive during a weekend shutdown, it should send > activation marker after started up again with traffic. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13208) Increase member visibility in AbstractHadoopConnectionPool
[ https://issues.apache.org/jira/browse/NIFI-13208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13208: -- Fix Version/s: 2.0.0-M3 > Increase member visibility in AbstractHadoopConnectionPool > -- > > Key: NIFI-13208 > URL: https://issues.apache.org/jira/browse/NIFI-13208 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Lehel Boér >Assignee: Lehel Boér >Priority: Minor > Fix For: 2.0.0-M3, 1.27.0 > > Time Spent: 20m > Remaining Estimate: 0h > > HADOOP_CONFIGURATION_RESOURCES access modifier should be changed to public in > order to used in any inherited class. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13315) ListAzureBlobStorage_v12 fails when Record Writer is used
[
https://issues.apache.org/jira/browse/NIFI-13315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-13315:
--
Fix Version/s: 2.0.0-M4
> ListAzureBlobStorage_v12 fails when Record Writer is used
> -
>
> Key: NIFI-13315
> URL: https://issues.apache.org/jira/browse/NIFI-13315
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Peter Turcsanyi
>Assignee: Peter Turcsanyi
>Priority: Major
> Fix For: 2.0.0-M4, 1.27.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> BlobInfo declares Container Name as type of boolean in the record schema
> which fails when the records are written.
> {code}
> 2024-05-30 12:23:54,794 ERROR [Timer-Driven Process Thread-9]
> o.a.n.p.a.s.ListAzureBlobStorage_v12
> ListAzureBlobStorage_v12[id=dee130d2-d89c-3894-40ba-bc5dd5e52d2b] Failed to
> write MapRecord[{container=container1,
> blobName=fea8dfbd-a53c-40f7-ad11-faaefdc7a99c,
> filename=fea8dfbd-a53c-40f7-ad11-faaefdc7a99c, secondaryUri=null,
> primaryUri=https://turcsanyiblob.blob.core.windows.net/container1/fea8dfbd-a53c-40f7-ad11-faaefdc7a99c,
> length=5, language=null, etag=0x8DC3A36EF89F172, lastModified=170932868,
> blobType=BlockBlob, contentType=application/octet-stream}] with reader schema
> ["blobName" : "STRING", "blobType" : "STRING", "filename" : "STRING",
> "container" : "BOOLEAN", "length" : "LONG", "lastModified" :
> "TIMESTAMP:-MM-dd HH:mm:ss", "etag" : "STRING", "language" : "STRING",
> "contentType" : "STRING", "primaryUri" : "STRING", "secondaryUri" : "STRING"]
> and writer schema ["blobName" : "STRING", "blobType" : "STRING", "filename" :
> "STRING", "container" : "BOOLEAN", "length" : "LONG", "lastModified" :
> "TIMESTAMP:-MM-dd HH:mm:ss", "etag" : "STRING", "language" : "STRING",
> "contentType" : "STRING", "primaryUri" : "STRING", "secondaryUri" : "STRING"]
> as a JSON Object due to
> org.apache.nifi.serialization.record.util.IllegalTypeConversionException:
> Cannot convert value [container1] of type class java.lang.String to Boolean
> for field container
> org.apache.nifi.serialization.record.util.IllegalTypeConversionException:
> Cannot convert value [container1] of type class java.lang.String to Boolean
> for field container
> at
> org.apache.nifi.serialization.record.util.DataTypeUtils.toBoolean(DataTypeUtils.java:1224)
> at
> org.apache.nifi.serialization.record.util.DataTypeUtils.convertType(DataTypeUtils.java:199)
> at
> org.apache.nifi.serialization.record.util.DataTypeUtils.convertType(DataTypeUtils.java:178)
> at
> org.apache.nifi.json.WriteJsonResult.writeValue(WriteJsonResult.java:349)
> at
> org.apache.nifi.json.WriteJsonResult.writeRecord(WriteJsonResult.java:233)
> at
> org.apache.nifi.json.WriteJsonResult.writeRecord(WriteJsonResult.java:157)
> at
> org.apache.nifi.serialization.AbstractRecordSetWriter.write(AbstractRecordSetWriter.java:59)
> at
> java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
> at java.base/java.lang.reflect.Method.invoke(Method.java:580)
> at
> org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:254)
> at
> org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler$ProxiedReturnObjectInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:240)
> at jdk.proxy55/jdk.proxy55.$Proxy220.write(Unknown Source)
> at
> org.apache.nifi.processor.util.list.AbstractListProcessor.createRecordsForEntities(AbstractListProcessor.java:940)
> at
> org.apache.nifi.processor.util.list.AbstractListProcessor.listByNoTracking(AbstractListProcessor.java:584)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-13429) EncryptContentPGP Packet Detection Invalid for JPEG Files
[
https://issues.apache.org/jira/browse/NIFI-13429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-13429:
--
Fix Version/s: 2.0.0-M4
> EncryptContentPGP Packet Detection Invalid for JPEG Files
> -
>
> Key: NIFI-13429
> URL: https://issues.apache.org/jira/browse/NIFI-13429
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.15.0, 1.26.0, 2.0.0-M3
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Major
> Fix For: 2.0.0-M4, 1.27.0
>
>
> The {{EncryptContentPGP}} Processor performs input content evaluation to
> avoid additional wrapping around signed OpenPGP payloads. This content
> evaluation inspects the initial bytes for an OpenPGP Packet Tag, but does not
> evaluate the Packet Type. As a result, some types of input files, such as
> JPEG, can result in incorrect evaluation, producing invalid output from
> {{EncryptContentPGP}}. When attempting to decrypt malformed files in
> {{DecryptContentPGP}}, the following error occurs:
> {noformat}
> DecryptContentPGP[id=3687fd8a-0190-1000-345b-fcaaba5a3e0c] Decryption Failed
> StandardFlowFileRecord[uuid=2c60ab6c-16cd-49c5-b2c8-f4e3d3a8f920,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1718901851045-2, container=default,
> section=2], offset=0, length=82192],offset=0,name=unsplash.jpg,size=82192]
> org.bouncycastle.openpgp.PGPRuntimeOperationException: Iterator failed to get
> next object: invalid header encountered
> at org.bouncycastle.openpgp.PGPObjectFactory$1.getObject(Unknown Source)
> at org.bouncycastle.openpgp.PGPObjectFactory$1.hasNext(Unknown Source)
> at
> org.apache.nifi.processors.pgp.DecryptContentPGP$DecryptStreamCallback.getLiteralData(DecryptContentPGP.java:357)
> at
> org.apache.nifi.processors.pgp.DecryptContentPGP$DecryptStreamCallback.getLiteralData(DecryptContentPGP.java:347)
> at
> org.apache.nifi.processors.pgp.DecryptContentPGP$DecryptStreamCallback.process(DecryptContentPGP.java:278)
> at
> org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:3425)
> at
> org.apache.nifi.processors.pgp.DecryptContentPGP.onTrigger(DecryptContentPGP.java:181)
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1274)
> at
> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:244)
> at
> org.apache.nifi.controller.scheduling.AbstractTimeBasedSchedulingAgent.lambda$doScheduleOnce$0(AbstractTimeBasedSchedulingAgent.java:59)
> at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
> at
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
> at
> java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
> at java.base/java.lang.Thread.run(Thread.java:1583)
> Caused by: java.io.IOException: invalid header encountered
> at org.bouncycastle.bcpg.BCPGInputStream.readPacket(Unknown Source)
> at org.bouncycastle.openpgp.PGPSignature.(Unknown Source)
> at org.bouncycastle.openpgp.PGPObjectFactory.nextObject(Unknown Source)
> ... 18 common frames omitted
> {noformat}
> The input packet evaluation should be improved to avoid incorrect
> identification of non-OpenPGP files.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-13429) EncryptContentPGP Packet Detection Invalid for JPEG Files
[
https://issues.apache.org/jira/browse/NIFI-13429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-13429:
--
Fix Version/s: 1.27.0
(was: 1.26.1)
(was: 2.0.0-M4)
> EncryptContentPGP Packet Detection Invalid for JPEG Files
> -
>
> Key: NIFI-13429
> URL: https://issues.apache.org/jira/browse/NIFI-13429
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.15.0, 1.26.0, 2.0.0-M3
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Major
> Fix For: 1.27.0
>
>
> The {{EncryptContentPGP}} Processor performs input content evaluation to
> avoid additional wrapping around signed OpenPGP payloads. This content
> evaluation inspects the initial bytes for an OpenPGP Packet Tag, but does not
> evaluate the Packet Type. As a result, some types of input files, such as
> JPEG, can result in incorrect evaluation, producing invalid output from
> {{EncryptContentPGP}}. When attempting to decrypt malformed files in
> {{DecryptContentPGP}}, the following error occurs:
> {noformat}
> DecryptContentPGP[id=3687fd8a-0190-1000-345b-fcaaba5a3e0c] Decryption Failed
> StandardFlowFileRecord[uuid=2c60ab6c-16cd-49c5-b2c8-f4e3d3a8f920,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1718901851045-2, container=default,
> section=2], offset=0, length=82192],offset=0,name=unsplash.jpg,size=82192]
> org.bouncycastle.openpgp.PGPRuntimeOperationException: Iterator failed to get
> next object: invalid header encountered
> at org.bouncycastle.openpgp.PGPObjectFactory$1.getObject(Unknown Source)
> at org.bouncycastle.openpgp.PGPObjectFactory$1.hasNext(Unknown Source)
> at
> org.apache.nifi.processors.pgp.DecryptContentPGP$DecryptStreamCallback.getLiteralData(DecryptContentPGP.java:357)
> at
> org.apache.nifi.processors.pgp.DecryptContentPGP$DecryptStreamCallback.getLiteralData(DecryptContentPGP.java:347)
> at
> org.apache.nifi.processors.pgp.DecryptContentPGP$DecryptStreamCallback.process(DecryptContentPGP.java:278)
> at
> org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:3425)
> at
> org.apache.nifi.processors.pgp.DecryptContentPGP.onTrigger(DecryptContentPGP.java:181)
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1274)
> at
> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:244)
> at
> org.apache.nifi.controller.scheduling.AbstractTimeBasedSchedulingAgent.lambda$doScheduleOnce$0(AbstractTimeBasedSchedulingAgent.java:59)
> at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
> at
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
> at
> java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
> at java.base/java.lang.Thread.run(Thread.java:1583)
> Caused by: java.io.IOException: invalid header encountered
> at org.bouncycastle.bcpg.BCPGInputStream.readPacket(Unknown Source)
> at org.bouncycastle.openpgp.PGPSignature.(Unknown Source)
> at org.bouncycastle.openpgp.PGPObjectFactory.nextObject(Unknown Source)
> ... 18 common frames omitted
> {noformat}
> The input packet evaluation should be improved to avoid incorrect
> identification of non-OpenPGP files.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-13315) ListAzureBlobStorage_v12 fails when Record Writer is used
[
https://issues.apache.org/jira/browse/NIFI-13315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-13315:
--
Fix Version/s: 1.27.0
(was: 1.26.1)
(was: 2.0.0-M4)
> ListAzureBlobStorage_v12 fails when Record Writer is used
> -
>
> Key: NIFI-13315
> URL: https://issues.apache.org/jira/browse/NIFI-13315
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Peter Turcsanyi
>Assignee: Peter Turcsanyi
>Priority: Major
> Fix For: 1.27.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> BlobInfo declares Container Name as type of boolean in the record schema
> which fails when the records are written.
> {code}
> 2024-05-30 12:23:54,794 ERROR [Timer-Driven Process Thread-9]
> o.a.n.p.a.s.ListAzureBlobStorage_v12
> ListAzureBlobStorage_v12[id=dee130d2-d89c-3894-40ba-bc5dd5e52d2b] Failed to
> write MapRecord[{container=container1,
> blobName=fea8dfbd-a53c-40f7-ad11-faaefdc7a99c,
> filename=fea8dfbd-a53c-40f7-ad11-faaefdc7a99c, secondaryUri=null,
> primaryUri=https://turcsanyiblob.blob.core.windows.net/container1/fea8dfbd-a53c-40f7-ad11-faaefdc7a99c,
> length=5, language=null, etag=0x8DC3A36EF89F172, lastModified=170932868,
> blobType=BlockBlob, contentType=application/octet-stream}] with reader schema
> ["blobName" : "STRING", "blobType" : "STRING", "filename" : "STRING",
> "container" : "BOOLEAN", "length" : "LONG", "lastModified" :
> "TIMESTAMP:-MM-dd HH:mm:ss", "etag" : "STRING", "language" : "STRING",
> "contentType" : "STRING", "primaryUri" : "STRING", "secondaryUri" : "STRING"]
> and writer schema ["blobName" : "STRING", "blobType" : "STRING", "filename" :
> "STRING", "container" : "BOOLEAN", "length" : "LONG", "lastModified" :
> "TIMESTAMP:-MM-dd HH:mm:ss", "etag" : "STRING", "language" : "STRING",
> "contentType" : "STRING", "primaryUri" : "STRING", "secondaryUri" : "STRING"]
> as a JSON Object due to
> org.apache.nifi.serialization.record.util.IllegalTypeConversionException:
> Cannot convert value [container1] of type class java.lang.String to Boolean
> for field container
> org.apache.nifi.serialization.record.util.IllegalTypeConversionException:
> Cannot convert value [container1] of type class java.lang.String to Boolean
> for field container
> at
> org.apache.nifi.serialization.record.util.DataTypeUtils.toBoolean(DataTypeUtils.java:1224)
> at
> org.apache.nifi.serialization.record.util.DataTypeUtils.convertType(DataTypeUtils.java:199)
> at
> org.apache.nifi.serialization.record.util.DataTypeUtils.convertType(DataTypeUtils.java:178)
> at
> org.apache.nifi.json.WriteJsonResult.writeValue(WriteJsonResult.java:349)
> at
> org.apache.nifi.json.WriteJsonResult.writeRecord(WriteJsonResult.java:233)
> at
> org.apache.nifi.json.WriteJsonResult.writeRecord(WriteJsonResult.java:157)
> at
> org.apache.nifi.serialization.AbstractRecordSetWriter.write(AbstractRecordSetWriter.java:59)
> at
> java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
> at java.base/java.lang.reflect.Method.invoke(Method.java:580)
> at
> org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:254)
> at
> org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler$ProxiedReturnObjectInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:240)
> at jdk.proxy55/jdk.proxy55.$Proxy220.write(Unknown Source)
> at
> org.apache.nifi.processor.util.list.AbstractListProcessor.createRecordsForEntities(AbstractListProcessor.java:940)
> at
> org.apache.nifi.processor.util.list.AbstractListProcessor.listByNoTracking(AbstractListProcessor.java:584)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-13374) Fix tooltip for Parameter Context in new Process Group dialog
[ https://issues.apache.org/jira/browse/NIFI-13374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13374: -- Fix Version/s: 1.27.0 (was: 1.26.1) > Fix tooltip for Parameter Context in new Process Group dialog > - > > Key: NIFI-13374 > URL: https://issues.apache.org/jira/browse/NIFI-13374 > Project: Apache NiFi > Issue Type: Bug >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Major > Fix For: 1.27.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13151) Deprecate Couchbase components
[ https://issues.apache.org/jira/browse/NIFI-13151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13151: -- Fix Version/s: 1.27.0 (was: 1.26.1) > Deprecate Couchbase components > -- > > Key: NIFI-13151 > URL: https://issues.apache.org/jira/browse/NIFI-13151 > Project: Apache NiFi > Issue Type: Sub-task >Reporter: Joe Witt >Assignee: Joe Witt >Priority: Major > Fix For: 1.27.0 > > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13208) Increase member visibility in AbstractHadoopConnectionPool
[ https://issues.apache.org/jira/browse/NIFI-13208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13208: -- Fix Version/s: 1.27.0 (was: 2.0.0-M3) (was: 1.26.1) > Increase member visibility in AbstractHadoopConnectionPool > -- > > Key: NIFI-13208 > URL: https://issues.apache.org/jira/browse/NIFI-13208 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Lehel Boér >Assignee: Lehel Boér >Priority: Minor > Fix For: 1.27.0 > > Time Spent: 20m > Remaining Estimate: 0h > > HADOOP_CONFIGURATION_RESOURCES access modifier should be changed to public in > order to used in any inherited class. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13359) Tune ExecuteSQL/Record to create fewer transient flow files
[ https://issues.apache.org/jira/browse/NIFI-13359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13359: -- Fix Version/s: 1.27.0 (was: 1.26.1) (was: 2.0.0-M4) > Tune ExecuteSQL/Record to create fewer transient flow files > --- > > Key: NIFI-13359 > URL: https://issues.apache.org/jira/browse/NIFI-13359 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.26.0, 2.0.0-M3 >Reporter: Jim Steinebrey >Assignee: Jim Steinebrey >Priority: Major > Fix For: 1.27.0 > > Time Spent: 20m > Remaining Estimate: 0h > > AbstractExecuteSQL (which is extended by ExecuteSQL and ExceuteSQLRecord) > creates several unneeded temp flow files by using putAttribute instead of > putAllAttributes. Tune the code to create fewer intermediate flow files. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13397) PutDatabaseRecord check for cause of ProcessException being a SQLTransientException
[ https://issues.apache.org/jira/browse/NIFI-13397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13397: -- Fix Version/s: 1.27.0 (was: 1.26.1) (was: 2.0.0-M4) > PutDatabaseRecord check for cause of ProcessException being a > SQLTransientException > --- > > Key: NIFI-13397 > URL: https://issues.apache.org/jira/browse/NIFI-13397 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Jim Steinebrey >Assignee: Jim Steinebrey >Priority: Minor > Fix For: 1.27.0 > > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13323) For 1.x, Remove the instantiation of Object arrays for arguments in ComponentLog log and org.slf4j.Logger statements
[ https://issues.apache.org/jira/browse/NIFI-13323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13323: -- Fix Version/s: 1.27.0 (was: 1.26.1) > For 1.x, Remove the instantiation of Object arrays for arguments in > ComponentLog log and org.slf4j.Logger statements > > > Key: NIFI-13323 > URL: https://issues.apache.org/jira/browse/NIFI-13323 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Daniel Stieglitz >Assignee: Daniel Stieglitz >Priority: Major > Fix For: 1.27.0 > > Time Spent: 1h 50m > Remaining Estimate: 0h > > This ticket is a follow up of NIFI-13265 which was only able to be applied to > the 2.x branch. This ticket aims to do the same type of changes in the 1.x > branch. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13411) Dependencies Updates for 1.26.1 including Spring 5.3.37
[ https://issues.apache.org/jira/browse/NIFI-13411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13411: -- Fix Version/s: 1.27.0 (was: 1.26.1) > Dependencies Updates for 1.26.1 including Spring 5.3.37 > --- > > Key: NIFI-13411 > URL: https://issues.apache.org/jira/browse/NIFI-13411 > Project: Apache NiFi > Issue Type: Task > Components: Core Framework >Affects Versions: 1.26.0 >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Major > Fix For: 1.27.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > * Lucene from 8.11.2 to 8.11.3 > * Groovy from 3.0.19 to 3.0.21 > * AWS SDK from 1.12.710 to 1.12.744 > * AWS SDK from 2.25.40 to 2.25.70 > * Kotlin from 1.9.23 to 1.9.24 > * Test containers from 1.19.7 to 1.19.8 > * SLF4J from 2.0.12 to 2.0.13 > * Jackson from 2.17.0 to 2.17.1 > * JSON Smart from 2.5.0 to 2.5.1 > * Jersey from 2.41 to 2.43 > * Netty 4 from 4.1.109.Final to 4.1.111.Final > * Spring from 5.3.34 to 5.3.37 > * Spring Security from 5.8.11 to 5.8.13 > * Swagger Annotations from 1.6.12 to 1.6.14 > * JUnit from 5.10.0 to 5.10.2 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13294) Deprecate Apache Knox SSO Integration for Removal
[
https://issues.apache.org/jira/browse/NIFI-13294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-13294:
--
Fix Version/s: 1.27.0
(was: 1.26.1)
> Deprecate Apache Knox SSO Integration for Removal
> -
>
> Key: NIFI-13294
> URL: https://issues.apache.org/jira/browse/NIFI-13294
> Project: Apache NiFi
> Issue Type: Improvement
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Major
> Fix For: 1.27.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> NiFi 1.4.0 introduced support for authentication with Apache Knox [Single
> Sign-On|https://knox.apache.org/books/knox-1-6-0/user-guide.html#SSO+Cookie+Provider]
> based on JSON Web Tokens provided through a cookie and verified using a
> configurable public key.
> Separate from Apache Knox SSO authentication, Apache Knox itself provides
> [gateway
> access|https://knox.apache.org/books/knox-1-6-0/user-guide.html#Nifi+UI] as a
> proxy using the {{X-ProxiedEntitiesChain}} HTTP Header. Proxy access should
> remain supported as it is part of the X.509 client certificate authentication
> strategy. Deployment patterns based on Apache Knox gateway access work
> without any features or configuration properties specific to Knox.
> With the implementation of standards-based Single Sign-On using OpenID
> Connect and SAML 2, custom cookie-based SSO with Apache Knox should be
> deprecated for removal.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-13072) MonitorActivity processor generates a false positive on a node restart
[ https://issues.apache.org/jira/browse/NIFI-13072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13072: -- Fix Version/s: 1.27.0 (was: 2.0.0-M3) (was: 1.26.1) > MonitorActivity processor generates a false positive on a node restart > -- > > Key: NIFI-13072 > URL: https://issues.apache.org/jira/browse/NIFI-13072 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 2.0.0-M1, 1.18.0, 1.19.0, 1.20.0, 1.19.1, 1.21.0, > 1.22.0, 1.23.0, 1.24.0, 1.23.1, 1.23.2, 1.25.0, 2.0.0-M2 >Reporter: Rajmund Takacs >Assignee: Rajmund Takacs >Priority: Major > Fix For: 1.27.0 > > Attachments: MonitorActivity_Flow_Example_NiFi_1.18_OSS.json > > Time Spent: 1h > Remaining Estimate: 0h > > If the monitoring scope is configured to "cluster", then there are multiple > restart cases when markers are incorrectly generated: > h4. CASE 1 > * The flow is already inactive, and there is no traffic. The processor is > still running and scheduled on all nodes. > * We restart any of the NiFi nodes in the cluster. > * *All nodes, except the restarted one, generates activation marker.* (Reason > for this is, that the common timestamp is updated by the restarted node > during initialization.) > h4. CASE 2: > * The flow is already inactive, and there is no traffic. The processor is > still running and scheduled on all nodes. Reporting is done by all nodes. > * We temporarily stop any of the nodes. > * We start traffic on any other node, so that an activation marker is > generated. > * We start the previously stopped node again. > * *This node does not send activation marker.* (Reason for this is that the > initial state is always active.) > Expected behavior for stop/start cases would be: > * Single node restart alone should not affect the flow activity state. > * If a marker has already been sent before stopping the processor, it should > not be repeated on next start, unless asked for, or there is a change in > activity state. > * If the flow was active during a weekend shutdown, it should not begin with > an inactivity marker right after startup on monday. Although it should send > inactivity marker after the threshold has expired since startup. > * If the flow was inactive during a weekend shutdown, it should send > activation marker after started up again with traffic. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-10666) PrometheusReportingTask does not use UTF-8 encoding on /metrics/ endpoint
[
https://issues.apache.org/jira/browse/NIFI-10666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-10666:
--
Fix Version/s: 1.27.0
(was: 1.26.1)
(was: 2.0.0-M4)
> PrometheusReportingTask does not use UTF-8 encoding on /metrics/ endpoint
> -
>
> Key: NIFI-10666
> URL: https://issues.apache.org/jira/browse/NIFI-10666
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.17.0, 1.16.3, 1.18.0, 1.23.2, 1.25.0, 2.0.0-M2
> Environment: JVM with non-UTF-8 default encoding (e.g. default
> Windows installation)
>Reporter: René Zeidler
>Assignee: Jim Steinebrey
>Priority: Minor
> Labels: encoding, prometheus, utf-8
> Fix For: 1.27.0
>
> Attachments: missing-header.png
>
>
> We have created a default PrometheusReportingTask for our NiFi instance and
> tried to consume the metrics with Prometheus. However, Prometheus threw the
> following error:
> {code:java}
> ts=2022-10-19T12:25:18.110Z caller=scrape.go:1332 level=debug
> component="scrape manager" scrape_pool=nifi-cluster
> target=http://***nifi***:9092/metrics msg="Append failed" err="invalid UTF-8
> label value" {code}
> Upon further inspection, we noticed that the /metrics/ endpoint exposed by
> the reporting task does not use UTF-8 encoding, which is required by
> Prometheus (as documented here: [Exposition formats |
> Prometheus|https://prometheus.io/docs/instrumenting/exposition_formats/]).
> Our flow uses non-ASCII characters (in our case German umlauts like "ü"). As
> a workaround, removing those characters fixes the Prometheus error, but this
> is not practical for a large flow in German language.
> Opening the /metrics/ endpoint in a browser confirms that the encoding used
> is not UTF-8:
> {code:java}
> > document.characterSet
> 'windows-1252' {code}
>
> The responsible code might be here:
> [https://github.com/apache/nifi/blob/2be5c26f287469f4f19f0fa759d6c1b56dc0e348/nifi-nar-bundles/nifi-prometheus-bundle/nifi-prometheus-reporting-task/src/main/java/org/apache/nifi/reporting/prometheus/PrometheusServer.java#L67]
> The PrometheusServer used by the reporting task uses an OutputStreamWriter
> with the default encoding, instead of explicitly using UTF-8. The
> Content-Type header set in that function also does not get passed along (see
> screenshot).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12231) Add completion strategy to FetchSmb
[ https://issues.apache.org/jira/browse/NIFI-12231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-12231: -- Fix Version/s: 1.27.0 (was: 2.0.0-M3) (was: 1.26.1) > Add completion strategy to FetchSmb > --- > > Key: NIFI-12231 > URL: https://issues.apache.org/jira/browse/NIFI-12231 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Affects Versions: 1.23.2, 1.26.0 >Reporter: Jens M Kofoed >Assignee: Peter Turcsanyi >Priority: Major > Fix For: 1.27.0 > > Time Spent: 50m > Remaining Estimate: 0h > > The newly added ListSmb and FetchSmb has been very much appreciated. But > sadly the FetchSmb does not have the same options like other Fetch processors. > FetchFTP and FetchSFTP process has a Completion Strategy where you can choose > to Move or Delete the file after it has been fetched. > It would be very helpfull if FetchSmb gets the same possibilities -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13201) Deprecate Accumulo Components for Removal
[ https://issues.apache.org/jira/browse/NIFI-13201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13201: -- Fix Version/s: 1.27.0 (was: 1.26.1) > Deprecate Accumulo Components for Removal > - > > Key: NIFI-13201 > URL: https://issues.apache.org/jira/browse/NIFI-13201 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Fix For: 1.27.0 > > Time Spent: 20m > Remaining Estimate: 0h > > The Processors and Services for Apache Accumulo have not received any > improvements or notable changes for several years. These components should be > deprecated for removal to reduce dependency maintenance and avoid including > outdated transitive dependencies. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13298) Remove unused instantiated java.util.HashSet from RouteAttribute
[ https://issues.apache.org/jira/browse/NIFI-13298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13298: -- Fix Version/s: 1.27.0 (was: 1.26.1) (was: 2.0.0-M4) > Remove unused instantiated java.util.HashSet from RouteAttribute > > > Key: NIFI-13298 > URL: https://issues.apache.org/jira/browse/NIFI-13298 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Daniel Stieglitz >Assignee: Daniel Stieglitz >Priority: Minor > Fix For: 1.27.0 > > Time Spent: 20m > Remaining Estimate: 0h > > In RouteOnAttribute on line 352 a set is instantiated > final Set clones = new HashSet<>(); > and on line 358 its populated > clones.add(cloneFlowFile); > Yet after that nothing is done with it (per Intellij it is updated but > never queried). > This should removed as it is not serving any purpose. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13172) Deprecate nifi-externals/kafka-connect component in 1.x
[ https://issues.apache.org/jira/browse/NIFI-13172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13172: -- Fix Version/s: 1.27.0 (was: 1.26.1) > Deprecate nifi-externals/kafka-connect component in 1.x > --- > > Key: NIFI-13172 > URL: https://issues.apache.org/jira/browse/NIFI-13172 > Project: Apache NiFi > Issue Type: Sub-task >Reporter: Joe Witt >Assignee: Joe Witt >Priority: Major > Fix For: 1.27.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > See NIFI-13171 description for context. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13137) Switch to Zulu for MacOS Java 8 action
[ https://issues.apache.org/jira/browse/NIFI-13137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13137: -- Fix Version/s: 1.27.0 (was: 1.26.1) > Switch to Zulu for MacOS Java 8 action > -- > > Key: NIFI-13137 > URL: https://issues.apache.org/jira/browse/NIFI-13137 > Project: Apache NiFi > Issue Type: Task > Components: Tools and Build >Affects Versions: 1.26.0 >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Major > Fix For: 1.27.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > Temurin Java 8 is not available for macOS 14 on AArch64. The most > straightforward solution would be switching to some other JDK, such as Azul. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13304) Add SplitExcel Processor
[ https://issues.apache.org/jira/browse/NIFI-13304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13304: -- Fix Version/s: (was: 1.26.1) (was: 2.0.0-M4) > Add SplitExcel Processor > > > Key: NIFI-13304 > URL: https://issues.apache.org/jira/browse/NIFI-13304 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Philipp Korniets >Assignee: Daniel Stieglitz >Priority: Major > Fix For: 1.27.0 > > > Currently ConvertExcelToCSV processor, if processing excel with multiple > required sheets in it, produces flowfile per sheet. > New ExcelReader produces one flowfile with all data in it > > There should be a way to configure output to be sheet-per-flowfile > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13156) Replace use of com.fasterxml.jackson.core.JsonParser deprecated getCurrentName() with currentName() in JsonParser
[ https://issues.apache.org/jira/browse/NIFI-13156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13156: -- Fix Version/s: 1.27.0 (was: 2.0.0-M3) (was: 1.26.1) > Replace use of com.fasterxml.jackson.core.JsonParser deprecated > getCurrentName() with currentName() in JsonParser > - > > Key: NIFI-13156 > URL: https://issues.apache.org/jira/browse/NIFI-13156 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Daniel Stieglitz >Assignee: Daniel Stieglitz >Priority: Minor > Fix For: 1.27.0 > > Time Spent: 20m > Remaining Estimate: 0h > > The getCurrentName() of com.fasterxml.jackson.core.JsonParser has been > deprecated and the suggested replacement is to use method currentName(). > This should be replaced in the following places > # > nifi-extension-bundles/nifi-extension-utils/nifi-record-utils/nifi-json-record-utils/src/main/j > ava/org/apache/nifi/json/AbstractJsonRowRecordReader.java > # > nifi-extension-bundles/nifi-hubspot-bundle/nifi-hubspot-processors/src/main/java/org/apache/nifi/processors/hubspot/GetHubSpot.java > # > nifi-extension-bundles/nifi-salesforce-bundle/nifi-salesforce-processors/src/main/java/org/apac > he/nifi/processors/salesforce/QuerySalesforceObject.java > # > nifi-extension-bundles/nifi-shopify-bundle/nifi-shopify-processors/src/main/java/org/apache/nifi/processors/shopify/GetShopify.java > # > nifi-extension-bundles/nifi-zendesk-bundle/nifi-zendesk-processors/src/main/java/org/apache/nifi/processors/zendesk/GetZendesk.java -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13340) Flowfiles stopped to be ingested before a processor group
[
https://issues.apache.org/jira/browse/NIFI-13340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-13340:
--
Fix Version/s: 1.27.0
(was: 1.26.1)
(was: 2.0.0-M4)
> Flowfiles stopped to be ingested before a processor group
> -
>
> Key: NIFI-13340
> URL: https://issues.apache.org/jira/browse/NIFI-13340
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.25.0
> Environment: Host OS :ubuntu 20.04 in wsl
> CPU: Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
> RAM: 128GB
> ZFS ARC cache was configured to be maximum 4 GB
>Reporter: Yuanhao Zhu
>Assignee: Mark Payne
>Priority: Major
> Labels: data-consistency, statestore
> Fix For: 1.27.0
>
> Attachments: image-2024-06-03-14-42-37-280.png,
> image-2024-06-03-14-44-15-590.png
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> *Background:* We run nifi as a standalone instance in a docker container in
> on all our stages, the statemanagement provider used by our instance is the
> local WAL backed provider.
> {*}Description{*}: We observed that the flowfiles in front of one of our
> processor groups stopped to be ingested once in a while and it happens on all
> our stages without noticeable pattern. The flowfile concurrency policy of the
> processor group that stopped ingesting data is set to SINGLE_BATCH_PER_NODE
> and the outbound policy is set to BATCH_OUTPUT. The ingestion should continue
> since the processor group had already send out every flowfile in it, but it
> stopped.
>
> There is only one brutal solution from our side(We need to delete the
> processor group and restore it from nifi registry again) and the occurrence
> of this issue had impacted our data ingestion.
> !image-2024-06-03-14-42-37-280.png!
> !image-2024-06-03-14-44-15-590.png!
> As you can see in the screenshot that the processor group 'Delete before
> Insert' has no more flowfile to output but still it does not ingest the data
> queued in the input port
>
> {{In the log file I found the following:}}
>
> {code:java}
> 2024-06-03 11:34:01,772 TRACE [Timer-Driven Process Thread-15]
> o.apache.nifi.groups.StandardDataValve Will not allow data to flow into
> StandardProcessGroup[identifier=5eb0ad69-e8ed-3ba0-52da-af94fb9836cd,name=Delete
> before Insert] because Outbound Policy is Batch Output and valve is already
> open to allow data to flow out of group{code}
>
> {{ }}
> {{Also in the diagnostics, I found the following for the 'Delete before
> Insert' processor group:}}
> {{ }}
> {code:java}
> Process Group e6510a87-aa78-3268-1b11-3c310f0ad144, Name = Search and Delete
> existing reports(This is the parent processor group of the Delete before
> Insert)
> Currently Have Data Flowing In: []
> Currently Have Data Flowing Out:
> [StandardProcessGroup[identifier=5eb0ad69-e8ed-3ba0-52da-af94fb9836cd,name=Delete
> before Insert]]
> Reason for Not allowing data to flow in:
> Data Valve is already allowing data to flow out of group:
>
> StandardProcessGroup[identifier=5eb0ad69-e8ed-3ba0-52da-af94fb9836cd,name=Delete
> before Insert]
> Reason for Not allowing data to flow out:
> Output is Allowed:
>
> StandardProcessGroup[identifier=5eb0ad69-e8ed-3ba0-52da-af94fb9836cd,name=Delete
> before Insert] {code}
> {{ }}
> {{Which is clearly {*}not correct{*}. since there are currently not data
> flowing out from the 'Delete before Insert' processor group.}}
> {{ }}
> {{We dig through the source code of StandardDataValve.java and found that
> that data valve's states are stored every time the data valve is opened and
> close so the potential reason causing this issue is that the processor group
> id was put into the statemap when data flowed in but somehow the removal of
> the entry was not successful. We are aware that if the statemap is not stored
> before the nifi restarts, it could lead to such circumstances, but in the
> recent occurrences of this issue, there were not nifi restart recorded or
> observed at the time when all those flowfiles started to queue in front of
> the processor group}}
> {{ }}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-13422) QueryNiFiReportingTask gives error when joining tables
[ https://issues.apache.org/jira/browse/NIFI-13422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13422: -- Fix Version/s: 1.27.0 (was: 1.26.1) > QueryNiFiReportingTask gives error when joining tables > -- > > Key: NIFI-13422 > URL: https://issues.apache.org/jira/browse/NIFI-13422 > Project: Apache NiFi > Issue Type: Bug >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Major > Fix For: 1.27.0 > > > Using QueryNiFiReportingTask in 1.x, an error occurs when doing a join on > tables, such as: > SELECT > CONNECTION_STATUS.id, > CONNECTION_STATUS.name, > CONNECTION_STATUS.sourceName, > CONNECTION_STATUS.destinationName, > CONNECTION_STATUS.queuedCount, > CONNECTION_STATUS.backPressureObjectThreshold, > CONNECTION_STATUS_PREDICTIONS.predictedTimeToCountBackpressureMillis > FROM CONNECTION_STATUS > JOIN CONNECTION_STATUS_PREDICTIONS ON > CONNECTION_STATUS_PREDICTIONS.connectionId = CONNECTION_STATUS.id > WHERE > predictedTimeToCountBackpressureMillis < 30 > OR (backPressureObjectThreshold <> 0 AND queuedCount * 100 / > backPressureObjectThreshold > 75) > The error is: > 2024-06-13 17:26:03,034 INFO > org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent: > QueryNiFiReportingTask[id=1229bc0c-0190-1000--ca3d5135] started. > 2024-06-13 17:26:03,048 ERROR > org.apache.nifi.reporting.sql.QueryNiFiReportingTask: > QueryNiFiReportingTask[id=1229bc0c-0190-1000--ca3d5135] Error running > task QueryNiFiReportingTask[id=1229bc0c-0190-1000--ca3d5135] due to > java.lang.AssertionError: Rule's description should be unique; existing > rule=ConnectionStatusProjectTableScanRule; new > rule=ConnectionStatusProjectTableScanRule > 2024-06-13 17:26:06,437 INFO > org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent: Stopped > scheduling QueryNiFiReportingTask[id=1229bc0c-0190-1000--ca3d5135] to > run > This is because the ScanRules use a singleton instance. The main (2.x) branch > does not suffer from this because that bundle has been since refactored, but > we should fix it on the 1.x branch. The ScanRule implementations should > create new instances as necessary, and the description field is not necessary. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12669) EvaluateXQuery processor incorrectly encodes result attributes
[
https://issues.apache.org/jira/browse/NIFI-12669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-12669:
--
Fix Version/s: 1.27.0
(was: 1.26.1)
(was: 2.0.0-M4)
> EvaluateXQuery processor incorrectly encodes result attributes
> --
>
> Key: NIFI-12669
> URL: https://issues.apache.org/jira/browse/NIFI-12669
> Project: Apache NiFi
> Issue Type: Bug
> Components: Configuration, Extensions
> Environment: JVM with non-UTF-8 default encoding (e.g. default
> Windows installation)
>Reporter: René Zeidler
>Assignee: Jim Steinebrey
>Priority: Major
> Labels: encoding, utf8, windows, xml
> Fix For: 1.27.0
>
> Attachments: EvaluateXQuery_Encoding_Bug.json,
> image-2024-01-25-10-24-17-005.png, image-2024-01-25-10-31-35-200.png
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> h2. Environment
> This issue affects environments where the JVM default encoding is not
> {{{}UTF-8{}}}. Standard Java installations on Windows are affected, as they
> usually use the default encoding {{{}windows-1252{}}}. To reproduce the issue
> on Linux, change the default encoding to {{windows-1252}} by adding the
> following line to your {{{}bootstrap.conf{}}}:
> {quote}{{java.arg.21=-Dfile.encoding=windows-1252}}
> {quote}
> h2. Summary
> The EvaluateXQuery incorrectly encodes result values when storing them in
> attributes. This causes non-ASCII characters to be garbled.
> Example:
> !image-2024-01-25-10-24-17-005.png!
> h2. Steps to reproduce
> # Make sure NiFi runs with a non-UTF-8 default encoding, see "Environment"
> # Create a GenerateFlowFile processor with the following content:
> {quote}{{}}
> {{}}
> {{ This text contains non-ASCII characters: ÄÖÜäöüßéèóò}}
> {{}}
> {quote}
> # Connect the processor to an EvaluateXQuery processor.
> Set the {{Destination}} to {{{}flowfile-attribute{}}}.
> Create a custom property {{myData}} with value {{{}string(/myRoot/myData){}}}.
> # Connect the outputs of the EvaluateXQuery processor to funnels to be able
> to observe the result in the queue.
> # Start the EvaluateXQuery processor and run the GenerateFlowFile processor
> once.
> The flow should look similar to this:
> !image-2024-01-25-10-31-35-200.png!
> I also attached a JSON export of the example flow.
> # Observe the attributes of the resulting FlowFile in the queue.
> h3. Expected Result
> The FlowFile should contain an attribute {{myData}} with the value {{{}"This
> text contains non-ASCII characters: ÄÖÜäöüßéèóò"{}}}.
> h3. Actual Result
> The attribute has the value "This text contains non-ASCII characters:
> ÄÖÜäöüßéèóò".
> h2. Root Cause Analysis
> EvaluateXQuery uses the method
> [{{formatItem}}|https://github.com/apache/nifi/blob/2e3f83eb54cbc040b5a1da5bce9a74a558f08ea4/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EvaluateXQuery.java#L368-L372]
> to write the query result to an attribute. This method calls
> {{{}ByteArrayOutputStream{}}}'s
> [toString|https://docs.oracle.com/en/java/javase/21/docs/api/java.base/java/io/ByteArrayOutputStream.html#toString()]
> method without an encoding argument, which then defaults to the default
> charset of the environment. Bytes are always written to this output stream
> using UTF-8
> ([.getBytes(StandardCharsets.UTF8)|https://github.com/apache/nifi/blob/2e3f83eb54cbc040b5a1da5bce9a74a558f08ea4/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EvaluateXQuery.java#L397]).
> When the default charset is not UTF-8, this results in UTF-8 bytes to be
> interpreted in a different encoding when converting to a string, resulting in
> garbled text (see above).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12704) Record Path function escapeJson raises NPE when given root "/" as argument
[
https://issues.apache.org/jira/browse/NIFI-12704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-12704:
--
Fix Version/s: 1.27.0
(was: 1.26.1)
(was: 2.0.0-M4)
> Record Path function escapeJson raises NPE when given root "/" as argument
> --
>
> Key: NIFI-12704
> URL: https://issues.apache.org/jira/browse/NIFI-12704
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.24.0, 2.0.0-M2
> Environment: Docker, RedHat 8
>Reporter: Stephen Jeffrey Hindmarch
>Assignee: endzeit
>Priority: Minor
> Fix For: 1.27.0
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> My use case is I want to create a field called "original" which is the
> escaped string of my whole record. This will preserve the original contents
> of the message before I start transforming it.
> My expectation is that I should be able to use an UpdateRecord processor to
> create the field using RecordPath language.
> What actually happens is that when I use
> {code:java}
> escapeJson(/)
> {code}
> as the function the result is that the processor throws a Null Pointer
> Exception (NPE).
> Detail:
> For any input flow file containing JSON records, define an UpdateRecord
> processor with these settings.
> * Replacement Value Strategy = Record Path Value
> * /original = escapeJson(/)
> When the flow file is presented to the processor, the processor generates an
> NPE.
> For example, I present this record.
> {noformat}
> [{"hello":"world","record":{"key":"one","value":"hello","subrecord":{"key":"two","value":"bob"}},"array":[0,1,2]}]{noformat}
> If the escapeJson function is offered an existing field to escape then it
> works as expected. Stack trace as follows.
> {noformat}
> 2024-01-31 12:17:15 2024-01-31 12:17:14,741 ERROR [Timer-Driven Process
> Thread-8] o.a.n.processors.standard.UpdateRecord
> UpdateRecord[id=5f6b498d-018d-1000--323c6b89] Failed to process
> StandardFlowFileRecord[uuid=2d4a3eab-e752-4117-93d3-3e5515b6f3f8,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1706703050563-1, container=default,
> section=1], offset=114,
> length=114],offset=0,name=2d4a3eab-e752-4117-93d3-3e5515b6f3f8,size=114];
> will route to failure
> 2024-01-31 12:17:15
> org.apache.nifi.record.path.exception.RecordPathException:
> java.lang.NullPointerException
> 2024-01-31 12:17:15 at
> org.apache.nifi.record.path.RecordPath.compile(RecordPath.java:105)
> 2024-01-31 12:17:15 at
> com.github.benmanes.caffeine.cache.LocalLoadingCache.lambda$newMappingFunction$2(LocalLoadingCache.java:145)
> 2024-01-31 12:17:15 at
> com.github.benmanes.caffeine.cache.BoundedLocalCache.lambda$doComputeIfAbsent$14(BoundedLocalCache.java:2406)
> 2024-01-31 12:17:15 at
> java.base/java.util.concurrent.ConcurrentHashMap.compute(Unknown Source)
> 2024-01-31 12:17:15 at
> com.github.benmanes.caffeine.cache.BoundedLocalCache.doComputeIfAbsent(BoundedLocalCache.java:2404)
> 2024-01-31 12:17:15 at
> com.github.benmanes.caffeine.cache.BoundedLocalCache.computeIfAbsent(BoundedLocalCache.java:2387)
> 2024-01-31 12:17:15 at
> com.github.benmanes.caffeine.cache.LocalCache.computeIfAbsent(LocalCache.java:108)
> 2024-01-31 12:17:15 at
> com.github.benmanes.caffeine.cache.LocalLoadingCache.get(LocalLoadingCache.java:56)
> 2024-01-31 12:17:15 at
> org.apache.nifi.record.path.util.RecordPathCache.getCompiled(RecordPathCache.java:34)
> 2024-01-31 12:17:15 at
> org.apache.nifi.processors.standard.UpdateRecord.process(UpdateRecord.java:166)
> 2024-01-31 12:17:15 at
> org.apache.nifi.processors.standard.AbstractRecordProcessor$1.process(AbstractRecordProcessor.java:147)
> 2024-01-31 12:17:15 at
> org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:3432)
> 2024-01-31 12:17:15 at
> org.apache.nifi.processors.standard.AbstractRecordProcessor.onTrigger(AbstractRecordProcessor.java:122)
> 2024-01-31 12:17:15 at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> 2024-01-31 12:17:15 at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1361)
> 2024-01-31 12:17:15 at
> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:247)
> 2024-01-31 12:17:15 at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:102)
> 2024-01-31 12:17:15 at
> org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
> 2024-01-31 12:17:15 at
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
> 2024-01-31 12:17:15
[jira] [Updated] (NIFI-13181) Azure Blob and ADLS processors throw NoSuchMethodError when Service Principal is used
[
https://issues.apache.org/jira/browse/NIFI-13181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-13181:
--
Fix Version/s: 1.27.0
(was: 1.26.1)
> Azure Blob and ADLS processors throw NoSuchMethodError when Service Principal
> is used
> -
>
> Key: NIFI-13181
> URL: https://issues.apache.org/jira/browse/NIFI-13181
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.26.0
>Reporter: Zoltán Kornél Török
>Assignee: Peter Turcsanyi
>Priority: Major
> Fix For: 1.27.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> I spent some time testing nifi 1.26 and found this error, when tried to use
> blobstorage related processors, like ListAzureBlobStorage_v12 .
> {code:java}
> 2024-05-08 09:09:06,416 WARN reactor.core.Exceptions: throwIfFatal detected a
> jvm fatal exception, which is thrown and logged below:
> java.lang.NoSuchMethodError:
> com.microsoft.aad.msal4j.ConfidentialClientApplication$Builder.logPii(Z)Lcom/microsoft/aad/msal4j/AbstractApplicationBase$Builder;
> at
> com.azure.identity.implementation.IdentityClientBase.getConfidentialClient(IdentityClientBase.java:233)
> at
> com.azure.identity.implementation.IdentityClient.lambda$getConfidentialClientApplication$4(IdentityClient.java:130)
> at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:45)
> at
> reactor.core.publisher.MonoCacheTime.subscribeOrReturn(MonoCacheTime.java:143)
> at
> reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:57)
> at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
> at
> reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64)
> at reactor.core.publisher.MonoUsing.subscribe(MonoUsing.java:102)
> at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
> at
> reactor.core.publisher.MonoFromFluxOperator.subscribe(MonoFromFluxOperator.java:81)
> at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
> at reactor.core.publisher.Mono.subscribe(Mono.java:4491)
> at
> reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.subscribeNext(MonoIgnoreThen.java:263)
> at reactor.core.publisher.MonoIgnoreThen.subscribe(MonoIgnoreThen.java:51)
> at
> reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64)
> at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
> at reactor.core.publisher.Mono.subscribe(Mono.java:4491)
> at
> reactor.core.publisher.FluxFlatMap.trySubscribeScalarMap(FluxFlatMap.java:203)
> at
> reactor.core.publisher.MonoFlatMap.subscribeOrReturn(MonoFlatMap.java:53)
> at
> reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:57)
> at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
> at reactor.core.publisher.Mono.subscribe(Mono.java:4491)
> at
> reactor.core.publisher.FluxFlatMap.trySubscribeScalarMap(FluxFlatMap.java:203)
> at
> reactor.core.publisher.MonoFlatMap.subscribeOrReturn(MonoFlatMap.java:53)
> at reactor.core.publisher.Flux.subscribe(Flux.java:8628)
> at reactor.core.publisher.Flux.blockLast(Flux.java:2760)
> at
> com.azure.core.util.paging.ContinuablePagedByIteratorBase.requestPage(ContinuablePagedByIteratorBase.java:102)
> at
> com.azure.core.util.paging.ContinuablePagedByItemIterable$ContinuablePagedByItemIterator.(ContinuablePagedByItemIterable.java:75)
> at
> com.azure.core.util.paging.ContinuablePagedByItemIterable.iterator(ContinuablePagedByItemIterable.java:55)
> at
> com.azure.core.util.paging.ContinuablePagedIterable.iterator(ContinuablePagedIterable.java:141)
> at
> org.apache.nifi.processors.azure.storage.ListAzureBlobStorage_v12.performListing(ListAzureBlobStorage_v12.java:230)
> at
> org.apache.nifi.processor.util.list.AbstractListProcessor.lambda$listByTrackingEntities$11(AbstractListProcessor.java:1126)
> at
> org.apache.nifi.processor.util.list.ListedEntityTracker.trackEntities(ListedEntityTracker.java:272)
> at
> org.apache.nifi.processor.util.list.AbstractListProcessor.listByTrackingEntities(AbstractListProcessor.java:1124)
> at
> org.apache.nifi.processor.util.list.AbstractListProcessor.onTrigger(AbstractListProcessor.java:529)
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1361)
> at
> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:247)
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:102)
> at
[jira] [Updated] (NIFI-12896) Add Endpoint Override Property to PutSNS
[ https://issues.apache.org/jira/browse/NIFI-12896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-12896: -- Fix Version/s: 1.27.0 (was: 1.26.1) (was: 2.0.0-M4) > Add Endpoint Override Property to PutSNS > > > Key: NIFI-12896 > URL: https://issues.apache.org/jira/browse/NIFI-12896 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Sue Kim >Assignee: Sue Kim >Priority: Minor > Fix For: 1.27.0 > > Time Spent: 3h 10m > Remaining Estimate: 0h > > Request for PutSNS processor to have an endpoint override property similar to > Get/PutSQS processors. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12750) ExecuteStreamCommand incorrectly decodes error stream
[
https://issues.apache.org/jira/browse/NIFI-12750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-12750:
--
Fix Version/s: 1.27.0
(was: 1.26.1)
(was: 2.0.0-M4)
> ExecuteStreamCommand incorrectly decodes error stream
> -
>
> Key: NIFI-12750
> URL: https://issues.apache.org/jira/browse/NIFI-12750
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.25.0, 2.0.0-M2
> Environment: any
>Reporter: René Zeidler
>Assignee: Jim Steinebrey
>Priority: Major
> Labels: ExecuteStreamCommand, encoding, iso-8859-1, utf-8
> Fix For: 1.27.0
>
> Attachments: ExecuteStreamCommand_Encoding_Bug.json, encodingTest.sh,
> image-2024-02-07-15-14-08-518.png, image-2024-02-07-15-14-54-841.png,
> image-2024-02-07-15-20-11-684.png
>
>
> h1. Summary
> The ExecuteStreamCommand processor stores everything the invoked command
> writes to the error stream (stderr) into the FlowFile attribute
> {{{}execution.error{}}}.
> When converting the bytes from the stream to a String, it interprets each
> individual byte as a Unicode codepoint. When reading only single bytes this
> effectively results in ISO-8859-1 (Latin-1).
> Instead, it should use the system default encoding (like it already does for
> writing stdout if Output Destination Attribute is set) or use a configurable
> encoding (for both stdout and stderr).
> h1. Details
> When reading/writing FlowFiles, NiFi always uses raw bytes, so encoding
> issues are the responsibility of the flow designer, and NiFi has the
> ConvertCharacterSet processor to deal with those issues.
> When writing to attributes, the API uses Java String objects, which are
> encoding agnostic (they represent Unicode codepoints, not bytes). Therefore,
> processors receiving bytes have to interpret them using an encoding.
> The ExecuteStreamCommand processor writes the output of the command (stdout)
> to the Output Destination Attribute (if set). To do that, it convertes bytes
> into a String using the system default encoding* by calling {{new String}}
> without an encoding argument:
> [https://github.com/apache/nifi/blob/72f6d8a6800c643d5f283ae9bff6d7de25b503e9/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ExecuteStreamCommand.java#L499]
> When converting stderr to a String to write into the {{execution.error}}
> attribute, it uses this weird algorithm:
> [https://github.com/apache/nifi/blob/72f6d8a6800c643d5f283ae9bff6d7de25b503e9/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ExecuteStreamCommand.java#L507-L517]
> It reads individual bytes from the error stream (as {{{}int{}}}s) and casts
> them to {{{}char{}}}s. What Java does in this case is interpret the integer
> as a Unicode code point. For single bytes, this matches the ISO-8859-1
> encoding. Instead, it should use the same decoding method as for stdout.
> h1. Reproduction steps
> These steps are for a Linux environment, but can be adapted with a different
> executable for Windows.
> # Create the file /opt/nifi/data/encodingTest.sh (attached) with the
> following contents and make it executable:
> {quote}{{#/bin/bash}}
> {{echo "|out static: ÄÖÜäöüß"}}
> {{{}echo "|error static: ÄÖÜäöüß" >&2{}}}{{{}echo "|out arg: $1"{}}}
> {{{}echo "|error arg: $1" >&2{}}}{{{}echo "|out arg hexdump:"{}}}
> {{printf '%s' "$1" | od -A x -t x1z -v}}
> {{echo "|error arg hexdump:" >&2}}
> {{printf '%s' "$1" | od -A x -t x1z -v >&2}}{quote}The script writes
> identical data to both stdout and stderr. It contains non-ASCII characters to
> make the encoding issues visible.
> # Import the attached flow or create it manually:
> !image-2024-02-07-15-14-08-518.png|width=324,height=373!!image-2024-02-07-15-14-54-841.png|width=326,height=120!
> # Run the GenerateFlowFile processor once and observe the attributes of the
> FlowFile in the final queue:
> !image-2024-02-07-15-20-11-684.png|width=523,height=195!
> The output attribute (stdout) is correctly decoded. The execution.error
> attribute (stderr) contains garbled text (UTF-8 bytes interpreted as
> ISO-8859-1 and reencoded in UTF-8).
> h1. *On the system default encoding
> The system default encoding is a property of the JVM. It is UTF-8 on Linux,
> but Windows-1252 (or a different copepage depending on locale) in Windows
> environments. It can be overriden using the {{file.encoding}} JVM arg on
> startup.
> Relying on the system default encoding is dangerous and can lead to subtle
> bugs, like the ones I previously reported (NIFI-12669 and NIFI-12670).
> In this case, it might make sense to use the system
[jira] [Updated] (NIFI-13296) Deprecate Kerberos SPNEGO Authentication for Removal
[ https://issues.apache.org/jira/browse/NIFI-13296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13296: -- Fix Version/s: 1.27.0 (was: 1.26.1) > Deprecate Kerberos SPNEGO Authentication for Removal > > > Key: NIFI-13296 > URL: https://issues.apache.org/jira/browse/NIFI-13296 > Project: Apache NiFi > Issue Type: Improvement >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Fix For: 1.27.0 > > Time Spent: 40m > Remaining Estimate: 0h > > NiFi 0.6.0 added Kerberos authentication with > [SPNEGO|https://en.wikipedia.org/wiki/SPNEGO] as a framework feature based on > Spring Security Kerberos. Although Spring Security Kerberos continues to be > maintained, SPNEGO authentication is not common, requiring specialized > [client browser > configuration|https://docs.spring.io/spring-security-kerberos/docs/current/reference/html/browserspnegoconfig.html] > for access. As noted in the linked instructions, popular web browsers do not > support SPNEGO in the default configuration, and Google Chrome requires > either a custom policy or launch from the command line with arguments that > list permitted DNS names. > Based on these considerations, and in light of more common Single Sign-On > strategies using OpenID Connect and SAML 2, NiFi framework support for > Kerberos authentication with SPNEGO should be deprecated for subsequent > removal in NiFi 2. > This deprecation should not impact the Kerberos Login Identity Provider, > which continues to support username and password authentication based on the > form-based login process. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11658) Streamline using single parameter context for nested PGs
[ https://issues.apache.org/jira/browse/NIFI-11658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-11658: -- Fix Version/s: 1.27.0 (was: 2.0.0-M1) (was: 1.26.1) > Streamline using single parameter context for nested PGs > > > Key: NIFI-11658 > URL: https://issues.apache.org/jira/browse/NIFI-11658 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Timea Barna >Assignee: Timea Barna >Priority: Major > Fix For: 1.27.0 > > Time Spent: 2h 20m > Remaining Estimate: 0h > > 1. when adding a new process group to the canvas, in the create PG view where > one should give the name of the process group, add a drop down menu to select > the parameter context to associate with the process group. It would default > to the parameter context associated to the parent process group (if there is > one specified and if the user has the permissions) > 2. in the configuration view of a process group, add a dedicated tab for > parameter context to have dedicated handling. In addition to the dropdown > menu for selecting a parameter context, add a checkbox to recursively apply > the change. This checkbox would only be used when clicking "Apply", it would > not be persisted. If checked, it means that the parameter context will be > applied to the process group as well to all the embedded process groups > (recursively) if and only if the user has the proper permissions on all > process groups. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13356) Protobuf Reader fails on parsing certain repeated fields
[ https://issues.apache.org/jira/browse/NIFI-13356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13356: -- Fix Version/s: 1.27.0 (was: 1.26.1) (was: 2.0.0-M4) > Protobuf Reader fails on parsing certain repeated fields > > > Key: NIFI-13356 > URL: https://issues.apache.org/jira/browse/NIFI-13356 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.26.0, 2.0.0-M3 >Reporter: Mark Bathori >Assignee: Mark Bathori >Priority: Major > Fix For: 1.27.0 > > Time Spent: 1h 20m > Remaining Estimate: 0h > > The Protobuf Reader fails on parsing repeated fields where the field type is > not LengthDelimited. The UnknownFieldSet parser is unable to process these > repeated fields, instead it converts them to byte array so they need to be > handled separately. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13191) Deprecate InvokeAWSGatewayApi Processor
[ https://issues.apache.org/jira/browse/NIFI-13191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13191: -- Fix Version/s: 1.27.0 (was: 1.26.1) > Deprecate InvokeAWSGatewayApi Processor > --- > > Key: NIFI-13191 > URL: https://issues.apache.org/jira/browse/NIFI-13191 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Joe Gresock >Assignee: Joe Gresock >Priority: Minor > Fix For: 1.27.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Similar to the other AWS client library upgrade issues, this targets the > InvokeAWSGatewayApi processor. > Note: After discussing with [~exceptionfactory] and [~otto], we agree that > this processor can be deprecated, and removed in NiFi 2.0. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13400) SmbDfsIT failures
[ https://issues.apache.org/jira/browse/NIFI-13400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13400: -- Fix Version/s: 1.27.0 (was: 1.26.1) (was: 2.0.0-M4) > SmbDfsIT failures > -- > > Key: NIFI-13400 > URL: https://issues.apache.org/jira/browse/NIFI-13400 > Project: Apache NiFi > Issue Type: Task > Environment: Apache Maven 3.9.6 > (bc0240f3c744dd6b6ec2920b3cd08dcc295161ae) > Maven home: /development/tools/apache-maven-3.9.6 > Java version: 21.0.3, vendor: Azul Systems, Inc., runtime: > /usr/lib/jvm/zulu21.34.19-ca-jdk21.0.3-linux_x64 > Default locale: en_US, platform encoding: UTF-8 > OS name: "linux", version: "6.8.9-200.fc39.x86_64", arch: "amd64", family: > "unix" >Reporter: Joe Witt >Assignee: Peter Turcsanyi >Priority: Major > Fix For: 1.27.0 > > Attachments: TEST-org.apache.nifi.processors.smb.SmbDfsIT.xml > > > [ERROR] Failures: > [ERROR] SmbDfsIT.testFetchSmb:94 expected: <1> but was: <0> > [ERROR] SmbDfsIT.testGetSmbFile:185 expected: <1> but was: <0> > [ERROR] SmbDfsIT.testListSmbWithDfsLink:121->testListSmb:143 expected: <1> > but was: <0> > [ERROR] SmbDfsIT.testPutSmbFile:165 expected: <1> but was: <0> > NIFI-12837 introduced this IT. It does not appear to work successfully on > linux hosts. Unclear exactly. Does seem to work on osx. Perhaps the test > is setup to assume arm or something. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13008) CLI command to upgrade all instances of a versioned flow
[ https://issues.apache.org/jira/browse/NIFI-13008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13008: -- Fix Version/s: 1.27.0 (was: 2.0.0-M3) (was: 1.26.1) > CLI command to upgrade all instances of a versioned flow > > > Key: NIFI-13008 > URL: https://issues.apache.org/jira/browse/NIFI-13008 > Project: Apache NiFi > Issue Type: Improvement > Components: Tools and Build >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Major > Labels: backport-needed > Fix For: 1.27.0 > > Time Spent: 1h 20m > Remaining Estimate: 0h > > Add a CLI command allowing someone to upgrade all the instances of a given > versioned flow. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13152) Deprecate Datadog Reporting Task
[ https://issues.apache.org/jira/browse/NIFI-13152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13152: -- Fix Version/s: 1.27.0 (was: 1.26.1) > Deprecate Datadog Reporting Task > > > Key: NIFI-13152 > URL: https://issues.apache.org/jira/browse/NIFI-13152 > Project: Apache NiFi > Issue Type: Sub-task >Reporter: Joe Witt >Assignee: Joe Witt >Priority: Major > Fix For: 1.27.0 > > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-13454) Rename FileParameterProvider to KubernetesSecretParameterProvider
[ https://issues.apache.org/jira/browse/NIFI-13454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17860489#comment-17860489 ] ASF subversion and git services commented on NIFI-13454: Commit b105b5bbef5df022b3df0c96bb6af39b19691fa1 in nifi's branch refs/heads/main from Joe Gresock [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=b105b5bbef ] NIFI-13454 Renamed FileParameterProvider to KubernetesSecretParameterProvider This closes #9008 Signed-off-by: David Handermann > Rename FileParameterProvider to KubernetesSecretParameterProvider > - > > Key: NIFI-13454 > URL: https://issues.apache.org/jira/browse/NIFI-13454 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 2.0.0-M4 >Reporter: Joe Gresock >Assignee: Joe Gresock >Priority: Major > > The FileParameterProvider was designed specifically to support Kubernetes > mounted Secrets, with a single K8s secret being represented by a single file, > mapped to a single parameter. However, the name of the parameter provider > obscures its use case, and makes it seem like a general purpose file-based > parameter provider. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13454) Rename FileParameterProvider to KubernetesSecretParameterProvider
[ https://issues.apache.org/jira/browse/NIFI-13454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-13454: Fix Version/s: 2.0.0-M4 Resolution: Fixed Status: Resolved (was: Patch Available) > Rename FileParameterProvider to KubernetesSecretParameterProvider > - > > Key: NIFI-13454 > URL: https://issues.apache.org/jira/browse/NIFI-13454 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 2.0.0-M4 >Reporter: Joe Gresock >Assignee: Joe Gresock >Priority: Major > Fix For: 2.0.0-M4 > > > The FileParameterProvider was designed specifically to support Kubernetes > mounted Secrets, with a single K8s secret being represented by a single file, > mapped to a single parameter. However, the name of the parameter provider > obscures its use case, and makes it seem like a general purpose file-based > parameter provider. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (MINIFICPP-2422) Remove ubuntu 20.04 support
Ferenc Gerlits created MINIFICPP-2422: - Summary: Remove ubuntu 20.04 support Key: MINIFICPP-2422 URL: https://issues.apache.org/jira/browse/MINIFICPP-2422 Project: Apache NiFi MiNiFi C++ Issue Type: Task Reporter: Ferenc Gerlits Assignee: Ferenc Gerlits Remove support for ubuntu 20.04 everywhere, replace with either ubuntu 22.04 or 24.04. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-13459) Update to 1.26.0 breaks some PutAzureDataLakeStorage functionality
[
https://issues.apache.org/jira/browse/NIFI-13459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Turcsanyi resolved NIFI-13459.
Resolution: Duplicate
> Update to 1.26.0 breaks some PutAzureDataLakeStorage functionality
> --
>
> Key: NIFI-13459
> URL: https://issues.apache.org/jira/browse/NIFI-13459
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.26.0
> Environment: Azure Kubernetes Service 1.28.9
> Standard_D4s_v3
>Reporter: Moritz Larndorfer
>Priority: Minor
> Attachments: error.png
>
>
> When updating from 1.25.0. to 1.26.0 a PutAzureDataLakeStorage processor
> started throwing the error shown in the attached image.
> These are the descriptions of the processor and its
> ADLSCredentialsControllerService:
> {
> "identifier": "",
> "instanceIdentifier": "",
> "name": "Store in Datalake",
> "type":
> "org.apache.nifi.processors.azure.storage.PutAzureDataLakeStorage",
> "bundle":
> { "group": "org.apache.nifi", "artifact": "nifi-azure-nar",
> "version": "1.25.0" }
> ,
> "properties": {
> "filesystem-name": "rawdata",
> "conflict-resolution-strategy": "ignore",
> "adls-credentials-service": "",
> "Resource Transfer Source": "FLOWFILE_CONTENT",
> "directory-name": "${path}",
> "file-name": "${filename}",
> "base-temporary-path": ""
> },
> "schedulingPeriod": "0 sec",
> "schedulingStrategy": "TIMER_DRIVEN",
> "executionNode": "ALL",
> "penaltyDuration": "30 sec",
> "yieldDuration": "1 sec",
> "bulletinLevel": "WARN",
> "runDurationMillis": 0,
> "concurrentlySchedulableTaskCount": 1,
> "autoTerminatedRelationships": [],
> "scheduledState": "RUNNING",
> "retryCount": 10,
> "retriedRelationships": [],
> "backoffMechanism": "PENALIZE_FLOWFILE",
> "maxBackoffPeriod": "10 mins",
> "componentType": "PROCESSOR",
> "groupIdentifier": ""
> }
> {
> "identifier": "",
> "instanceIdentifier": "",
> "name": "ADLSCredentialsControllerService FillDatalake",
> "comments": "",
> "type":
> "org.apache.nifi.services.azure.storage.ADLSCredentialsControllerService",
> "bundle":
> { "group": "org.apache.nifi", "artifact": "nifi-azure-nar",
> "version": "1.25.0" }
> ,
> "properties":
> { "storage-account-name": "",
> "service-principal-client-secret": "",
> "storage-use-managed-identity": "false", "storage-endpoint-suffix":
> "dfs.core.windows.net", "service-principal-tenant-id": "",
> "service-principal-client-id": "" }
> ,
> "propertyDescriptors": {},
> "controllerServiceApis": [{
> "type":
> "org.apache.nifi.services.azure.storage.ADLSCredentialsService",
> "bundle":
> { "group": "org.apache.nifi", "artifact":
> "nifi-azure-services-api-nar", "version": "1.25.0"
> }
> }
> ],
> "scheduledState": "ENABLED",
> "bulletinLevel": "WARN",
> "componentType": "CONTROLLER_SERVICE",
> "groupIdentifier": ""
> }
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (NIFI-13450) MiNiFi Agent Manifest is not regenerated between Heartbeats
[ https://issues.apache.org/jira/browse/NIFI-13450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Csaba Bejan resolved NIFI-13450. Resolution: Fixed > MiNiFi Agent Manifest is not regenerated between Heartbeats > --- > > Key: NIFI-13450 > URL: https://issues.apache.org/jira/browse/NIFI-13450 > Project: Apache NiFi > Issue Type: Bug > Components: MiNiFi >Affects Versions: 2.0.0-M3 >Reporter: Ferenc Erdei >Assignee: Ferenc Erdei >Priority: Major > Labels: minifi-java > Fix For: 2.0.0-M3, 2.0.0-M2, 2.0.0-M1 > > > The RuntimeInfo is passed into the C2HeartbeatManager's constructor as a > variable, therefore it's not being regenerated for each heartbeat. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-13459) Update to 1.26.0 breaks some PutAzureDataLakeStorage functionality
[
https://issues.apache.org/jira/browse/NIFI-13459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17860449#comment-17860449
]
Peter Turcsanyi commented on NIFI-13459:
[~mosbythegreat] Thanks for reporting the issue! It has been fixed in
NIFI-13181 and is going to be shipped in the upcoming 1.26.1 release.
> Update to 1.26.0 breaks some PutAzureDataLakeStorage functionality
> --
>
> Key: NIFI-13459
> URL: https://issues.apache.org/jira/browse/NIFI-13459
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.26.0
> Environment: Azure Kubernetes Service 1.28.9
> Standard_D4s_v3
>Reporter: Moritz Larndorfer
>Priority: Minor
> Attachments: error.png
>
>
> When updating from 1.25.0. to 1.26.0 a PutAzureDataLakeStorage processor
> started throwing the error shown in the attached image.
> These are the descriptions of the processor and its
> ADLSCredentialsControllerService:
> {
> "identifier": "",
> "instanceIdentifier": "",
> "name": "Store in Datalake",
> "type":
> "org.apache.nifi.processors.azure.storage.PutAzureDataLakeStorage",
> "bundle":
> { "group": "org.apache.nifi", "artifact": "nifi-azure-nar",
> "version": "1.25.0" }
> ,
> "properties": {
> "filesystem-name": "rawdata",
> "conflict-resolution-strategy": "ignore",
> "adls-credentials-service": "",
> "Resource Transfer Source": "FLOWFILE_CONTENT",
> "directory-name": "${path}",
> "file-name": "${filename}",
> "base-temporary-path": ""
> },
> "schedulingPeriod": "0 sec",
> "schedulingStrategy": "TIMER_DRIVEN",
> "executionNode": "ALL",
> "penaltyDuration": "30 sec",
> "yieldDuration": "1 sec",
> "bulletinLevel": "WARN",
> "runDurationMillis": 0,
> "concurrentlySchedulableTaskCount": 1,
> "autoTerminatedRelationships": [],
> "scheduledState": "RUNNING",
> "retryCount": 10,
> "retriedRelationships": [],
> "backoffMechanism": "PENALIZE_FLOWFILE",
> "maxBackoffPeriod": "10 mins",
> "componentType": "PROCESSOR",
> "groupIdentifier": ""
> }
> {
> "identifier": "",
> "instanceIdentifier": "",
> "name": "ADLSCredentialsControllerService FillDatalake",
> "comments": "",
> "type":
> "org.apache.nifi.services.azure.storage.ADLSCredentialsControllerService",
> "bundle":
> { "group": "org.apache.nifi", "artifact": "nifi-azure-nar",
> "version": "1.25.0" }
> ,
> "properties":
> { "storage-account-name": "",
> "service-principal-client-secret": "",
> "storage-use-managed-identity": "false", "storage-endpoint-suffix":
> "dfs.core.windows.net", "service-principal-tenant-id": "",
> "service-principal-client-id": "" }
> ,
> "propertyDescriptors": {},
> "controllerServiceApis": [{
> "type":
> "org.apache.nifi.services.azure.storage.ADLSCredentialsService",
> "bundle":
> { "group": "org.apache.nifi", "artifact":
> "nifi-azure-services-api-nar", "version": "1.25.0"
> }
> }
> ],
> "scheduledState": "ENABLED",
> "bulletinLevel": "WARN",
> "componentType": "CONTROLLER_SERVICE",
> "groupIdentifier": ""
> }
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (NIFI-13399) NiFi Registry login page crash on unicode characters in password
[
https://issues.apache.org/jira/browse/NIFI-13399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sash Sujith reassigned NIFI-13399:
--
Assignee: (was: Sash Sujith)
> NiFi Registry login page crash on unicode characters in password
>
>
> Key: NIFI-13399
> URL: https://issues.apache.org/jira/browse/NIFI-13399
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core UI, NiFi Registry
>Affects Versions: 2.0.0-M3
> Environment: Ubuntu 22.04
> Open JDK 21
> NiFi Registry 2.0.0.M3 + LDAP (Microsoft AD)
>Reporter: Night Gryphon
>Priority: Blocker
>
> Registry login page crash with unicode non latin1 characters in user
> password. While trying to submit such password UI just do nothing. No error
> popup for user while browser Javascript console shows an error:
> {code:java}
> vendor.min.4dc8237ea10851b143e5.js:424 ERROR DOMException: Failed to execute
> 'btoa' on 'Window': The string to be encoded contains characters outside of
> the Latin1 range.
> at A.postToLogin
> (https://registry.nifi.local:8443/nifi-registry/nf-registry.bundle.min.51cc86964e9d4d96daf1.js:1:154149)
> at Y.login
> (https://registry.nifi.local:8443/nifi-registry/nf-registry.bundle.min.51cc86964e9d4d96daf1.js:1:186924)
> at Object.eval [as handleEvent] (ng:///Te/Y.ngfactory.js:133:35)
> at Object.handleEvent
> (https://registry.nifi.local:8443/nifi-registry/vendor.min.4dc8237ea10851b143e5.js:1499:1487)
> at Object.handleEvent
> (https://registry.nifi.local:8443/nifi-registry/vendor.min.4dc8237ea10851b143e5.js:1506:1059)
> at fg
> (https://registry.nifi.local:8443/nifi-registry/vendor.min.4dc8237ea10851b143e5.js:1182:1000)
> at
> https://registry.nifi.local:8443/nifi-registry/vendor.min.4dc8237ea10851b143e5.js:1463:2610
> at
> https://registry.nifi.local:8443/nifi-registry/vendor.min.4dc8237ea10851b143e5.js:5609:594
> at
> https://registry.nifi.local:8443/nifi-registry/vendor.min.4dc8237ea10851b143e5.js:5638:209
> at c.invoke
> (https://registry.nifi.local:8443/nifi-registry/vendor.min.4dc8237ea10851b143e5.js:6537:6401)
> Ls @ vendor.min.4dc8237ea10851b143e5.js:424
> vendor.min.4dc8237ea10851b143e5.js:424 ERROR DOMException: Failed to execute
> 'btoa' on 'Window': The string to be encoded contains characters outside of
> the Latin1 range.
> at A.postToLogin
> (https://registry.nifi.local:8443/nifi-registry/nf-registry.bundle.min.51cc86964e9d4d96daf1.js:1:154149)
> at Y.login
> (https://registry.nifi.local:8443/nifi-registry/nf-registry.bundle.min.51cc86964e9d4d96daf1.js:1:186924)
> at Object.eval [as handleEvent] (ng:///Te/Y.ngfactory.js:168:31)
> at Object.handleEvent
> (https://registry.nifi.local:8443/nifi-registry/vendor.min.4dc8237ea10851b143e5.js:1499:1487)
> at Object.handleEvent
> (https://registry.nifi.local:8443/nifi-registry/vendor.min.4dc8237ea10851b143e5.js:1506:1059)
> at fg
> (https://registry.nifi.local:8443/nifi-registry/vendor.min.4dc8237ea10851b143e5.js:1182:1000)
> at
> https://registry.nifi.local:8443/nifi-registry/vendor.min.4dc8237ea10851b143e5.js:1463:2610
> at HTMLButtonElement.
> (https://registry.nifi.local:8443/nifi-registry/vendor.min.4dc8237ea10851b143e5.js:5609:594)
> at c.invokeTask
> (https://registry.nifi.local:8443/nifi-registry/vendor.min.4dc8237ea10851b143e5.js:6537:7019)
> at Object.onInvokeTask
> (https://registry.nifi.local:8443/nifi-registry/vendor.min.4dc8237ea10851b143e5.js:1377:1589)
> Ls @ vendor.min.4dc8237ea10851b143e5.js:424
> vendor.min.4dc8237ea10851b143e5.js:424 ERROR DOMException: Failed to execute
> 'btoa' on 'Window': The string to be encoded contains characters outside of
> the Latin1 range.
> at A.postToLogin
> (https://registry.nifi.local:8443/nifi-registry/nf-registry.bundle.min.51cc86964e9d4d96daf1.js:1:154149)
> at Y.login
> (https://registry.nifi.local:8443/nifi-registry/nf-registry.bundle.min.51cc86964e9d4d96daf1.js:1:186924)
> at Object.eval [as handleEvent] (ng:///Te/Y.ngfactory.js:133:35)
> at Object.handleEvent
> (https://registry.nifi.local:8443/nifi-registry/vendor.min.4dc8237ea10851b143e5.js:1499:1487)
> at Object.handleEvent
> (https://registry.nifi.local:8443/nifi-registry/vendor.min.4dc8237ea10851b143e5.js:1506:1059)
> at fg
> (https://registry.nifi.local:8443/nifi-registry/vendor.min.4dc8237ea10851b143e5.js:1182:1000)
> at
> https://registry.nifi.local:8443/nifi-registry/vendor.min.4dc8237ea10851b143e5.js:1463:2610
> at
> https://registry.nifi.local:8443/nifi-registry/vendor.min.4dc8237ea10851b143e5.js:5609:594
> at
> https://registry.nifi.local:8443/nifi-registry/vendor.min.4dc8237ea10851b143e5.js:5638:209
> at c.invoke
>
[jira] [Assigned] (NIFI-13399) NiFi Registry login page crash on unicode characters in password
[
https://issues.apache.org/jira/browse/NIFI-13399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sash Sujith reassigned NIFI-13399:
--
Assignee: Sash Sujith
> NiFi Registry login page crash on unicode characters in password
>
>
> Key: NIFI-13399
> URL: https://issues.apache.org/jira/browse/NIFI-13399
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core UI, NiFi Registry
>Affects Versions: 2.0.0-M3
> Environment: Ubuntu 22.04
> Open JDK 21
> NiFi Registry 2.0.0.M3 + LDAP (Microsoft AD)
>Reporter: Night Gryphon
>Assignee: Sash Sujith
>Priority: Blocker
>
> Registry login page crash with unicode non latin1 characters in user
> password. While trying to submit such password UI just do nothing. No error
> popup for user while browser Javascript console shows an error:
> {code:java}
> vendor.min.4dc8237ea10851b143e5.js:424 ERROR DOMException: Failed to execute
> 'btoa' on 'Window': The string to be encoded contains characters outside of
> the Latin1 range.
> at A.postToLogin
> (https://registry.nifi.local:8443/nifi-registry/nf-registry.bundle.min.51cc86964e9d4d96daf1.js:1:154149)
> at Y.login
> (https://registry.nifi.local:8443/nifi-registry/nf-registry.bundle.min.51cc86964e9d4d96daf1.js:1:186924)
> at Object.eval [as handleEvent] (ng:///Te/Y.ngfactory.js:133:35)
> at Object.handleEvent
> (https://registry.nifi.local:8443/nifi-registry/vendor.min.4dc8237ea10851b143e5.js:1499:1487)
> at Object.handleEvent
> (https://registry.nifi.local:8443/nifi-registry/vendor.min.4dc8237ea10851b143e5.js:1506:1059)
> at fg
> (https://registry.nifi.local:8443/nifi-registry/vendor.min.4dc8237ea10851b143e5.js:1182:1000)
> at
> https://registry.nifi.local:8443/nifi-registry/vendor.min.4dc8237ea10851b143e5.js:1463:2610
> at
> https://registry.nifi.local:8443/nifi-registry/vendor.min.4dc8237ea10851b143e5.js:5609:594
> at
> https://registry.nifi.local:8443/nifi-registry/vendor.min.4dc8237ea10851b143e5.js:5638:209
> at c.invoke
> (https://registry.nifi.local:8443/nifi-registry/vendor.min.4dc8237ea10851b143e5.js:6537:6401)
> Ls @ vendor.min.4dc8237ea10851b143e5.js:424
> vendor.min.4dc8237ea10851b143e5.js:424 ERROR DOMException: Failed to execute
> 'btoa' on 'Window': The string to be encoded contains characters outside of
> the Latin1 range.
> at A.postToLogin
> (https://registry.nifi.local:8443/nifi-registry/nf-registry.bundle.min.51cc86964e9d4d96daf1.js:1:154149)
> at Y.login
> (https://registry.nifi.local:8443/nifi-registry/nf-registry.bundle.min.51cc86964e9d4d96daf1.js:1:186924)
> at Object.eval [as handleEvent] (ng:///Te/Y.ngfactory.js:168:31)
> at Object.handleEvent
> (https://registry.nifi.local:8443/nifi-registry/vendor.min.4dc8237ea10851b143e5.js:1499:1487)
> at Object.handleEvent
> (https://registry.nifi.local:8443/nifi-registry/vendor.min.4dc8237ea10851b143e5.js:1506:1059)
> at fg
> (https://registry.nifi.local:8443/nifi-registry/vendor.min.4dc8237ea10851b143e5.js:1182:1000)
> at
> https://registry.nifi.local:8443/nifi-registry/vendor.min.4dc8237ea10851b143e5.js:1463:2610
> at HTMLButtonElement.
> (https://registry.nifi.local:8443/nifi-registry/vendor.min.4dc8237ea10851b143e5.js:5609:594)
> at c.invokeTask
> (https://registry.nifi.local:8443/nifi-registry/vendor.min.4dc8237ea10851b143e5.js:6537:7019)
> at Object.onInvokeTask
> (https://registry.nifi.local:8443/nifi-registry/vendor.min.4dc8237ea10851b143e5.js:1377:1589)
> Ls @ vendor.min.4dc8237ea10851b143e5.js:424
> vendor.min.4dc8237ea10851b143e5.js:424 ERROR DOMException: Failed to execute
> 'btoa' on 'Window': The string to be encoded contains characters outside of
> the Latin1 range.
> at A.postToLogin
> (https://registry.nifi.local:8443/nifi-registry/nf-registry.bundle.min.51cc86964e9d4d96daf1.js:1:154149)
> at Y.login
> (https://registry.nifi.local:8443/nifi-registry/nf-registry.bundle.min.51cc86964e9d4d96daf1.js:1:186924)
> at Object.eval [as handleEvent] (ng:///Te/Y.ngfactory.js:133:35)
> at Object.handleEvent
> (https://registry.nifi.local:8443/nifi-registry/vendor.min.4dc8237ea10851b143e5.js:1499:1487)
> at Object.handleEvent
> (https://registry.nifi.local:8443/nifi-registry/vendor.min.4dc8237ea10851b143e5.js:1506:1059)
> at fg
> (https://registry.nifi.local:8443/nifi-registry/vendor.min.4dc8237ea10851b143e5.js:1182:1000)
> at
> https://registry.nifi.local:8443/nifi-registry/vendor.min.4dc8237ea10851b143e5.js:1463:2610
> at
> https://registry.nifi.local:8443/nifi-registry/vendor.min.4dc8237ea10851b143e5.js:5609:594
> at
> https://registry.nifi.local:8443/nifi-registry/vendor.min.4dc8237ea10851b143e5.js:5638:209
> at c.invoke
>
[jira] [Updated] (NIFI-13459) Update to 1.26.0 breaks some PutAzureDataLakeStorage functionality
[
https://issues.apache.org/jira/browse/NIFI-13459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Moritz Larndorfer updated NIFI-13459:
-
Description:
When updating from 1.25.0. to 1.26.0 a PutAzureDataLakeStorage processor
started throwing the error shown in the attached image.
These are the descriptions of the processor and its
ADLSCredentialsControllerService:
{
"identifier": "",
"instanceIdentifier": "",
"name": "Store in Datalake",
"type": "org.apache.nifi.processors.azure.storage.PutAzureDataLakeStorage",
"bundle":
{ "group": "org.apache.nifi", "artifact": "nifi-azure-nar",
"version": "1.25.0" }
,
"properties": {
"filesystem-name": "rawdata",
"conflict-resolution-strategy": "ignore",
"adls-credentials-service": "",
"Resource Transfer Source": "FLOWFILE_CONTENT",
"directory-name": "${path}",
"file-name": "${filename}",
"base-temporary-path": ""
},
"schedulingPeriod": "0 sec",
"schedulingStrategy": "TIMER_DRIVEN",
"executionNode": "ALL",
"penaltyDuration": "30 sec",
"yieldDuration": "1 sec",
"bulletinLevel": "WARN",
"runDurationMillis": 0,
"concurrentlySchedulableTaskCount": 1,
"autoTerminatedRelationships": [],
"scheduledState": "RUNNING",
"retryCount": 10,
"retriedRelationships": [],
"backoffMechanism": "PENALIZE_FLOWFILE",
"maxBackoffPeriod": "10 mins",
"componentType": "PROCESSOR",
"groupIdentifier": ""
}
{
"identifier": "",
"instanceIdentifier": "",
"name": "ADLSCredentialsControllerService FillDatalake",
"comments": "",
"type":
"org.apache.nifi.services.azure.storage.ADLSCredentialsControllerService",
"bundle":
{ "group": "org.apache.nifi", "artifact": "nifi-azure-nar",
"version": "1.25.0" }
,
"properties":
{ "storage-account-name": "",
"service-principal-client-secret": "", "storage-use-managed-identity":
"false", "storage-endpoint-suffix": "dfs.core.windows.net",
"service-principal-tenant-id": "", "service-principal-client-id": ""
}
,
"propertyDescriptors": {},
"controllerServiceApis": [{
"type":
"org.apache.nifi.services.azure.storage.ADLSCredentialsService",
"bundle":
{ "group": "org.apache.nifi", "artifact":
"nifi-azure-services-api-nar", "version": "1.25.0" }
}
],
"scheduledState": "ENABLED",
"bulletinLevel": "WARN",
"componentType": "CONTROLLER_SERVICE",
"groupIdentifier": ""
}
was:
When updating from 1.25.0. to 1.26.0 a PutAzureDataLakeStorage processor
started throwing errors.
These are the descriptions of the processor and its
ADLSCredentialsControllerService:
{
"identifier": "",
"instanceIdentifier": "",
"name": "Store in Datalake",
"type": "org.apache.nifi.processors.azure.storage.PutAzureDataLakeStorage",
"bundle": {
"group": "org.apache.nifi",
"artifact": "nifi-azure-nar",
"version": "1.25.0"
},
"properties": {
"filesystem-name": "rawdata",
"conflict-resolution-strategy": "ignore",
"adls-credentials-service": "",
"Resource Transfer Source": "FLOWFILE_CONTENT",
"directory-name": "${path}",
"file-name": "${filename}",
"base-temporary-path": ""
},
"schedulingPeriod": "0 sec",
"schedulingStrategy": "TIMER_DRIVEN",
"executionNode": "ALL",
"penaltyDuration": "30 sec",
"yieldDuration": "1 sec",
"bulletinLevel": "WARN",
"runDurationMillis": 0,
"concurrentlySchedulableTaskCount": 1,
"autoTerminatedRelationships": [],
"scheduledState": "RUNNING",
"retryCount": 10,
"retriedRelationships": [],
"backoffMechanism": "PENALIZE_FLOWFILE",
"maxBackoffPeriod": "10 mins",
"componentType": "PROCESSOR",
"groupIdentifier": ""
}
{
"identifier": "",
"instanceIdentifier": "",
"name": "ADLSCredentialsControllerService FillDatalake",
"comments": "",
"type":
"org.apache.nifi.services.azure.storage.ADLSCredentialsControllerService",
"bundle": {
"group": "org.apache.nifi",
"artifact": "nifi-azure-nar",
"version": "1.25.0"
},
"properties": {
"storage-account-name": "",
"service-principal-client-secret": "",
"storage-use-managed-identity": "false",
"storage-endpoint-suffix": "dfs.core.windows.net",
"service-principal-tenant-id": "",
"service-principal-client-id": ""
},
"propertyDescriptors": {},
"controllerServiceApis": [{
"type":
"org.apache.nifi.services.azure.storage.ADLSCredentialsService",
"bundle": {
"group": "org.apache.nifi",
"artifact": "nifi-azure-services-api-nar",
[jira] [Created] (NIFI-13459) Update to 1.26.0 breaks some PutAzureDataLakeStorage functionality
Moritz Larndorfer created NIFI-13459:
Summary: Update to 1.26.0 breaks some PutAzureDataLakeStorage
functionality
Key: NIFI-13459
URL: https://issues.apache.org/jira/browse/NIFI-13459
Project: Apache NiFi
Issue Type: Bug
Components: Extensions
Affects Versions: 1.26.0
Environment: Azure Kubernetes Service 1.28.9
Standard_D4s_v3
Reporter: Moritz Larndorfer
Attachments: error.png
When updating from 1.25.0. to 1.26.0 a PutAzureDataLakeStorage processor
started throwing errors.
These are the descriptions of the processor and its
ADLSCredentialsControllerService:
{
"identifier": "",
"instanceIdentifier": "",
"name": "Store in Datalake",
"type": "org.apache.nifi.processors.azure.storage.PutAzureDataLakeStorage",
"bundle": {
"group": "org.apache.nifi",
"artifact": "nifi-azure-nar",
"version": "1.25.0"
},
"properties": {
"filesystem-name": "rawdata",
"conflict-resolution-strategy": "ignore",
"adls-credentials-service": "",
"Resource Transfer Source": "FLOWFILE_CONTENT",
"directory-name": "${path}",
"file-name": "${filename}",
"base-temporary-path": ""
},
"schedulingPeriod": "0 sec",
"schedulingStrategy": "TIMER_DRIVEN",
"executionNode": "ALL",
"penaltyDuration": "30 sec",
"yieldDuration": "1 sec",
"bulletinLevel": "WARN",
"runDurationMillis": 0,
"concurrentlySchedulableTaskCount": 1,
"autoTerminatedRelationships": [],
"scheduledState": "RUNNING",
"retryCount": 10,
"retriedRelationships": [],
"backoffMechanism": "PENALIZE_FLOWFILE",
"maxBackoffPeriod": "10 mins",
"componentType": "PROCESSOR",
"groupIdentifier": ""
}
{
"identifier": "",
"instanceIdentifier": "",
"name": "ADLSCredentialsControllerService FillDatalake",
"comments": "",
"type":
"org.apache.nifi.services.azure.storage.ADLSCredentialsControllerService",
"bundle": {
"group": "org.apache.nifi",
"artifact": "nifi-azure-nar",
"version": "1.25.0"
},
"properties": {
"storage-account-name": "",
"service-principal-client-secret": "",
"storage-use-managed-identity": "false",
"storage-endpoint-suffix": "dfs.core.windows.net",
"service-principal-tenant-id": "",
"service-principal-client-id": ""
},
"propertyDescriptors": {},
"controllerServiceApis": [{
"type":
"org.apache.nifi.services.azure.storage.ADLSCredentialsService",
"bundle": {
"group": "org.apache.nifi",
"artifact": "nifi-azure-services-api-nar",
"version": "1.25.0"
}
}
],
"scheduledState": "ENABLED",
"bulletinLevel": "WARN",
"componentType": "CONTROLLER_SERVICE",
"groupIdentifier": ""
}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (MINIFICPP-2294) Add config migration step
[ https://issues.apache.org/jira/browse/MINIFICPP-2294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martin Zink reassigned MINIFICPP-2294: -- Assignee: Martin Zink > Add config migration step > - > > Key: MINIFICPP-2294 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2294 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Martin Zink >Assignee: Martin Zink >Priority: Major > > Often there is a comprimise fixing bugs and selecting the best approach due > to the consideration of old config files. > We could add a migration step (with config file versioning), and when we load > an older config file we should do the neccessary update steps to fix the > issues and save the config file with the new version. (Im not yet sure how it > would work with C2 issued configs though) > e.g. > https://issues.apache.org/jira/browse/MINIFICPP-2042 > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-13452) GetSlackReaction processor to fetch reactions
[ https://issues.apache.org/jira/browse/NIFI-13452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17860399#comment-17860399 ] Zsihovszki Krisztina commented on NIFI-13452: - [~joewitt] ConsumeSlack is able to fetch the reactions if "reactions.get" scope is provided, but the goal is not to fetch reactions for all messages in a given channel but really being able to fetch reactions for specific messages that have been consumed by ConsumeSlack or published by PublishSlack. Modifying the ConsumeSlack processor to serve this purpose would make it more complicated in terms of configuration. It seemed like a better choice to have a dedicated processor for fetching the reactions only. > GetSlackReaction processor to fetch reactions > - > > Key: NIFI-13452 > URL: https://issues.apache.org/jira/browse/NIFI-13452 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Zsihovszki Krisztina >Assignee: Zsihovszki Krisztina >Priority: Major > > The goal of this JIRA is to create a new processor so it'd possible to be > fetch the emojis added to a message in Slack. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13372) Add DeleteSFTP processor
[
https://issues.apache.org/jira/browse/NIFI-13372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
endzeit updated NIFI-13372:
---
Priority: Minor (was: Major)
> Add DeleteSFTP processor
>
>
> Key: NIFI-13372
> URL: https://issues.apache.org/jira/browse/NIFI-13372
> Project: Apache NiFi
> Issue Type: New Feature
>Reporter: endzeit
>Assignee: endzeit
>Priority: Minor
> Time Spent: 2h
> Remaining Estimate: 0h
>
> The existing processor to retrieve a file from a remote system over SFTP,
> namely {{FetchSFTP}} and {{{}GetSFTP{}}}, support the removal of the file
> from the file system once the content has been copied into the FlowFile.
> However, deleting the file from the file system immediately might not be
> feasible in certain circumstances.
> In cases where the content repository of NiFi does not meet sufficient data
> durability guarantees, it might be desired to remove the source file only
> after it has been processed successfully and its result transferred to a
> system that satisfies those durability constraints.
> Additionally, the integrated deletion might fail "silently", that is, the
> FlowFile is still sent to the "success" relationship. This results in the
> file remaining at the source. Depending on the listing behaviour it may be
> listed again (in the worst case over and over). Having the deletion as an
> extra step with a failure relationship in case of failure avoids this
> behaviour.
> As of now, there is no built-in solution to achieve such behavior using the
> standard NiFi distribution.
> Current workarounds involve the usage of a scripted processor or the creation
> of a custom processor, that provides the desired functionality.
> This issue proposes the addition of a {{DeleteSFTP}} processor to the NiFi
> standard-processors bundle, that fills this gap.
> It should expect a FlowFile and delete the file at the path derived from the
> FlowFile attributes. The default values to determine the file path should be
> compatible with the attributes written by the existing {{ListSFTP}} processor.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-13372) Add DeleteSFTP processor
[
https://issues.apache.org/jira/browse/NIFI-13372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
endzeit updated NIFI-13372:
---
Description:
The existing processor to retrieve a file from a remote system over SFTP,
namely {{FetchSFTP}} and {{{}GetSFTP{}}}, support the removal of the file from
the file system once the content has been copied into the FlowFile.
However, deleting the file from the file system immediately might not be
feasible in certain circumstances.
In cases where the content repository of NiFi does not meet sufficient data
durability guarantees, it might be desired to remove the source file only after
it has been processed successfully and its result transferred to a system that
satisfies those durability constraints.
Additionally, the integrated deletion might fail "silently", that is, the
FlowFile is still sent to the "success" relationship. This results in the file
remaining at the source. Depending on the listing behaviour it may be listed
again (in the worst case over and over). Having the deletion as an extra step
with a failure relationship in case of failure avoids this behaviour.
As of now, there is no built-in solution to achieve such behavior using the
standard NiFi distribution.
Current workarounds involve the usage of a scripted processor or the creation
of a custom processor, that provides the desired functionality.
This issue proposes the addition of a {{DeleteSFTP}} processor to the NiFi
standard-processors bundle, that fills this gap.
It should expect a FlowFile and delete the file at the path derived from the
FlowFile attributes. The default values to determine the file path should be
compatible with the attributes written by the existing {{ListSFTP}} processor.
was:
The existing processor to retrieve a file from a remote system over SFTP,
namely {{FetchSFTP}} and {{{}GetSFTP{}}}, support the removal of the file from
the file system once the content has been copied into the FlowFile.
However, deleting the file from the file system immediately might not be
feasible in certain circumstances.
In cases where the content repository of NiFi does not meet sufficient data
durability guarantees, it might be desired to remove the source file only after
it has been processed successfully and its result transferred to a system that
satisfies those durability constraints.
As of now, there is no built-in solution to achieve such behavior using the
standard NiFi distribution.
Current workarounds involve the usage of a scripted processor or the creation
of a custom processor, that provides the desired functionality.
This issue proposes the addition of a {{DeleteSFTP}} processor to the NiFi
standard-processors bundle, that fills this gap.
It should expect a FlowFile and delete the file at the path derived from the
FlowFile attributes. The default values to determine the file path should be
compatible with the attributes written by the existing {{ListSFTP}} processor.
> Add DeleteSFTP processor
>
>
> Key: NIFI-13372
> URL: https://issues.apache.org/jira/browse/NIFI-13372
> Project: Apache NiFi
> Issue Type: New Feature
>Reporter: endzeit
>Assignee: endzeit
>Priority: Major
>
> The existing processor to retrieve a file from a remote system over SFTP,
> namely {{FetchSFTP}} and {{{}GetSFTP{}}}, support the removal of the file
> from the file system once the content has been copied into the FlowFile.
> However, deleting the file from the file system immediately might not be
> feasible in certain circumstances.
> In cases where the content repository of NiFi does not meet sufficient data
> durability guarantees, it might be desired to remove the source file only
> after it has been processed successfully and its result transferred to a
> system that satisfies those durability constraints.
> Additionally, the integrated deletion might fail "silently", that is, the
> FlowFile is still sent to the "success" relationship. This results in the
> file remaining at the source. Depending on the listing behaviour it may be
> listed again (in the worst case over and over). Having the deletion as an
> extra step with a failure relationship in case of failure avoids this
> behaviour.
> As of now, there is no built-in solution to achieve such behavior using the
> standard NiFi distribution.
> Current workarounds involve the usage of a scripted processor or the creation
> of a custom processor, that provides the desired functionality.
> This issue proposes the addition of a {{DeleteSFTP}} processor to the NiFi
> standard-processors bundle, that fills this gap.
> It should expect a FlowFile and delete the file at the path derived from the
> FlowFile attributes. The default values to determine the file path should be
> compatible with the attributes written by the existing
