[jira] [Updated] (MINIFICPP-2346) Speed up MiNiFi Build with Conan package manager (C++)
[
https://issues.apache.org/jira/browse/MINIFICPP-2346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Guzman (Medel) updated MINIFICPP-2346:
Description:
*NOTE:* I noticed you guys recently switched from 0.15 to 0.99, so at certain
point, I will pull in lastest updates from MiNiFi C++ main branch on parent
level and rebase or merge into my PR.
*Description of Issue:* Building MiNiFi C++ with just CMake is really slow.
During the the build process, it clones all the external dependencies and then
builds those external dependencies while its building MiNiFi CPP. Sometimes the
MiNiFi CPP build fails too when external lib dependencies fail to download or
fail to build. It would be faster to build MiNiFi C++ with the external
dependencies already preinstalled.
*Potential Solution:* Integrate *conan version 2* into MiNiFi C++ project to
enable support for using *conan install* to install all or most of MiNiFi's
external dependencies, using *conan install --build=missing* to build them as
prebuilt binary conan packages, upload them to conancenter with {*}conan
upload{*}, and then run cmake generate to generate the appropriate build files
for the OS, then run make to build MiNiFi CPP. Also conan has really good
support for cross platform compatibility for Linux and Windows. At this point
because we already have most of MiNiFi CPP's external lib dependencies
installed with conan, our CMake build will just focus on building the MiNiFi
CPP code, enabling faster builds. Also I will need to account for backward
compatibility support with the previous way of installing MiNiFi CPP external
lib dependencies using CMake FetchContent_Declare(...) approach until
conanfile.py approach covers installing all these dependencies. A valid
verification of the MiNiFi C++ build would be to run GTESTs and if all the
expected test cases pass, then it may be fair to conclude we would be ready to
deprecate the slow CMake FetchContent_Declare(...) approach.
*Steps to Perform MiNiFi C++ Build Enhancement (Plan to Integrate into
script(s)):*
# Install conan version 2 into MiNiFi C++ dev environment (my preference is
using docker container)
# Create a conanfile.py file at root of MiNiFi C++ project where we run "conan
install" or "conan install --build=missing" to install MiNiFi C++ external lib
dependencies first. Then we run CMake to build MiNiFi C++.
# Find all MiNiFi C++ find_package(...) CMake function calls and check if
there is already supported conan packages on the conancenter, so we can install
those packages.
# Find all FetchContent_Declare(...) CMake function calls and check if there
is already supported conan packages on the conancenter, so we can install those
packages.
# At this point we will have used conan version 2 C++ package manager to
install almost all MiNiFi C++'s external dependencies.
## With Conan, we will have installed these MiNiFi external lib dependencies
as prebuilt binary conan packages.
## Or we will have instructed conan to first build each C++ external lib
dependency, upload each lib to conancenter.
# Thus, conan manages installing MiNiFi C++'s external lib dependencies, so we
can then run CMake to build MiNiFi faster.
Here is my draft PR for this Jira Ticket for your reference:
[https://github.com/apache/nifi-minifi-cpp/pull/1775]
Here are the commands that I run:
{code:java}
cd $HOME/src/pipeline/nifi-minifi-cpp
# NOTE: I also plan to add smoother integration of conan commands into MiNiFI
CPP too
# For instance, I know you guys have your bootstrap scripts too
# You also have your github actions workflow scripts too
conan install . --output-folder=build
-pr=$HOME/src/nifi-minifi-cpp/etc/build/conan/profiles/release-linux
# install conan packages and build C++ for minifi using Conan & CMake, we run
this way, so conan's tc.cache_variables
# are like passing -D{minifi_option} upon running cmake generate, then make
to build MiNiFi C++
conan build . --output-folder=build
-pr=/home/bizon/src/jam-repo/main/etc/build/conan/profiles/release-linux{code}
*UPDATE(May 14, 2024):* Updated due date to May 19, 2024 since the refactoring
of MiNiFi CPP build infra over to conan version 2 is taking longer than
expected. The good news is that I believe I have more than half of MiNiFi
external libraries now managed by conan with backward compatibility for
standalone CMake approach.
*UPDATE (May 13, 2024)* -
*[22205c9|https://github.com/apache/nifi-minifi-cpp/pull/1775/commits/22205c98cef30ee1ebe252880ae113724589bd78]:*
Fetch and Pulled latest upstream main branch of parent MiNiFi to my fork main
branch of MiNiFi repo; thus, merged in the latest updates from main branch of
parent MiNiFi to my MINIFICPP-2346 branch. Resolved conflicts between a few
CMake files coming from Date.cmake and BundledRocksdb.cmake files. For
standalone CMake build appraoch,
[jira] [Updated] (MINIFICPP-2346) Speed up MiNiFi Build with Conan package manager (C++)
[
https://issues.apache.org/jira/browse/MINIFICPP-2346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Guzman (Medel) updated MINIFICPP-2346:
Description:
*NOTE:* I noticed you guys recently switched from 0.15 to 0.99, so at certain
point, I will pull in lastest updates from MiNiFi C++ main branch on parent
level and rebase or merge into my PR.
*Description of Issue:* Building MiNiFi C++ with just CMake is really slow.
During the the build process, it clones all the external dependencies and then
builds those external dependencies while its building MiNiFi CPP. Sometimes the
MiNiFi CPP build fails too when external lib dependencies fail to download or
fail to build. It would be faster to build MiNiFi C++ with the external
dependencies already preinstalled.
*Potential Solution:* Integrate *conan version 2* into MiNiFi C++ project to
enable support for using *conan install* to install all or most of MiNiFi's
external dependencies, using *conan install --build=missing* to build them as
prebuilt binary conan packages, upload them to conancenter with {*}conan
upload{*}, and then run cmake generate to generate the appropriate build files
for the OS, then run make to build MiNiFi CPP. Also conan has really good
support for cross platform compatibility for Linux and Windows. At this point
because we already have most of MiNiFi CPP's external lib dependencies
installed with conan, our CMake build will just focus on building the MiNiFi
CPP code, enabling faster builds. Also I will need to account for backward
compatibility support with the previous way of installing MiNiFi CPP external
lib dependencies using CMake FetchContent_Declare(...) approach until
conanfile.py approach covers installing all these dependencies. A valid
verification of the MiNiFi C++ build would be to run GTESTs and if all the
expected test cases pass, then it may be fair to conclude we would be ready to
deprecate the slow CMake FetchContent_Declare(...) approach.
*Steps to Perform MiNiFi C++ Build Enhancement (Plan to Integrate into
script(s)):*
# Install conan version 2 into MiNiFi C++ dev environment (my preference is
using docker container)
# Create a conanfile.py file at root of MiNiFi C++ project where we run "conan
install" or "conan install --build=missing" to install MiNiFi C++ external lib
dependencies first. Then we run CMake to build MiNiFi C++.
# Find all MiNiFi C++ find_package(...) CMake function calls and check if
there is already supported conan packages on the conancenter, so we can install
those packages.
# Find all FetchContent_Declare(...) CMake function calls and check if there
is already supported conan packages on the conancenter, so we can install those
packages.
# At this point we will have used conan version 2 C++ package manager to
install almost all MiNiFi C++'s external dependencies.
## With Conan, we will have installed these MiNiFi external lib dependencies
as prebuilt binary conan packages.
## Or we will have instructed conan to first build each C++ external lib
dependency, upload each lib to conancenter.
# Thus, conan manages installing MiNiFi C++'s external lib dependencies, so we
can then run CMake to build MiNiFi faster.
Here is my draft PR for this Jira Ticket for your reference:
[https://github.com/apache/nifi-minifi-cpp/pull/1775]
Here are the commands that I run:
{code:java}
cd $HOME/src/pipeline/nifi-minifi-cpp
# NOTE: I also plan to add smoother integration of conan commands into MiNiFI
CPP too
# For instance, I know you guys have your bootstrap scripts too
# You also have your github actions workflow scripts too
conan install . --output-folder=build
-pr=$HOME/src/nifi-minifi-cpp/etc/build/conan/profiles/release-linux
# install conan packages and build C++ for minifi using Conan & CMake, we run
this way, so conan's tc.cache_variables
# are like passing -D{minifi_option} upon running cmake generate, then make
to build MiNiFi C++
conan build . --output-folder=build
-pr=/home/bizon/src/jam-repo/main/etc/build/conan/profiles/release-linux{code}
*UPDATE (May 13, 2024)* -
[59ffefb|https://github.com/apache/nifi-minifi-cpp/pull/1775/commits/59ffefb1caf662874054f55f7171692c6b9d345f]:
{*}Enabled LIBRDKAFKA & GRPC_FOR_LOKI MiNiFi C++ extension using Conan &
CMake{*}, *Improved Running Python Maven Executable for JNI & SFTP Test Server
for conan approach* and {*}Improved standalone CMake approach for building
PcapPlusPlus with its dependency on LibPcap{*}. LIBRDKAFKA required creating a
librdkafka conan package passing similar CMake definitions to it to build it
like we did for standalone CMake approach. GRPC_FOR_LOKI required creating a
grpc conan package that uses minifi's openssl and libsystemd conan packages. In
both LIBRDKAFKA & GRPC_FOR_LOKI, I updated their .cmake Bundles to account for
the condition of using their conan package,
[jira] [Updated] (MINIFICPP-2346) Speed up MiNiFi Build with Conan package manager (C++)
[
https://issues.apache.org/jira/browse/MINIFICPP-2346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Guzman (Medel) updated MINIFICPP-2346:
Due Date: 19/May/24 (was: 12/May/24)
> Speed up MiNiFi Build with Conan package manager (C++)
> --
>
> Key: MINIFICPP-2346
> URL: https://issues.apache.org/jira/browse/MINIFICPP-2346
> Project: Apache NiFi MiNiFi C++
> Issue Type: Improvement
>Affects Versions: 0.15.0
> Environment: Ubuntu 22.04
>Reporter: James Guzman (Medel)
>Assignee: James Guzman (Medel)
>Priority: Minor
> Labels: build, conan, ubuntu
> Attachments: minifi_cpp_cmake_standalone_build_7eb8beb.log,
> minifi_cpp_conan_build_2d1b884.log, minifi_cpp_conan_build_7eb8beb.log
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> *NOTE:* I noticed you guys recently switched from 0.15 to 0.99, so at certain
> point, I will pull in lastest updates from MiNiFi C++ main branch on parent
> level and rebase or merge into my PR.
> *Description of Issue:* Building MiNiFi C++ with just CMake is really slow.
> During the the build process, it clones all the external dependencies and
> then builds those external dependencies while its building MiNiFi CPP.
> Sometimes the MiNiFi CPP build fails too when external lib dependencies fail
> to download or fail to build. It would be faster to build MiNiFi C++ with the
> external dependencies already preinstalled.
>
> *Potential Solution:* Integrate *conan version 2* into MiNiFi C++ project to
> enable support for using *conan install* to install all or most of MiNiFi's
> external dependencies, using *conan install --build=missing* to build them as
> prebuilt binary conan packages, upload them to conancenter with {*}conan
> upload{*}, and then run cmake generate to generate the appropriate build
> files for the OS, then run make to build MiNiFi CPP. Also conan has really
> good support for cross platform compatibility for Linux and Windows. At this
> point because we already have most of MiNiFi CPP's external lib dependencies
> installed with conan, our CMake build will just focus on building the MiNiFi
> CPP code, enabling faster builds. Also I will need to account for backward
> compatibility support with the previous way of installing MiNiFi CPP external
> lib dependencies using CMake FetchContent_Declare(...) approach until
> conanfile.py approach covers installing all these dependencies. A valid
> verification of the MiNiFi C++ build would be to run GTESTs and if all the
> expected test cases pass, then it may be fair to conclude we would be ready
> to deprecate the slow CMake FetchContent_Declare(...) approach.
>
> *Steps to Perform MiNiFi C++ Build Enhancement (Plan to Integrate into
> script(s)):*
> # Install conan version 2 into MiNiFi C++ dev environment (my preference is
> using docker container)
> # Create a conanfile.py file at root of MiNiFi C++ project where we run
> "conan install" or "conan install --build=missing" to install MiNiFi C++
> external lib dependencies first. Then we run CMake to build MiNiFi C++.
> # Find all MiNiFi C++ find_package(...) CMake function calls and check if
> there is already supported conan packages on the conancenter, so we can
> install those packages.
> # Find all FetchContent_Declare(...) CMake function calls and check if there
> is already supported conan packages on the conancenter, so we can install
> those packages.
> # At this point we will have used conan version 2 C++ package manager to
> install almost all MiNiFi C++'s external dependencies.
> ## With Conan, we will have installed these MiNiFi external lib dependencies
> as prebuilt binary conan packages.
> ## Or we will have instructed conan to first build each C++ external lib
> dependency, upload each lib to conancenter.
> # Thus, conan manages installing MiNiFi C++'s external lib dependencies, so
> we can then run CMake to build MiNiFi faster.
>
> Here is my draft PR for this Jira Ticket for your reference:
> [https://github.com/apache/nifi-minifi-cpp/pull/1775]
> Here are the commands that I run:
>
> {code:java}
> cd $HOME/src/pipeline/nifi-minifi-cpp
> # NOTE: I also plan to add smoother integration of conan commands into MiNiFI
> CPP too
> # For instance, I know you guys have your bootstrap scripts too
> # You also have your github actions workflow scripts too
> conan install . --output-folder=build
> -pr=$HOME/src/nifi-minifi-cpp/etc/build/conan/profiles/release-linux
> # install conan packages and build C++ for minifi using Conan & CMake, we run
> this way, so conan's tc.cache_variables
> # are like passing -D{minifi_option} upon running cmake generate, then
> make to build MiNiFi C++
> conan build . --output-folder=build
>
Re: [PR] [NIFI-13207] page headers and refresh containers are consistently pos… [nifi]
scottyaslan commented on PR #8804: URL: https://github.com/apache/nifi/pull/8804#issuecomment-2111436636 Update: All page headers should have 1.25rem padding from the navigation toolbar at the top of each page and they should have 1.25rem padding below: https://github.com/apache/nifi/assets/6797571/e3c79985-f9cf-4a86-95fa-e7f96f93a1f2;> Any text above the filter form fields should have 0.75rem padding below: https://github.com/apache/nifi/assets/6797571/279de58a-02ea-47f8-8f8c-f327b9c2543d;> Any hint text should have 0.25rem padding above and 0.5rem padding below: https://github.com/apache/nifi/assets/6797571/a9844a4e-971b-438a-bf62-acdf48fb327b;> https://github.com/apache/nifi/assets/6797571/f9b909a5-818f-49cb-8d80-390661b70a90;> For the footer (last updated etc) it should have 0.5rem padding above and 1.25rem padding below: https://github.com/apache/nifi/assets/6797571/8a9153d9-ea6c-4dbf-886b-d772e205f922;> https://github.com/apache/nifi/assets/6797571/85bba52f-5115-4cde-b620-704bf566b218;> Tabs (not in dialogs) should have 1.25rem padding below: https://github.com/apache/nifi/assets/6797571/d08ebfc8-ad5c-4107-8ff9-8ca82f18c016;> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [NIFI-13207] page headers and refresh containers are consistently pos… [nifi]
scottyaslan commented on code in PR #8804: URL: https://github.com/apache/nifi/pull/8804#discussion_r1600842421 ## nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-frontend/src/main/nifi/src/app/pages/provenance/ui/provenance-event-listing/provenance-event-table/provenance-event-table.component.html: ## @@ -17,7 +17,7 @@ - + Review Comment: Ok, I have updated the layouts to display filter hint text below the form fields. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-13241) [BUG] Escape button does not close extension creation dialogs
[ https://issues.apache.org/jira/browse/NIFI-13241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Aslan updated NIFI-13241: --- Description: This issue affects all creation dialogs that set focus to an input. > [BUG] Escape button does not close extension creation dialogs > - > > Key: NIFI-13241 > URL: https://issues.apache.org/jira/browse/NIFI-13241 > Project: Apache NiFi > Issue Type: Sub-task >Reporter: Scott Aslan >Priority: Major > > This issue affects all creation dialogs that set focus to an input. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-13241) [BUG] Escape button does not close extension creation dialogs
Scott Aslan created NIFI-13241: -- Summary: [BUG] Escape button does not close extension creation dialogs Key: NIFI-13241 URL: https://issues.apache.org/jira/browse/NIFI-13241 Project: Apache NiFi Issue Type: Sub-task Reporter: Scott Aslan -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12595 Introduce 'Track Last Listing Time' mode to ListedEntityTracker [nifi]
github-actions[bot] commented on PR #8244: URL: https://github.com/apache/nifi/pull/8244#issuecomment-2111361400 Automated review is marking this PR as stale due to lack of updates in the past four months. This PR will be closed in 15 days if the stale label is not removed. This stale label and automated closure does not indicate a judgement of the PR, just lack of reviewer bandwidth and helps us keep the PR queue more manageable. If you would like this PR re-opened you can do so and a committer can remove the stale label. Or you can open a new PR. Try to help review other PRs to increase PR review bandwidth which in turn helps yours. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-13197) Clicking outside of dialog cancels edit
[ https://issues.apache.org/jira/browse/NIFI-13197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt updated NIFI-13197: Fix Version/s: 2.0.0-M4 > Clicking outside of dialog cancels edit > --- > > Key: NIFI-13197 > URL: https://issues.apache.org/jira/browse/NIFI-13197 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Matt Gilman >Assignee: Rob Fellows >Priority: Major > Fix For: 2.0.0-M4 > > Time Spent: 50m > Remaining Estimate: 0h > > Default behavior of dialog is to support closing when the user clicks off. > Incidental clicks could result in the loss of unsaved changes. We should > consider > * Always preventing closing > * Only prevent closing for Edit dialogs > * Only prevent closing when dialogs contain a dirty form -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13197) Clicking outside of dialog cancels edit
[ https://issues.apache.org/jira/browse/NIFI-13197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Aslan updated NIFI-13197: --- Resolution: Fixed Status: Resolved (was: Patch Available) > Clicking outside of dialog cancels edit > --- > > Key: NIFI-13197 > URL: https://issues.apache.org/jira/browse/NIFI-13197 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Matt Gilman >Assignee: Rob Fellows >Priority: Major > Time Spent: 50m > Remaining Estimate: 0h > > Default behavior of dialog is to support closing when the user clicks off. > Incidental clicks could result in the loss of unsaved changes. We should > consider > * Always preventing closing > * Only prevent closing for Edit dialogs > * Only prevent closing when dialogs contain a dirty form -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-13197) Clicking outside of dialog cancels edit
[ https://issues.apache.org/jira/browse/NIFI-13197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17846456#comment-17846456 ] ASF subversion and git services commented on NIFI-13197: Commit 63a12b06b5f9dddabe5a44874194c745e067bcb5 in nifi's branch refs/heads/main from Rob Fellows [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=63a12b06b5 ] [NIFI-13197] - Clicking outside of a dialog no longer closes the dialog (#8811) * [NIFI-13197] - Clicking outside of a dialog no longer closes the dialog. Also, escape will only close the dialog if the underlying form is not dirty. * add license header * allow ESC to close even if the for is dirty This closes #8811 > Clicking outside of dialog cancels edit > --- > > Key: NIFI-13197 > URL: https://issues.apache.org/jira/browse/NIFI-13197 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Matt Gilman >Assignee: Rob Fellows >Priority: Major > Time Spent: 50m > Remaining Estimate: 0h > > Default behavior of dialog is to support closing when the user clicks off. > Incidental clicks could result in the loss of unsaved changes. We should > consider > * Always preventing closing > * Only prevent closing for Edit dialogs > * Only prevent closing when dialogs contain a dirty form -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] [NIFI-13197] - Clicking outside of a dialog no longer closes the dialog [nifi]
scottyaslan merged PR #8811: URL: https://github.com/apache/nifi/pull/8811 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [NIFI-13197] - Clicking outside of a dialog no longer closes the dialog [nifi]
scottyaslan commented on code in PR #8811:
URL: https://github.com/apache/nifi/pull/8811#discussion_r1600724133
##
nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-frontend/src/main/nifi/src/app/ui/common/close-on-escape-dialog/close-on-escape-dialog.component.ts:
##
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import { Component, inject } from '@angular/core';
+import { CommonModule } from '@angular/common';
+import { filter } from 'rxjs';
+import { MatDialogRef } from '@angular/material/dialog';
+import { takeUntilDestroyed } from '@angular/core/rxjs-interop';
+
+@Component({
+selector: 'close-on-escape-dialog',
+standalone: true,
+imports: [CommonModule],
+template: ''
+})
+export abstract class CloseOnEscapeDialog {
+private dialogRef: MatDialogRef =
inject(MatDialogRef);
+
+protected constructor() {
+if (this.dialogRef) {
+this.dialogRef
+.keydownEvents()
+.pipe(
+filter((event: KeyboardEvent) => event.key === 'Escape'),
+takeUntilDestroyed()
+)
+.subscribe(() => {
+if (!this.isDirty()) {
+this.dialogRef.close();
+}
+});
+}
Review Comment:
Let's just update the UX to match legacy nifi.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [NIFI-13082] Created SplitPcap processor, Pcap supporting class, and … [nifi]
JackHintonSmartDCSIT commented on code in PR #8691:
URL: https://github.com/apache/nifi/pull/8691#discussion_r1600715641
##
nifi-extension-bundles/nifi-network-bundle/nifi-network-processors/src/main/java/org/apache/nifi/processors/network/SplitPCAP.java:
##
@@ -0,0 +1,203 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.network;
+
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.processors.network.util.PCAP;
+import org.apache.nifi.processors.network.util.PCAP.ByteBufferInterface;
+import org.apache.nifi.processors.network.util.PCAP.Packet;
+
+import java.io.ByteArrayOutputStream;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Set;
+
+@Tags({"PCAP", "Splitter", "Network", "Packet", "Capture", "Wireshark",
"TShark"})
+@CapabilityDescription("Splits a pcap file into multiple pcap files based on a
maximum size.")
+@WritesAttribute(attribute = "ERROR_REASON", description = "The reason the
flowfile was sent to the failure relationship.")
+
+public class SplitPCAP extends AbstractProcessor {
+
Review Comment:
I've changed it to `protected` and that seems to work.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Created] (NIFI-13240) Create Marker Interface for AbstractSingleAttributeControllerService
Jim Steinebrey created NIFI-13240: - Summary: Create Marker Interface for AbstractSingleAttributeControllerService Key: NIFI-13240 URL: https://issues.apache.org/jira/browse/NIFI-13240 Project: Apache NiFi Issue Type: Improvement Reporter: Jim Steinebrey -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-8134) DataTypeUtils.toRecord methods do not recursively convert Maps into Records
[ https://issues.apache.org/jira/browse/NIFI-8134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17846449#comment-17846449 ] ASF subversion and git services commented on NIFI-8134: --- Commit 2d112871db40c8f599d31974d14dd4a2227bf7da in nifi's branch refs/heads/main from Chris Sampson [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=2d112871db ] NIFI-8134 allow unescapeJson Record Path function to recursively convert Maps to Records (#7745) * NIFI-8134 recursively convert Java Objects to NiFi Records > DataTypeUtils.toRecord methods do not recursively convert Maps into Records > --- > > Key: NIFI-8134 > URL: https://issues.apache.org/jira/browse/NIFI-8134 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.11.4 >Reporter: Chris Sampson >Assignee: Chris Sampson >Priority: Major > Time Spent: 3h > Remaining Estimate: 0h > > Given a java Map that contains one or more Maps as values (optionally nested > within arrays), the DataTypeUtils.toRecord method should convert the child > Maps to Records before converting the to level Map. > This assumes the associated schema for the data represents these objects as > Records (including as part of a Choice of Array type). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-8134) DataTypeUtils.toRecord methods do not recursively convert Maps into Records
[ https://issues.apache.org/jira/browse/NIFI-8134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-8134: - Fix Version/s: 2.0.0-M4 Resolution: Fixed Status: Resolved (was: Patch Available) > DataTypeUtils.toRecord methods do not recursively convert Maps into Records > --- > > Key: NIFI-8134 > URL: https://issues.apache.org/jira/browse/NIFI-8134 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.11.4 >Reporter: Chris Sampson >Assignee: Chris Sampson >Priority: Major > Fix For: 2.0.0-M4 > > Time Spent: 3h > Remaining Estimate: 0h > > Given a java Map that contains one or more Maps as values (optionally nested > within arrays), the DataTypeUtils.toRecord method should convert the child > Maps to Records before converting the to level Map. > This assumes the associated schema for the data represents these objects as > Records (including as part of a Choice of Array type). -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-8134 allow unescapeJson Record Path function to recursively convert Maps to Records [nifi]
markap14 merged PR #7745: URL: https://github.com/apache/nifi/pull/7745 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-8134 allow unescapeJson Record Path function to recursively convert Maps to Records [nifi]
markap14 commented on PR #7745: URL: https://github.com/apache/nifi/pull/7745#issuecomment-249151 Thanks for the update @ChrisSamo632 all looks good to me at this point. Sorry this has taken so long, and thanks for sticking with it! +1 will merge to main -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (MINIFICPP-2373) ConsumeJournald is broken with 'Process Old Messages'=false
Marton Szasz created MINIFICPP-2373:
---
Summary: ConsumeJournald is broken with 'Process Old
Messages'=false
Key: MINIFICPP-2373
URL: https://issues.apache.org/jira/browse/MINIFICPP-2373
Project: Apache NiFi MiNiFi C++
Issue Type: Bug
Reporter: Marton Szasz
{{[2024-05-14 22:58:24.171] [org::apache::nifi::minifi::SchedulingAgent]
[warning] Caught Exception during SchedulingAgent::onTrigger of processor
ConsumeJournald (uuid: fa39ba8a-6ea8-4c4d-90bd-b8da329f0f6e), type:
N3org6apache4nifi6minifi20SystemErrorExceptionE, what: sd_journal_get_cursor:
Cannot assign requested address}}
Maybe libsystemd broke API?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] NIFI-12896: Add Endpoint Override URL property for PutSNS processor [nifi]
exceptionfactory commented on code in PR #8529:
URL: https://github.com/apache/nifi/pull/8529#discussion_r1600636094
##
nifi-extension-bundles/nifi-aws-bundle/nifi-aws-processors/src/test/java/org/apache/nifi/processors/aws/sns/ITPutSNS.java:
##
@@ -0,0 +1,119 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.aws.sns;
+
+import org.apache.nifi.processor.Processor;
+import org.apache.nifi.processors.aws.testutil.AuthUtils;
+import org.apache.nifi.processors.aws.v2.AbstractAwsProcessor;
+import org.apache.nifi.util.TestRunner;
+import org.apache.nifi.util.TestRunners;
+import org.junit.jupiter.api.AfterAll;
+import org.junit.jupiter.api.BeforeAll;
+import org.junit.jupiter.api.Test;
+import org.testcontainers.containers.localstack.LocalStackContainer;
+import org.testcontainers.utility.DockerImageName;
+import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
+import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider;
+import software.amazon.awssdk.regions.Region;
+import software.amazon.awssdk.services.sns.SnsClient;
+import software.amazon.awssdk.services.sns.model.CreateTopicRequest;
+import software.amazon.awssdk.services.sns.model.CreateTopicResponse;
+
+import java.io.IOException;
+import java.nio.file.Paths;
+import java.util.HashMap;
+import java.util.Map;
+
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+/**
+ * Provides integration level testing with actual AWS S3 resources for {@link
PutSNS} and requires additional configuration and resources to work.
+ */
+public class ITPutSNS {
+
+private static final DockerImageName localstackImage =
DockerImageName.parse("localstack/localstack:latest");
+
+private static final LocalStackContainer localstack = new
LocalStackContainer(localstackImage)
+.withServices(LocalStackContainer.Service.SNS);
+
+private static final String CREDENTIALS_FILE =
"src/test/resources/mock-aws-credentials.properties";
+private static String topicARN;
+private static SnsClient client;
+
+@BeforeAll
+public static void setup() throws InterruptedException {
+System.setProperty("software.amazon.awssdk.http.service.impl",
"software.amazon.awssdk.http.urlconnection.UrlConnectionSdkHttpService");
Review Comment:
Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] NIFI-12896: Add Endpoint Override URL property for PutSNS processor [nifi]
sujkm commented on code in PR #8529:
URL: https://github.com/apache/nifi/pull/8529#discussion_r1600620568
##
nifi-extension-bundles/nifi-aws-bundle/nifi-aws-processors/src/test/java/org/apache/nifi/processors/aws/sns/ITPutSNS.java:
##
@@ -0,0 +1,119 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.aws.sns;
+
+import org.apache.nifi.processor.Processor;
+import org.apache.nifi.processors.aws.testutil.AuthUtils;
+import org.apache.nifi.processors.aws.v2.AbstractAwsProcessor;
+import org.apache.nifi.util.TestRunner;
+import org.apache.nifi.util.TestRunners;
+import org.junit.jupiter.api.AfterAll;
+import org.junit.jupiter.api.BeforeAll;
+import org.junit.jupiter.api.Test;
+import org.testcontainers.containers.localstack.LocalStackContainer;
+import org.testcontainers.utility.DockerImageName;
+import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
+import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider;
+import software.amazon.awssdk.regions.Region;
+import software.amazon.awssdk.services.sns.SnsClient;
+import software.amazon.awssdk.services.sns.model.CreateTopicRequest;
+import software.amazon.awssdk.services.sns.model.CreateTopicResponse;
+
+import java.io.IOException;
+import java.nio.file.Paths;
+import java.util.HashMap;
+import java.util.Map;
+
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+/**
+ * Provides integration level testing with actual AWS S3 resources for {@link
PutSNS} and requires additional configuration and resources to work.
+ */
+public class ITPutSNS {
+
+private static final DockerImageName localstackImage =
DockerImageName.parse("localstack/localstack:latest");
+
+private static final LocalStackContainer localstack = new
LocalStackContainer(localstackImage)
+.withServices(LocalStackContainer.Service.SNS);
+
+private static final String CREDENTIALS_FILE =
"src/test/resources/mock-aws-credentials.properties";
+private static String topicARN;
+private static SnsClient client;
+
+@BeforeAll
+public static void setup() throws InterruptedException {
+System.setProperty("software.amazon.awssdk.http.service.impl",
"software.amazon.awssdk.http.urlconnection.UrlConnectionSdkHttpService");
Review Comment:
It was not required it has been removed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Created] (NIFI-13239) Prevent routing away from a dirty form without prompting the user
Rob Fellows created NIFI-13239: -- Summary: Prevent routing away from a dirty form without prompting the user Key: NIFI-13239 URL: https://issues.apache.org/jira/browse/NIFI-13239 Project: Apache NiFi Issue Type: Sub-task Components: Core UI Reporter: Rob Fellows If users make changes to a configuration component and hit the browser back button or refresh button, they aren't prompted about saving unfinished changes. This is a potential for data loss that might be solvable with canDeactivate guards. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12896: Add Endpoint Override URL property for PutSNS processor [nifi]
exceptionfactory commented on code in PR #8529:
URL: https://github.com/apache/nifi/pull/8529#discussion_r1600591314
##
nifi-extension-bundles/nifi-aws-bundle/nifi-aws-processors/src/test/java/org/apache/nifi/processors/aws/sns/ITPutSNS.java:
##
@@ -0,0 +1,119 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.aws.sns;
+
+import org.apache.nifi.processor.Processor;
+import org.apache.nifi.processors.aws.testutil.AuthUtils;
+import org.apache.nifi.processors.aws.v2.AbstractAwsProcessor;
+import org.apache.nifi.util.TestRunner;
+import org.apache.nifi.util.TestRunners;
+import org.junit.jupiter.api.AfterAll;
+import org.junit.jupiter.api.BeforeAll;
+import org.junit.jupiter.api.Test;
+import org.testcontainers.containers.localstack.LocalStackContainer;
+import org.testcontainers.utility.DockerImageName;
+import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
+import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider;
+import software.amazon.awssdk.regions.Region;
+import software.amazon.awssdk.services.sns.SnsClient;
+import software.amazon.awssdk.services.sns.model.CreateTopicRequest;
+import software.amazon.awssdk.services.sns.model.CreateTopicResponse;
+
+import java.io.IOException;
+import java.nio.file.Paths;
+import java.util.HashMap;
+import java.util.Map;
+
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+/**
+ * Provides integration level testing with actual AWS S3 resources for {@link
PutSNS} and requires additional configuration and resources to work.
+ */
+public class ITPutSNS {
+
+private static final DockerImageName localstackImage =
DockerImageName.parse("localstack/localstack:latest");
+
+private static final LocalStackContainer localstack = new
LocalStackContainer(localstackImage)
+.withServices(LocalStackContainer.Service.SNS);
+
+private static final String CREDENTIALS_FILE =
"src/test/resources/mock-aws-credentials.properties";
+private static String topicARN;
+private static SnsClient client;
+
+@BeforeAll
+public static void setup() throws InterruptedException {
+System.setProperty("software.amazon.awssdk.http.service.impl",
"software.amazon.awssdk.http.urlconnection.UrlConnectionSdkHttpService");
Review Comment:
Is this system property required? If it is needed, this property should be
removed in the `@AfterAll` method.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] NIFI-12411: Update PublishAMQP to read AMQP headers value from FlowFile attributes and `amq$headers` string [nifi]
exceptionfactory commented on PR #8105: URL: https://github.com/apache/nifi/pull/8105#issuecomment-2111054120 > @exceptionfactory @joewitt > > I have added one more change, which will allow users to set amqp processor's password via env variables. I think this will be helpful in cases where we use AMQP processors in multiple NiFI instances. Instead of copying and pasting different passwords in processor properties, we can reference the env variable via its name. Let me know your thoughts on it. Thanks. Thanks for calling out this particular change @umarhussain15. Given the scope of the current pull request, I recommend considering that change separately. In general, sensitive properties such as passwords do not support reading from environment variables. The reason is that environment variables are also accessible through Expression Language, so they are not necessarily secure. The use case is still supported through Parameter Contexts and Parameter Providers, where the Environment Variable Parameter Provider can be used. This is an additional layer of indirection, but it follows the general convention for other sensitive properties. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-13211) Decouple UIs from old NiFi UI
[ https://issues.apache.org/jira/browse/NIFI-13211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rob Fellows updated NIFI-13211: --- Fix Version/s: 2.0.0-M4 Resolution: Fixed Status: Resolved (was: Patch Available) > Decouple UIs from old NiFi UI > - > > Key: NIFI-13211 > URL: https://issues.apache.org/jira/browse/NIFI-13211 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Major > Fix For: 2.0.0-M4 > > Time Spent: 40m > Remaining Estimate: 0h > > NiFi is comprised of numerous UIs that all get deployed at start up. These > UIs piggybacked on some shared stylesheets, components, and dependencies by > directly loading them through a relative path. In preparation for moving away > from the existing UI, these UIs are being updated to bundle these items > directly. In time, these UIs will all be modernized as well so this is just a > transitional step in that process. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-13211) Decouple UIs from old NiFi UI
[ https://issues.apache.org/jira/browse/NIFI-13211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17846411#comment-17846411 ] ASF subversion and git services commented on NIFI-13211: Commit 3a786cee23313ae365845fc796542469e035e48a in nifi's branch refs/heads/main from Matt Gilman [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=3a786cee23 ] NIFI-13211: Decoupling other UIs from exiting NiFi UI (#8828) * NIFI-13211: - Decoupling the content viewer from the NiFi UI which is currently pulling in shared dependencies, scripts, styles, and images. * NIFI-13211: - Decoupling nifi documentation from the NiFi UI which is currently pulling in shared dependencies, scripts, styles, and images. * NIFI-13211: - Decoupling nifi image viewer from the NiFi UI which is currently pulling in shared dependencies, scripts, styles, and images. * NIFI-13211: - Decoupling nifi standard content viewer from the NiFi UI which is currently pulling in shared dependencies, scripts, styles, and images. * NIFI-13211: - Decoupling nifi jolt transform json from the NiFi UI which is currently pulling in shared dependencies, scripts, styles, and images. * NIFI-13211: - Decoupling nifi update attribute from the NiFi UI which is currently pulling in shared dependencies, scripts, styles, and images. * NIFI-13211: - Updating rat config. * NIFI-13211: - Bumping dependency check timeout. * NIFI-13211: - Since this page is included through the context viewer, the dependency imports must directly reference through its context path. * NIFI-13211: - Since this page is included through the context viewer, the dependency imports must directly reference through its context path. This closes #8828 > Decouple UIs from old NiFi UI > - > > Key: NIFI-13211 > URL: https://issues.apache.org/jira/browse/NIFI-13211 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Major > Time Spent: 40m > Remaining Estimate: 0h > > NiFi is comprised of numerous UIs that all get deployed at start up. These > UIs piggybacked on some shared stylesheets, components, and dependencies by > directly loading them through a relative path. In preparation for moving away > from the existing UI, these UIs are being updated to bundle these items > directly. In time, these UIs will all be modernized as well so this is just a > transitional step in that process. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-13211: Decoupling other UIs from exiting NiFi UI [nifi]
rfellows merged PR #8828: URL: https://github.com/apache/nifi/pull/8828 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-13238) Checkstyle - rules for whitespace
[ https://issues.apache.org/jira/browse/NIFI-13238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-13238: -- Status: Patch Available (was: Open) > Checkstyle - rules for whitespace > - > > Key: NIFI-13238 > URL: https://issues.apache.org/jira/browse/NIFI-13238 > Project: Apache NiFi > Issue Type: Improvement > Components: Tools and Build >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Adding the following rules: > * WhitespaceAfter > * NoWhitespaceAfter > * NoWhitespaceBefore > * WhitespaceAround -- This message was sent by Atlassian Jira (v8.20.10#820010)
[PR] NIFI-13238 - Adding checkstyle rules for whitespaces [nifi]
pvillard31 opened a new pull request, #8832: URL: https://github.com/apache/nifi/pull/8832 # Summary [NIFI-13238](https://issues.apache.org/jira/browse/NIFI-13238) - Adding checkstyle rules for whitespaces # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 21 ### UI Contributions - [ ] NiFi is modernizing its UI. Any contributions that update the [current UI](https://github.com/apache/nifi/tree/main/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui) also need to be implemented in the [new UI](https://github.com/apache/nifi/tree/main/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-frontend/src/main/nifi). ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-13238) Checkstyle - rules for whitespace
Pierre Villard created NIFI-13238: - Summary: Checkstyle - rules for whitespace Key: NIFI-13238 URL: https://issues.apache.org/jira/browse/NIFI-13238 Project: Apache NiFi Issue Type: Improvement Components: Tools and Build Reporter: Pierre Villard Assignee: Pierre Villard Adding the following rules: * WhitespaceAfter * NoWhitespaceAfter * NoWhitespaceBefore * WhitespaceAround -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-13237) Checkstyle improvements
Pierre Villard created NIFI-13237: - Summary: Checkstyle improvements Key: NIFI-13237 URL: https://issues.apache.org/jira/browse/NIFI-13237 Project: Apache NiFi Issue Type: Epic Reporter: Pierre Villard Assignee: Pierre Villard Epic to track the addition of some rules in the checkstyle configuration of the NiFi project. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-13236) Issue when viewing and downloading content
Matt Gilman created NIFI-13236:
--
Summary: Issue when viewing and downloading content
Key: NIFI-13236
URL: https://issues.apache.org/jira/browse/NIFI-13236
Project: Apache NiFi
Issue Type: Bug
Components: Core Framework
Reporter: Matt Gilman
When attempting to view or download content the flowfile's filename (from the
filename attribute) is used as the proposed downloaded filename. These
endpoints support gzip encoding. If the filename contains characters are
invalid VCHAR based on RFC9110, the endpoint fails with a connection error. The
log contains the following exception.
{noformat}
2024-05-14 14:52:40,290 ERROR [NiFi Web Server-138]
o.g.j.server.ServerRuntime$Responder An I/O error has occurred while writing a
response message entity to the container output stream.
org.glassfish.jersey.server.internal.process.MappableException:
java.io.IOException:
org.eclipse.jetty.http2.hpack.HpackException$SessionException: Could not hpack
encode HTTP/2.0{s=200,h=11,cl=-1}
at
org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor.aroundWriteTo(MappableExceptionWrapperInterceptor.java:67)
at
org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:139)
at
org.glassfish.jersey.message.internal.MessageBodyFactory.writeTo(MessageBodyFactory.java:1116)
at
org.glassfish.jersey.server.ServerRuntime$Responder.writeResponse(ServerRuntime.java:678)
at
org.glassfish.jersey.server.ServerRuntime$Responder.processResponse(ServerRuntime.java:387)
at
org.glassfish.jersey.server.ServerRuntime$Responder.process(ServerRuntime.java:377)
at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:266)
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248)
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244)
at org.glassfish.jersey.internal.Errors.process(Errors.java:292)
at org.glassfish.jersey.internal.Errors.process(Errors.java:274)
at org.glassfish.jersey.internal.Errors.process(Errors.java:244)
at
org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:266)
at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:242)
at
org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:697)
at
org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394)
at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346)
at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:358)
at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:312)
at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205)
at
org.eclipse.jetty.ee10.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1379)
at
org.eclipse.jetty.ee10.servlet.ServletHolder.handle(ServletHolder.java:736)
at
org.eclipse.jetty.ee10.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1614)
at
org.springframework.security.web.FilterChainProxy.lambda$doFilterInternal$3(FilterChainProxy.java:231)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:365)
at
org.springframework.security.web.access.intercept.AuthorizationFilter.doFilter(AuthorizationFilter.java:100)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:374)
at
org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:126)
at
org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:120)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:374)
at
org.apache.nifi.web.security.log.AuthenticationUserFilter.doFilterInternal(AuthenticationUserFilter.java:57)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:374)
at
org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:100)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:374)
at
org.springframework.security.oauth2.server.resource.web.authentication.BearerTokenAuthenticationFilter.doFilterInternal(BearerTokenAuthenticationFilter.java:145)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:374)
at
Re: [PR] NIFI-12411: Update PublishAMQP to read AMQP headers value from FlowFile attributes and `amq$headers` string [nifi]
umarhussain15 commented on PR #8105: URL: https://github.com/apache/nifi/pull/8105#issuecomment-2110984083 @exceptionfactory @joewitt I have added one more change, which will allow users to set amqp processor's password via env variables. I think this will be helpful in cases where we use AMQP processors in multiple NiFI instances. Instead of copying and pasting different passwords in processor properties, we can reference the env variable via its name. Let me know your thoughts on it. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12896: Add Endpoint Override URL property for PutSNS processor [nifi]
sujkm commented on PR #8529: URL: https://github.com/apache/nifi/pull/8529#issuecomment-2110945875 PR has been rebased on latest main branch -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-13234) Unauthorized canvas components issues
[ https://issues.apache.org/jira/browse/NIFI-13234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Aslan updated NIFI-13234: --- Summary: Unauthorized canvas components issues (was: Unauthorized PG issues) > Unauthorized canvas components issues > - > > Key: NIFI-13234 > URL: https://issues.apache.org/jira/browse/NIFI-13234 > Project: Apache NiFi > Issue Type: Sub-task >Reporter: Scott Aslan >Priority: Major > Attachments: Screenshot 2024-05-14 at 1.01.15 PM.png > > > There are contrast ratio issues in the unauthorized PG on the canvas: > > !Screenshot 2024-05-14 at 1.01.15 PM.png|width=415,height=222! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13234) Unauthorized canvas components issues
[ https://issues.apache.org/jira/browse/NIFI-13234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Aslan updated NIFI-13234: --- Description: There are color and contrast ratio issues for unauthorized components on the canvas: !Screenshot 2024-05-14 at 1.01.15 PM.png|width=415,height=222! was: There are contrast ratio issues in the unauthorized PG on the canvas: !Screenshot 2024-05-14 at 1.01.15 PM.png|width=415,height=222! > Unauthorized canvas components issues > - > > Key: NIFI-13234 > URL: https://issues.apache.org/jira/browse/NIFI-13234 > Project: Apache NiFi > Issue Type: Sub-task >Reporter: Scott Aslan >Priority: Major > Attachments: Screenshot 2024-05-14 at 1.01.15 PM.png > > > There are color and contrast ratio issues for unauthorized components on the > canvas: > > !Screenshot 2024-05-14 at 1.01.15 PM.png|width=415,height=222! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[PR] NIFI-13235 Use the PG id from the entity because component may be nul… [nifi]
bbende opened a new pull request, #8831: URL: https://github.com/apache/nifi/pull/8831 …l if current user does not have READ permissions # Summary [NIFI-13235](https://issues.apache.org/jira/browse/NIFI-13235) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [X] Build completed using `mvn clean install -P contrib-check` - [X] JDK 21 ### UI Contributions - [ ] NiFi is modernizing its UI. Any contributions that update the [current UI](https://github.com/apache/nifi/tree/main/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui) also need to be implemented in the [new UI](https://github.com/apache/nifi/tree/main/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-frontend/src/main/nifi). ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-13234) Unauthorized PG issues
[ https://issues.apache.org/jira/browse/NIFI-13234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Aslan updated NIFI-13234: --- Summary: Unauthorized PG issues (was: Ghosted PG issues) > Unauthorized PG issues > -- > > Key: NIFI-13234 > URL: https://issues.apache.org/jira/browse/NIFI-13234 > Project: Apache NiFi > Issue Type: Sub-task >Reporter: Scott Aslan >Priority: Major > Attachments: Screenshot 2024-05-14 at 1.01.15 PM.png > > > There are contrast ratio issues in the ghosted PG on the canvas: > > !Screenshot 2024-05-14 at 1.01.15 PM.png|width=415,height=222! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13234) Unauthorized PG issues
[ https://issues.apache.org/jira/browse/NIFI-13234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Aslan updated NIFI-13234: --- Description: There are contrast ratio issues in the unauthorized PG on the canvas: !Screenshot 2024-05-14 at 1.01.15 PM.png|width=415,height=222! was: There are contrast ratio issues in the ghosted PG on the canvas: !Screenshot 2024-05-14 at 1.01.15 PM.png|width=415,height=222! > Unauthorized PG issues > -- > > Key: NIFI-13234 > URL: https://issues.apache.org/jira/browse/NIFI-13234 > Project: Apache NiFi > Issue Type: Sub-task >Reporter: Scott Aslan >Priority: Major > Attachments: Screenshot 2024-05-14 at 1.01.15 PM.png > > > There are contrast ratio issues in the unauthorized PG on the canvas: > > !Screenshot 2024-05-14 at 1.01.15 PM.png|width=415,height=222! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-13235) Import from registry to a PG without read permissions causes NPE
Bryan Bende created NIFI-13235:
--
Summary: Import from registry to a PG without read permissions
causes NPE
Key: NIFI-13235
URL: https://issues.apache.org/jira/browse/NIFI-13235
Project: Apache NiFi
Issue Type: Bug
Affects Versions: 2.0.0-M2
Reporter: Bryan Bende
If you are inside a PG where you have WRITE permissions, but not READ
permissions, and you import a new PG from a registry client, it results in an
unexpected error page and an NPE in the logs:
{code:java}
2024-05-14 13:41:46,946 ERROR [NiFi Web Server-139]
o.a.nifi.web.api.config.ThrowableMapper An unexpected error has occurred:
java.lang.NullPointerException: Cannot invoke
"org.apache.nifi.web.api.dto.ProcessGroupDTO.getId()" because the return value
of "org.apache.nifi.web.api.entity.ProcessGroupEntity.getComponent()" is null.
Returning Internal Server Error response.
java.lang.NullPointerException: Cannot invoke
"org.apache.nifi.web.api.dto.ProcessGroupDTO.getId()" because the return value
of "org.apache.nifi.web.api.entity.ProcessGroupEntity.getComponent()" is null
at
org.apache.nifi.web.api.ProcessGroupResource.lambda$createProcessGroup$18(ProcessGroupResource.java:1016)
at
org.apache.nifi.web.api.ApplicationResource.withWriteLock(ApplicationResource.java:691)
at
org.apache.nifi.web.api.ProcessGroupResource.createProcessGroup(ProcessGroupResource.java:988)
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (NIFI-13234) Ghosted PG issues
Scott Aslan created NIFI-13234: -- Summary: Ghosted PG issues Key: NIFI-13234 URL: https://issues.apache.org/jira/browse/NIFI-13234 Project: Apache NiFi Issue Type: Sub-task Reporter: Scott Aslan Attachments: Screenshot 2024-05-14 at 1.01.15 PM.png There are contrast ratio issues in the ghosted PG on the canvas: !Screenshot 2024-05-14 at 1.01.15 PM.png|width=415,height=222! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-13233) Change version dialog issues
Scott Aslan created NIFI-13233: -- Summary: Change version dialog issues Key: NIFI-13233 URL: https://issues.apache.org/jira/browse/NIFI-13233 Project: Apache NiFi Issue Type: Sub-task Reporter: Scott Aslan Attachments: Screenshot 2024-05-14 at 1.21.56 PM.png There are some text overflow issues and the version column does not have enough width for the new git backed version name. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-13229) Submit branch, when appropriate, to backend when starting version control
[ https://issues.apache.org/jira/browse/NIFI-13229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Bende resolved NIFI-13229. Fix Version/s: 2.0.0-M4 Resolution: Fixed > Submit branch, when appropriate, to backend when starting version control > - > > Key: NIFI-13229 > URL: https://issues.apache.org/jira/browse/NIFI-13229 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Rob Fellows >Assignee: Rob Fellows >Priority: Major > Fix For: 2.0.0-M4 > > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-13229) Submit branch, when appropriate, to backend when starting version control
[ https://issues.apache.org/jira/browse/NIFI-13229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17846379#comment-17846379 ] ASF subversion and git services commented on NIFI-13229: Commit d20fa9417dfaddb7394be017256735ae1ef17c02 in nifi's branch refs/heads/main from Rob Fellows [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=d20fa9417d ] [NIFI-13229] - Include branch when submitting to save version control endpoint. (#8830) > Submit branch, when appropriate, to backend when starting version control > - > > Key: NIFI-13229 > URL: https://issues.apache.org/jira/browse/NIFI-13229 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Rob Fellows >Assignee: Rob Fellows >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] [NIFI-13229] - Include branch when submitting to save version control endpoint [nifi]
bbende merged PR #8830: URL: https://github.com/apache/nifi/pull/8830 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [NIFI-13082] Created SplitPcap processor, Pcap supporting class, and … [nifi]
dan-s1 commented on code in PR #8691:
URL: https://github.com/apache/nifi/pull/8691#discussion_r1600325204
##
nifi-extension-bundles/nifi-network-bundle/nifi-network-processors/src/main/java/org/apache/nifi/processors/network/SplitPCAP.java:
##
@@ -0,0 +1,203 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.network;
+
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.processors.network.util.PCAP;
+import org.apache.nifi.processors.network.util.PCAP.ByteBufferInterface;
+import org.apache.nifi.processors.network.util.PCAP.Packet;
+
+import java.io.ByteArrayOutputStream;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Set;
+
+@Tags({"PCAP", "Splitter", "Network", "Packet", "Capture", "Wireshark",
"TShark"})
+@CapabilityDescription("Splits a pcap file into multiple pcap files based on a
maximum size.")
+@WritesAttribute(attribute = "ERROR_REASON", description = "The reason the
flowfile was sent to the failure relationship.")
+
+public class SplitPCAP extends AbstractProcessor {
+
Review Comment:
Sorry about that. I am not sure `public` is needed. I believe package
visibility would work also.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [NIFI-13082] Created SplitPcap processor, Pcap supporting class, and … [nifi]
dan-s1 commented on code in PR #8691:
URL: https://github.com/apache/nifi/pull/8691#discussion_r1600325204
##
nifi-extension-bundles/nifi-network-bundle/nifi-network-processors/src/main/java/org/apache/nifi/processors/network/SplitPCAP.java:
##
@@ -0,0 +1,203 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.network;
+
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.processors.network.util.PCAP;
+import org.apache.nifi.processors.network.util.PCAP.ByteBufferInterface;
+import org.apache.nifi.processors.network.util.PCAP.Packet;
+
+import java.io.ByteArrayOutputStream;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Set;
+
+@Tags({"PCAP", "Splitter", "Network", "Packet", "Capture", "Wireshark",
"TShark"})
+@CapabilityDescription("Splits a pcap file into multiple pcap files based on a
maximum size.")
+@WritesAttribute(attribute = "ERROR_REASON", description = "The reason the
flowfile was sent to the failure relationship.")
+
+public class SplitPCAP extends AbstractProcessor {
+
Review Comment:
Sorry about that. I am not sure `public `is needed. I believe package
visibility would work also.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[PR] MINIFICPP-2358 Skip File permission tests while running as root [nifi-minifi-cpp]
martinzink opened a new pull request, #1789: URL: https://github.com/apache/nifi-minifi-cpp/pull/1789 Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFICPP- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically main)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check GitHub Actions CI results for build issues and submit an update to your PR as soon as possible. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [NIFI-13082] Created SplitPcap processor, Pcap supporting class, and … [nifi]
JackHintonSmartDCSIT commented on code in PR #8691:
URL: https://github.com/apache/nifi/pull/8691#discussion_r1600263921
##
nifi-extension-bundles/nifi-network-bundle/nifi-network-processors/src/main/java/org/apache/nifi/processors/network/SplitPCAP.java:
##
@@ -0,0 +1,203 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.network;
+
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.processors.network.util.PCAP;
+import org.apache.nifi.processors.network.util.PCAP.ByteBufferInterface;
+import org.apache.nifi.processors.network.util.PCAP.Packet;
+
+import java.io.ByteArrayOutputStream;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Set;
+
+@Tags({"PCAP", "Splitter", "Network", "Packet", "Capture", "Wireshark",
"TShark"})
+@CapabilityDescription("Splits a pcap file into multiple pcap files based on a
maximum size.")
+@WritesAttribute(attribute = "ERROR_REASON", description = "The reason the
flowfile was sent to the failure relationship.")
+
+public class SplitPCAP extends AbstractProcessor {
+
Review Comment:
I had to make this public or it wasn't visible from line 41
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (NIFI-12669) EvaluateXQuery processor incorrectly encodes result attributes
[
https://issues.apache.org/jira/browse/NIFI-12669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Burgess updated NIFI-12669:
Fix Version/s: 2.0.0-M4
(was: 2.0.0)
> EvaluateXQuery processor incorrectly encodes result attributes
> --
>
> Key: NIFI-12669
> URL: https://issues.apache.org/jira/browse/NIFI-12669
> Project: Apache NiFi
> Issue Type: Bug
> Components: Configuration, Extensions
> Environment: JVM with non-UTF-8 default encoding (e.g. default
> Windows installation)
>Reporter: René Zeidler
>Assignee: Jim Steinebrey
>Priority: Major
> Labels: encoding, utf8, windows, xml
> Fix For: 1.27.0, 2.0.0-M4
>
> Attachments: EvaluateXQuery_Encoding_Bug.json,
> image-2024-01-25-10-24-17-005.png, image-2024-01-25-10-31-35-200.png
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> h2. Environment
> This issue affects environments where the JVM default encoding is not
> {{{}UTF-8{}}}. Standard Java installations on Windows are affected, as they
> usually use the default encoding {{{}windows-1252{}}}. To reproduce the issue
> on Linux, change the default encoding to {{windows-1252}} by adding the
> following line to your {{{}bootstrap.conf{}}}:
> {quote}{{java.arg.21=-Dfile.encoding=windows-1252}}
> {quote}
> h2. Summary
> The EvaluateXQuery incorrectly encodes result values when storing them in
> attributes. This causes non-ASCII characters to be garbled.
> Example:
> !image-2024-01-25-10-24-17-005.png!
> h2. Steps to reproduce
> # Make sure NiFi runs with a non-UTF-8 default encoding, see "Environment"
> # Create a GenerateFlowFile processor with the following content:
> {quote}{{}}
> {{}}
> {{ This text contains non-ASCII characters: ÄÖÜäöüßéèóò}}
> {{}}
> {quote}
> # Connect the processor to an EvaluateXQuery processor.
> Set the {{Destination}} to {{{}flowfile-attribute{}}}.
> Create a custom property {{myData}} with value {{{}string(/myRoot/myData){}}}.
> # Connect the outputs of the EvaluateXQuery processor to funnels to be able
> to observe the result in the queue.
> # Start the EvaluateXQuery processor and run the GenerateFlowFile processor
> once.
> The flow should look similar to this:
> !image-2024-01-25-10-31-35-200.png!
> I also attached a JSON export of the example flow.
> # Observe the attributes of the resulting FlowFile in the queue.
> h3. Expected Result
> The FlowFile should contain an attribute {{myData}} with the value {{{}"This
> text contains non-ASCII characters: ÄÖÜäöüßéèóò"{}}}.
> h3. Actual Result
> The attribute has the value "This text contains non-ASCII characters:
> ÄÖÜäöüßéèóò".
> h2. Root Cause Analysis
> EvaluateXQuery uses the method
> [{{formatItem}}|https://github.com/apache/nifi/blob/2e3f83eb54cbc040b5a1da5bce9a74a558f08ea4/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EvaluateXQuery.java#L368-L372]
> to write the query result to an attribute. This method calls
> {{{}ByteArrayOutputStream{}}}'s
> [toString|https://docs.oracle.com/en/java/javase/21/docs/api/java.base/java/io/ByteArrayOutputStream.html#toString()]
> method without an encoding argument, which then defaults to the default
> charset of the environment. Bytes are always written to this output stream
> using UTF-8
> ([.getBytes(StandardCharsets.UTF8)|https://github.com/apache/nifi/blob/2e3f83eb54cbc040b5a1da5bce9a74a558f08ea4/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EvaluateXQuery.java#L397]).
> When the default charset is not UTF-8, this results in UTF-8 bytes to be
> interpreted in a different encoding when converting to a string, resulting in
> garbled text (see above).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] MINIFICPP-2346: Build MiNiFi Core, MainExe, Standard-Processors with Conan2 [nifi-minifi-cpp]
james94 commented on PR #1775: URL: https://github.com/apache/nifi-minifi-cpp/pull/1775#issuecomment-2110550068 @martinzink thanks for the suggestion. I brought MINIFI-2346 to up to date with latest upstream MiNiFi C++ main branch. Ubuntu 20.04 and Docker build for integration tests passed. The other CI/CDs failed. Its probably related to some of the changes I am introducing to the build infrastructure. I have mainly been testing the updates in a Ubuntu 22.04 docker container. I haven't tested them yet on Windows or Mac OS X. Those 2 last OSs, I'd test later. I also would need to look into why CI/CD is failing too. After I finish refactoring build infrastructure to support Conan version 2 and keep backward compatibility with the standalone CMake approach, I will work toward updating the MiNiFi bootstrap py script with conan support too. I will get to the CI/CD issues as soon as I can, but those will probably be the last updates. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] MINIFICPP-2338 Move out ExecuteProcess from standard processors [nifi-minifi-cpp]
martinzink opened a new pull request, #1788: URL: https://github.com/apache/nifi-minifi-cpp/pull/1788 Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFICPP- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically main)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check GitHub Actions CI results for build issues and submit an update to your PR as soon as possible. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-13232) Add REST API Resource for Authentication Configuration
David Handermann created NIFI-13232: --- Summary: Add REST API Resource for Authentication Configuration Key: NIFI-13232 URL: https://issues.apache.org/jira/browse/NIFI-13232 Project: Apache NiFi Issue Type: Improvement Components: Core Framework Reporter: David Handermann Assignee: David Handermann The NiFi user interface applications include Servlet Filters to handle Login and Logout redirection based on the configured authentication strategy. Rather than continuing to maintain these filters, a new REST API resource should be created to provide login and logout URLs based on application configuration. The response should also indicate whether the system supports login and logout operations so that the user interface can present the appropriate forms and links according to the current system configuration. Implementing this new REST API should support the removal of the Login and Logout filters from the new user interface, simplifying the build and deployment process. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12669) EvaluateXQuery processor incorrectly encodes result attributes
[
https://issues.apache.org/jira/browse/NIFI-12669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann updated NIFI-12669:
Resolution: Fixed
Status: Resolved (was: Patch Available)
> EvaluateXQuery processor incorrectly encodes result attributes
> --
>
> Key: NIFI-12669
> URL: https://issues.apache.org/jira/browse/NIFI-12669
> Project: Apache NiFi
> Issue Type: Bug
> Components: Configuration, Extensions
> Environment: JVM with non-UTF-8 default encoding (e.g. default
> Windows installation)
>Reporter: René Zeidler
>Assignee: Jim Steinebrey
>Priority: Major
> Labels: encoding, utf8, windows, xml
> Fix For: 1.27.0, 2.0.0
>
> Attachments: EvaluateXQuery_Encoding_Bug.json,
> image-2024-01-25-10-24-17-005.png, image-2024-01-25-10-31-35-200.png
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> h2. Environment
> This issue affects environments where the JVM default encoding is not
> {{{}UTF-8{}}}. Standard Java installations on Windows are affected, as they
> usually use the default encoding {{{}windows-1252{}}}. To reproduce the issue
> on Linux, change the default encoding to {{windows-1252}} by adding the
> following line to your {{{}bootstrap.conf{}}}:
> {quote}{{java.arg.21=-Dfile.encoding=windows-1252}}
> {quote}
> h2. Summary
> The EvaluateXQuery incorrectly encodes result values when storing them in
> attributes. This causes non-ASCII characters to be garbled.
> Example:
> !image-2024-01-25-10-24-17-005.png!
> h2. Steps to reproduce
> # Make sure NiFi runs with a non-UTF-8 default encoding, see "Environment"
> # Create a GenerateFlowFile processor with the following content:
> {quote}{{}}
> {{}}
> {{ This text contains non-ASCII characters: ÄÖÜäöüßéèóò}}
> {{}}
> {quote}
> # Connect the processor to an EvaluateXQuery processor.
> Set the {{Destination}} to {{{}flowfile-attribute{}}}.
> Create a custom property {{myData}} with value {{{}string(/myRoot/myData){}}}.
> # Connect the outputs of the EvaluateXQuery processor to funnels to be able
> to observe the result in the queue.
> # Start the EvaluateXQuery processor and run the GenerateFlowFile processor
> once.
> The flow should look similar to this:
> !image-2024-01-25-10-31-35-200.png!
> I also attached a JSON export of the example flow.
> # Observe the attributes of the resulting FlowFile in the queue.
> h3. Expected Result
> The FlowFile should contain an attribute {{myData}} with the value {{{}"This
> text contains non-ASCII characters: ÄÖÜäöüßéèóò"{}}}.
> h3. Actual Result
> The attribute has the value "This text contains non-ASCII characters:
> ÄÖÜäöüßéèóò".
> h2. Root Cause Analysis
> EvaluateXQuery uses the method
> [{{formatItem}}|https://github.com/apache/nifi/blob/2e3f83eb54cbc040b5a1da5bce9a74a558f08ea4/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EvaluateXQuery.java#L368-L372]
> to write the query result to an attribute. This method calls
> {{{}ByteArrayOutputStream{}}}'s
> [toString|https://docs.oracle.com/en/java/javase/21/docs/api/java.base/java/io/ByteArrayOutputStream.html#toString()]
> method without an encoding argument, which then defaults to the default
> charset of the environment. Bytes are always written to this output stream
> using UTF-8
> ([.getBytes(StandardCharsets.UTF8)|https://github.com/apache/nifi/blob/2e3f83eb54cbc040b5a1da5bce9a74a558f08ea4/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EvaluateXQuery.java#L397]).
> When the default charset is not UTF-8, this results in UTF-8 bytes to be
> interpreted in a different encoding when converting to a string, resulting in
> garbled text (see above).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12669) EvaluateXQuery processor incorrectly encodes result attributes
[
https://issues.apache.org/jira/browse/NIFI-12669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann updated NIFI-12669:
Fix Version/s: 1.27.0
2.0.0
> EvaluateXQuery processor incorrectly encodes result attributes
> --
>
> Key: NIFI-12669
> URL: https://issues.apache.org/jira/browse/NIFI-12669
> Project: Apache NiFi
> Issue Type: Bug
> Components: Configuration, Extensions
> Environment: JVM with non-UTF-8 default encoding (e.g. default
> Windows installation)
>Reporter: René Zeidler
>Assignee: Jim Steinebrey
>Priority: Major
> Labels: encoding, utf8, windows, xml
> Fix For: 1.27.0, 2.0.0
>
> Attachments: EvaluateXQuery_Encoding_Bug.json,
> image-2024-01-25-10-24-17-005.png, image-2024-01-25-10-31-35-200.png
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> h2. Environment
> This issue affects environments where the JVM default encoding is not
> {{{}UTF-8{}}}. Standard Java installations on Windows are affected, as they
> usually use the default encoding {{{}windows-1252{}}}. To reproduce the issue
> on Linux, change the default encoding to {{windows-1252}} by adding the
> following line to your {{{}bootstrap.conf{}}}:
> {quote}{{java.arg.21=-Dfile.encoding=windows-1252}}
> {quote}
> h2. Summary
> The EvaluateXQuery incorrectly encodes result values when storing them in
> attributes. This causes non-ASCII characters to be garbled.
> Example:
> !image-2024-01-25-10-24-17-005.png!
> h2. Steps to reproduce
> # Make sure NiFi runs with a non-UTF-8 default encoding, see "Environment"
> # Create a GenerateFlowFile processor with the following content:
> {quote}{{}}
> {{}}
> {{ This text contains non-ASCII characters: ÄÖÜäöüßéèóò}}
> {{}}
> {quote}
> # Connect the processor to an EvaluateXQuery processor.
> Set the {{Destination}} to {{{}flowfile-attribute{}}}.
> Create a custom property {{myData}} with value {{{}string(/myRoot/myData){}}}.
> # Connect the outputs of the EvaluateXQuery processor to funnels to be able
> to observe the result in the queue.
> # Start the EvaluateXQuery processor and run the GenerateFlowFile processor
> once.
> The flow should look similar to this:
> !image-2024-01-25-10-31-35-200.png!
> I also attached a JSON export of the example flow.
> # Observe the attributes of the resulting FlowFile in the queue.
> h3. Expected Result
> The FlowFile should contain an attribute {{myData}} with the value {{{}"This
> text contains non-ASCII characters: ÄÖÜäöüßéèóò"{}}}.
> h3. Actual Result
> The attribute has the value "This text contains non-ASCII characters:
> ÄÖÜäöüßéèóò".
> h2. Root Cause Analysis
> EvaluateXQuery uses the method
> [{{formatItem}}|https://github.com/apache/nifi/blob/2e3f83eb54cbc040b5a1da5bce9a74a558f08ea4/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EvaluateXQuery.java#L368-L372]
> to write the query result to an attribute. This method calls
> {{{}ByteArrayOutputStream{}}}'s
> [toString|https://docs.oracle.com/en/java/javase/21/docs/api/java.base/java/io/ByteArrayOutputStream.html#toString()]
> method without an encoding argument, which then defaults to the default
> charset of the environment. Bytes are always written to this output stream
> using UTF-8
> ([.getBytes(StandardCharsets.UTF8)|https://github.com/apache/nifi/blob/2e3f83eb54cbc040b5a1da5bce9a74a558f08ea4/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EvaluateXQuery.java#L397]).
> When the default charset is not UTF-8, this results in UTF-8 bytes to be
> interpreted in a different encoding when converting to a string, resulting in
> garbled text (see above).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (NIFI-13231) Update GitHubFlowRegistryClient to generate App Installation Token
Brian Ghigiarelli created NIFI-13231: Summary: Update GitHubFlowRegistryClient to generate App Installation Token Key: NIFI-13231 URL: https://issues.apache.org/jira/browse/NIFI-13231 Project: Apache NiFi Issue Type: Improvement Affects Versions: 2.0.0-M3 Reporter: Brian Ghigiarelli The new GitHubFlowRegistryClient accepts two types of authentication: # Personal Access Token # App Installation Token The App Installation Token requires the user to input the token as a property, then uses that to communicate with GitHub. However, this token is short-lived and would require frequent updates (~hourly). Instead of directly providing the token, the GitHubFlowRegistryClient should accept as properties: # App ID # Private Key (PEM text format) and use these values to generate a short-lived token, following the guide at https://docs.github.com/en/apps/creating-github-apps/authenticating-with-a-github-app/generating-a-json-web-token-jwt-for-a-github-app#generating-a-json-web-token-jwt -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-13230) Include Flow Registry Clients in Docs
Brian Ghigiarelli created NIFI-13230: Summary: Include Flow Registry Clients in Docs Key: NIFI-13230 URL: https://issues.apache.org/jira/browse/NIFI-13230 Project: Apache NiFi Issue Type: Improvement Components: Documentation Website Affects Versions: 2.0.0-M2, 2.0.0-M3 Reporter: Brian Ghigiarelli Flow Registry Clients are not yet included in the generated help docs. There is only a section in the User Guide for how to configure a registry client: https://nifi.apache.org/documentation/nifi-2.0.0-M2/html/user-guide.html#connecting-to-a-nifi-registry Two types exist now (one for NiFi Registry and one for GitHub), but there is room to expand to other registries. Flow Registry Clients can be included in a new section within the docs at the same level as Processors, Flow Analysis Rules, Reporting Tasks, etc. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12669) EvaluateXQuery processor incorrectly encodes result attributes
[
https://issues.apache.org/jira/browse/NIFI-12669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17846342#comment-17846342
]
ASF subversion and git services commented on NIFI-12669:
Commit e6a90eb1e8609eae2ef22f2490327451186bbe7b in nifi's branch
refs/heads/main from Jim Steinebrey
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=e6a90eb1e8 ]
NIFI-12669 Fix EvaluateXQuery processor which incorrectly encodes result
attributes in certain case
Signed-off-by: Matt Burgess
This closes #8826
> EvaluateXQuery processor incorrectly encodes result attributes
> --
>
> Key: NIFI-12669
> URL: https://issues.apache.org/jira/browse/NIFI-12669
> Project: Apache NiFi
> Issue Type: Bug
> Components: Configuration, Extensions
> Environment: JVM with non-UTF-8 default encoding (e.g. default
> Windows installation)
>Reporter: René Zeidler
>Assignee: Jim Steinebrey
>Priority: Major
> Labels: encoding, utf8, windows, xml
> Attachments: EvaluateXQuery_Encoding_Bug.json,
> image-2024-01-25-10-24-17-005.png, image-2024-01-25-10-31-35-200.png
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> h2. Environment
> This issue affects environments where the JVM default encoding is not
> {{{}UTF-8{}}}. Standard Java installations on Windows are affected, as they
> usually use the default encoding {{{}windows-1252{}}}. To reproduce the issue
> on Linux, change the default encoding to {{windows-1252}} by adding the
> following line to your {{{}bootstrap.conf{}}}:
> {quote}{{java.arg.21=-Dfile.encoding=windows-1252}}
> {quote}
> h2. Summary
> The EvaluateXQuery incorrectly encodes result values when storing them in
> attributes. This causes non-ASCII characters to be garbled.
> Example:
> !image-2024-01-25-10-24-17-005.png!
> h2. Steps to reproduce
> # Make sure NiFi runs with a non-UTF-8 default encoding, see "Environment"
> # Create a GenerateFlowFile processor with the following content:
> {quote}{{}}
> {{}}
> {{ This text contains non-ASCII characters: ÄÖÜäöüßéèóò}}
> {{}}
> {quote}
> # Connect the processor to an EvaluateXQuery processor.
> Set the {{Destination}} to {{{}flowfile-attribute{}}}.
> Create a custom property {{myData}} with value {{{}string(/myRoot/myData){}}}.
> # Connect the outputs of the EvaluateXQuery processor to funnels to be able
> to observe the result in the queue.
> # Start the EvaluateXQuery processor and run the GenerateFlowFile processor
> once.
> The flow should look similar to this:
> !image-2024-01-25-10-31-35-200.png!
> I also attached a JSON export of the example flow.
> # Observe the attributes of the resulting FlowFile in the queue.
> h3. Expected Result
> The FlowFile should contain an attribute {{myData}} with the value {{{}"This
> text contains non-ASCII characters: ÄÖÜäöüßéèóò"{}}}.
> h3. Actual Result
> The attribute has the value "This text contains non-ASCII characters:
> ÄÖÜäöüßéèóò".
> h2. Root Cause Analysis
> EvaluateXQuery uses the method
> [{{formatItem}}|https://github.com/apache/nifi/blob/2e3f83eb54cbc040b5a1da5bce9a74a558f08ea4/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EvaluateXQuery.java#L368-L372]
> to write the query result to an attribute. This method calls
> {{{}ByteArrayOutputStream{}}}'s
> [toString|https://docs.oracle.com/en/java/javase/21/docs/api/java.base/java/io/ByteArrayOutputStream.html#toString()]
> method without an encoding argument, which then defaults to the default
> charset of the environment. Bytes are always written to this output stream
> using UTF-8
> ([.getBytes(StandardCharsets.UTF8)|https://github.com/apache/nifi/blob/2e3f83eb54cbc040b5a1da5bce9a74a558f08ea4/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EvaluateXQuery.java#L397]).
> When the default charset is not UTF-8, this results in UTF-8 bytes to be
> interpreted in a different encoding when converting to a string, resulting in
> garbled text (see above).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] NIFI-12669 Fix EvaluateXQuery processor to encode result attr correctly on Windows [nifi]
mattyb149 commented on PR #8826: URL: https://github.com/apache/nifi/pull/8826#issuecomment-2110490625 +1 LGTM, thanks for the fix! Merging to main and support/nifi-1.x -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12669 Fix EvaluateXQuery processor to encode result attr correctly on Windows [nifi]
mattyb149 closed pull request #8826: NIFI-12669 Fix EvaluateXQuery processor to encode result attr correctly on Windows URL: https://github.com/apache/nifi/pull/8826 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-12669) EvaluateXQuery processor incorrectly encodes result attributes
[
https://issues.apache.org/jira/browse/NIFI-12669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17846339#comment-17846339
]
ASF subversion and git services commented on NIFI-12669:
Commit 47101f760e60f2a93e524c2adb01ebe29ebb754d in nifi's branch
refs/heads/support/nifi-1.x from Jim Steinebrey
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=47101f760e ]
NIFI-12669 Fix EvaluateXQuery processor which incorrectly encodes result
attributes in certain case
Signed-off-by: Matt Burgess
> EvaluateXQuery processor incorrectly encodes result attributes
> --
>
> Key: NIFI-12669
> URL: https://issues.apache.org/jira/browse/NIFI-12669
> Project: Apache NiFi
> Issue Type: Bug
> Components: Configuration, Extensions
> Environment: JVM with non-UTF-8 default encoding (e.g. default
> Windows installation)
>Reporter: René Zeidler
>Assignee: Jim Steinebrey
>Priority: Major
> Labels: encoding, utf8, windows, xml
> Attachments: EvaluateXQuery_Encoding_Bug.json,
> image-2024-01-25-10-24-17-005.png, image-2024-01-25-10-31-35-200.png
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> h2. Environment
> This issue affects environments where the JVM default encoding is not
> {{{}UTF-8{}}}. Standard Java installations on Windows are affected, as they
> usually use the default encoding {{{}windows-1252{}}}. To reproduce the issue
> on Linux, change the default encoding to {{windows-1252}} by adding the
> following line to your {{{}bootstrap.conf{}}}:
> {quote}{{java.arg.21=-Dfile.encoding=windows-1252}}
> {quote}
> h2. Summary
> The EvaluateXQuery incorrectly encodes result values when storing them in
> attributes. This causes non-ASCII characters to be garbled.
> Example:
> !image-2024-01-25-10-24-17-005.png!
> h2. Steps to reproduce
> # Make sure NiFi runs with a non-UTF-8 default encoding, see "Environment"
> # Create a GenerateFlowFile processor with the following content:
> {quote}{{}}
> {{}}
> {{ This text contains non-ASCII characters: ÄÖÜäöüßéèóò}}
> {{}}
> {quote}
> # Connect the processor to an EvaluateXQuery processor.
> Set the {{Destination}} to {{{}flowfile-attribute{}}}.
> Create a custom property {{myData}} with value {{{}string(/myRoot/myData){}}}.
> # Connect the outputs of the EvaluateXQuery processor to funnels to be able
> to observe the result in the queue.
> # Start the EvaluateXQuery processor and run the GenerateFlowFile processor
> once.
> The flow should look similar to this:
> !image-2024-01-25-10-31-35-200.png!
> I also attached a JSON export of the example flow.
> # Observe the attributes of the resulting FlowFile in the queue.
> h3. Expected Result
> The FlowFile should contain an attribute {{myData}} with the value {{{}"This
> text contains non-ASCII characters: ÄÖÜäöüßéèóò"{}}}.
> h3. Actual Result
> The attribute has the value "This text contains non-ASCII characters:
> ÄÖÜäöüßéèóò".
> h2. Root Cause Analysis
> EvaluateXQuery uses the method
> [{{formatItem}}|https://github.com/apache/nifi/blob/2e3f83eb54cbc040b5a1da5bce9a74a558f08ea4/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EvaluateXQuery.java#L368-L372]
> to write the query result to an attribute. This method calls
> {{{}ByteArrayOutputStream{}}}'s
> [toString|https://docs.oracle.com/en/java/javase/21/docs/api/java.base/java/io/ByteArrayOutputStream.html#toString()]
> method without an encoding argument, which then defaults to the default
> charset of the environment. Bytes are always written to this output stream
> using UTF-8
> ([.getBytes(StandardCharsets.UTF8)|https://github.com/apache/nifi/blob/2e3f83eb54cbc040b5a1da5bce9a74a558f08ea4/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EvaluateXQuery.java#L397]).
> When the default charset is not UTF-8, this results in UTF-8 bytes to be
> interpreted in a different encoding when converting to a string, resulting in
> garbled text (see above).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12669) EvaluateXQuery processor incorrectly encodes result attributes
[
https://issues.apache.org/jira/browse/NIFI-12669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Burgess updated NIFI-12669:
Affects Version/s: (was: 2.0.0-M1)
(was: 1.24.0)
(was: 1.25.0)
(was: 2.0.0-M2)
Status: Patch Available (was: Open)
> EvaluateXQuery processor incorrectly encodes result attributes
> --
>
> Key: NIFI-12669
> URL: https://issues.apache.org/jira/browse/NIFI-12669
> Project: Apache NiFi
> Issue Type: Bug
> Components: Configuration, Extensions
> Environment: JVM with non-UTF-8 default encoding (e.g. default
> Windows installation)
>Reporter: René Zeidler
>Assignee: Jim Steinebrey
>Priority: Major
> Labels: encoding, utf8, windows, xml
> Attachments: EvaluateXQuery_Encoding_Bug.json,
> image-2024-01-25-10-24-17-005.png, image-2024-01-25-10-31-35-200.png
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> h2. Environment
> This issue affects environments where the JVM default encoding is not
> {{{}UTF-8{}}}. Standard Java installations on Windows are affected, as they
> usually use the default encoding {{{}windows-1252{}}}. To reproduce the issue
> on Linux, change the default encoding to {{windows-1252}} by adding the
> following line to your {{{}bootstrap.conf{}}}:
> {quote}{{java.arg.21=-Dfile.encoding=windows-1252}}
> {quote}
> h2. Summary
> The EvaluateXQuery incorrectly encodes result values when storing them in
> attributes. This causes non-ASCII characters to be garbled.
> Example:
> !image-2024-01-25-10-24-17-005.png!
> h2. Steps to reproduce
> # Make sure NiFi runs with a non-UTF-8 default encoding, see "Environment"
> # Create a GenerateFlowFile processor with the following content:
> {quote}{{}}
> {{}}
> {{ This text contains non-ASCII characters: ÄÖÜäöüßéèóò}}
> {{}}
> {quote}
> # Connect the processor to an EvaluateXQuery processor.
> Set the {{Destination}} to {{{}flowfile-attribute{}}}.
> Create a custom property {{myData}} with value {{{}string(/myRoot/myData){}}}.
> # Connect the outputs of the EvaluateXQuery processor to funnels to be able
> to observe the result in the queue.
> # Start the EvaluateXQuery processor and run the GenerateFlowFile processor
> once.
> The flow should look similar to this:
> !image-2024-01-25-10-31-35-200.png!
> I also attached a JSON export of the example flow.
> # Observe the attributes of the resulting FlowFile in the queue.
> h3. Expected Result
> The FlowFile should contain an attribute {{myData}} with the value {{{}"This
> text contains non-ASCII characters: ÄÖÜäöüßéèóò"{}}}.
> h3. Actual Result
> The attribute has the value "This text contains non-ASCII characters:
> ÄÖÜäöüßéèóò".
> h2. Root Cause Analysis
> EvaluateXQuery uses the method
> [{{formatItem}}|https://github.com/apache/nifi/blob/2e3f83eb54cbc040b5a1da5bce9a74a558f08ea4/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EvaluateXQuery.java#L368-L372]
> to write the query result to an attribute. This method calls
> {{{}ByteArrayOutputStream{}}}'s
> [toString|https://docs.oracle.com/en/java/javase/21/docs/api/java.base/java/io/ByteArrayOutputStream.html#toString()]
> method without an encoding argument, which then defaults to the default
> charset of the environment. Bytes are always written to this output stream
> using UTF-8
> ([.getBytes(StandardCharsets.UTF8)|https://github.com/apache/nifi/blob/2e3f83eb54cbc040b5a1da5bce9a74a558f08ea4/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EvaluateXQuery.java#L397]).
> When the default charset is not UTF-8, this results in UTF-8 bytes to be
> interpreted in a different encoding when converting to a string, resulting in
> garbled text (see above).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (NIFI-13229) Submit branch, when appropriate, to backend when starting version control
Rob Fellows created NIFI-13229: -- Summary: Submit branch, when appropriate, to backend when starting version control Key: NIFI-13229 URL: https://issues.apache.org/jira/browse/NIFI-13229 Project: Apache NiFi Issue Type: Sub-task Components: Core UI Reporter: Rob Fellows Assignee: Rob Fellows -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-13221) Display hotkey combos in context menu
[ https://issues.apache.org/jira/browse/NIFI-13221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17846321#comment-17846321 ] ASF subversion and git services commented on NIFI-13221: Commit 7aaa5b42247e9ded4fa8d43186713b5f15717867 in nifi's branch refs/heads/main from Rob Fellows [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=7aaa5b4224 ] [NIFI-13221] - display supported shortcut/hotkey combos in context menu (#8821) This closes #8821 > Display hotkey combos in context menu > - > > Key: NIFI-13221 > URL: https://issues.apache.org/jira/browse/NIFI-13221 > Project: Apache NiFi > Issue Type: Sub-task >Reporter: Rob Fellows >Assignee: Rob Fellows >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > When appropriate, display the hotkey combinations in the context menu to > bring awareness to them. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13221) Display hotkey combos in context menu
[ https://issues.apache.org/jira/browse/NIFI-13221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Aslan updated NIFI-13221: --- Resolution: Fixed Status: Resolved (was: Patch Available) > Display hotkey combos in context menu > - > > Key: NIFI-13221 > URL: https://issues.apache.org/jira/browse/NIFI-13221 > Project: Apache NiFi > Issue Type: Sub-task >Reporter: Rob Fellows >Assignee: Rob Fellows >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > When appropriate, display the hotkey combinations in the context menu to > bring awareness to them. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] [NIFI-13221] - display supported shortcut/hotkey combos in context menu [nifi]
scottyaslan merged PR #8821: URL: https://github.com/apache/nifi/pull/8821 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-13144) Improve dependency-update detection configuration and bump various detected dependency versions
[
https://issues.apache.org/jira/browse/NIFI-13144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann updated NIFI-13144:
Issue Type: Improvement (was: Task)
> Improve dependency-update detection configuration and bump various detected
> dependency versions
> ---
>
> Key: NIFI-13144
> URL: https://issues.apache.org/jira/browse/NIFI-13144
> Project: Apache NiFi
> Issue Type: Improvement
>Reporter: Joe Witt
>Assignee: Joe Witt
>Priority: Major
> Fix For: 2.0.0-M3
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> {noformat}
> com.puppycrawl.tools:checkstyle ... 10.15.0 -> 10.16.0
> software.amazon.awssdk:xray ... 2.25.40 -> 2.25.45
> com.amazonaws:aws-java-sdk-keyspaces 1.12.710 -> 1.12.715
> com.azure:azure-sdk-bom 1.2.21 -> 1.2.23
> com.google.cloud:libraries-bom ... 26.37.0 -> 26.38.0
> net.bytebuddy:byte-buddy .. 1.14.12 -> 1.14.14
> org.checkerframework:checker-qual ... 3.37.0 -> 3.43.0
> org.eclipse.jdt:ecj . 3.33.0 -> 3.37.0
> com.microsoft.azure:msal4j .. 1.14.3 -> 1.15.0
> com.google.guava:guava .. 33.1.0-jre -> 33.2.0-jre
> com.ibm.icu:icu4j ... 74.2 -> 75.1
> org.opensaml:opensaml-* 4.3.1 -> 4.3.2
> org.apache.sshd:sshd-* ... 2.12.0 -> 2.12.1
> com.nimbusds:oauth2-oidc-sdk ... 9.43.3 -> 9.43.4
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12914) Upgrade follow-redirects library to 1.15.6
[ https://issues.apache.org/jira/browse/NIFI-12914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-12914: Issue Type: Improvement (was: Task) > Upgrade follow-redirects library to 1.15.6 > -- > > Key: NIFI-12914 > URL: https://issues.apache.org/jira/browse/NIFI-12914 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Reporter: Matt Gilman >Priority: Minor > Fix For: 2.0.0-M3 > > > Bump follow-redirects from 1.15.5 to 1.15.6 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12853) Refactor FlowFilePrioritizer using updated Java APIs
[ https://issues.apache.org/jira/browse/NIFI-12853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-12853: Issue Type: Improvement (was: Task) > Refactor FlowFilePrioritizer using updated Java APIs > > > Key: NIFI-12853 > URL: https://issues.apache.org/jira/browse/NIFI-12853 > Project: Apache NiFi > Issue Type: Improvement >Reporter: endzeit >Assignee: endzeit >Priority: Minor > Fix For: 2.0.0-M3 > > Time Spent: 0.5h > Remaining Estimate: 0h > > The FlowFilePrioritizer implementations and test contain some boilerplate / > duplication that can be reduced by using updated Java APIs. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12914) Upgrade follow-redirects library to 1.15.6
[ https://issues.apache.org/jira/browse/NIFI-12914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-12914: Priority: Minor (was: Major) > Upgrade follow-redirects library to 1.15.6 > -- > > Key: NIFI-12914 > URL: https://issues.apache.org/jira/browse/NIFI-12914 > Project: Apache NiFi > Issue Type: Task > Components: Core UI >Reporter: Matt Gilman >Priority: Minor > Fix For: 2.0.0-M3 > > > Bump follow-redirects from 1.15.5 to 1.15.6 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12914) Upgrade follow-redirects library to 1.15.6
[ https://issues.apache.org/jira/browse/NIFI-12914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-12914: Summary: Upgrade follow-redirects library to 1.15.6 (was: Dependabot: follow-redirects) > Upgrade follow-redirects library to 1.15.6 > -- > > Key: NIFI-12914 > URL: https://issues.apache.org/jira/browse/NIFI-12914 > Project: Apache NiFi > Issue Type: Task > Components: Core UI >Reporter: Matt Gilman >Priority: Major > Fix For: 2.0.0-M3 > > > Bump follow-redirects from 1.15.5 to 1.15.6 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13215) Update AWS, OpenTelemetry, latest dependency scan findings
[ https://issues.apache.org/jira/browse/NIFI-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-13215: Priority: Minor (was: Major) > Update AWS, OpenTelemetry, latest dependency scan findings > -- > > Key: NIFI-13215 > URL: https://issues.apache.org/jira/browse/NIFI-13215 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Joe Witt >Assignee: Joe Witt >Priority: Minor > Fix For: 2.0.0-M3 > > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13215) Update AWS, OpenTelemetry, latest dependency scan findings
[ https://issues.apache.org/jira/browse/NIFI-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-13215: Issue Type: Improvement (was: Task) > Update AWS, OpenTelemetry, latest dependency scan findings > -- > > Key: NIFI-13215 > URL: https://issues.apache.org/jira/browse/NIFI-13215 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Joe Witt >Assignee: Joe Witt >Priority: Major > Fix For: 2.0.0-M3 > > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12853) Refactor FlowFilePrioritizer using updated Java APIs
[ https://issues.apache.org/jira/browse/NIFI-12853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-12853: Priority: Minor (was: Major) > Refactor FlowFilePrioritizer using updated Java APIs > > > Key: NIFI-12853 > URL: https://issues.apache.org/jira/browse/NIFI-12853 > Project: Apache NiFi > Issue Type: Task >Reporter: endzeit >Assignee: endzeit >Priority: Minor > Fix For: 2.0.0-M3 > > Time Spent: 0.5h > Remaining Estimate: 0h > > The FlowFilePrioritizer implementations and test contain some boilerplate / > duplication that can be reduced by using updated Java APIs. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13113) Upgrade Iceberg from 1.5.0 to 1.5.2
[ https://issues.apache.org/jira/browse/NIFI-13113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-13113: Summary: Upgrade Iceberg from 1.5.0 to 1.5.2 (was: upgrade Iceberg from 1.5.0 to 1.5.2) > Upgrade Iceberg from 1.5.0 to 1.5.2 > --- > > Key: NIFI-13113 > URL: https://issues.apache.org/jira/browse/NIFI-13113 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Sean Sullivan >Assignee: Joe Witt >Priority: Minor > Fix For: 2.0.0-M3 > > Time Spent: 1h 20m > Remaining Estimate: 0h > > > Iceberg 1.5.2 > > [https://iceberg.apache.org/releases/] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13193) Update maven plugins to latest and require minimum of Apache Maven 3.9.6
[
https://issues.apache.org/jira/browse/NIFI-13193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann updated NIFI-13193:
Issue Type: Improvement (was: Task)
> Update maven plugins to latest and require minimum of Apache Maven 3.9.6
>
>
> Key: NIFI-13193
> URL: https://issues.apache.org/jira/browse/NIFI-13193
> Project: Apache NiFi
> Issue Type: Improvement
>Reporter: Joe Witt
>Assignee: Joe Witt
>Priority: Major
> Fix For: 2.0.0-M3
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Apache Maven 3.8.5 is more than two years old already and many plugin updates
> suggest they require at least that. Bumping versions found during plugin
> update scanning and bumping to that minimum maven version.
> {noformat}
> Update Maven Plugins
> org.codehaus.mojo:exec-maven-plugin 3.1.0 -> 3.2.0
> maven-clean-plugin . 3.3.1 -> 3.3.2
> com.github.eirslett:frontend-maven-plugin 1.14.2 -> 1.15.0
> com.coderplus.maven.plugins:copy-rename-maven-plugin . 1.0 -> 1.0.1
> org.codehaus.mojo:license-maven-plugin . 2.2.0 -> 2.4.0
> org.asciidoctor:asciidoctor-maven-plugin ... 2.2.4 -> 2.2.6 (or 3.0.0
> if we want to require maven 3.8.5)
> com.googlecode.maven-download-plugin:download-maven-plugin 1.7.1 -> 1.9.0
> com.github.spotbugs:spotbugs-maven-plugin .. 4.7.3.5 -> 4.8.5.0
> maven-jar-plugin ... 3.3.0 -> 3.4.1
> org.codehaus.mojo:jaxb2-maven-plugin ... 3.1.0 -> 3.2.0
> org.patrodyne.jvnet:hisrc-higherjaxb40-maven-plugin 2.1.1 -> 2.2.0 (require
> maven 3.8.4)
> org.jacoco:jacoco-maven-plugin ... 0.8.11 -> 0.8.12
> io.swagger.core.v3:swagger-maven-plugin-jakarta .. 2.2.20 -> 2.2.21
> io.swagger.codegen.v3:swagger-codegen-maven-plugin 3.0.52 -> 3.0.55
> {noformat}
> Not worth doing now as it causes some failure and we need to run that down.
> io.fabric8:docker-maven-plugin ... 0.43.4 -> 0.44.0
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-13178) Upgrade Apache Mina, UnboundID Ldap, oktthp-digest, reactor-netty, jetbrains-annotations, reactor-core
[ https://issues.apache.org/jira/browse/NIFI-13178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-13178: Issue Type: Improvement (was: Task) > Upgrade Apache Mina, UnboundID Ldap, oktthp-digest, reactor-netty, > jetbrains-annotations, reactor-core > -- > > Key: NIFI-13178 > URL: https://issues.apache.org/jira/browse/NIFI-13178 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Joe Witt >Assignee: Joe Witt >Priority: Major > Fix For: 2.0.0-M3 > > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13179) Bump oauth-oidc-sdk 9.43.4 to 11.10.1
[ https://issues.apache.org/jira/browse/NIFI-13179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-13179: Issue Type: Improvement (was: Task) > Bump oauth-oidc-sdk 9.43.4 to 11.10.1 > - > > Key: NIFI-13179 > URL: https://issues.apache.org/jira/browse/NIFI-13179 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Joe Witt >Assignee: Joe Witt >Priority: Major > Fix For: 2.0.0-M3 > > Time Spent: 0.5h > Remaining Estimate: 0h > > oauth-oidc-sdk 9.43.4 (sep 22) to 11.10.1 (Mar 24) > Seems relatively safe. > https://bitbucket.org/connect2id/oauth-2.0-sdk-with-openid-connect-extensions/src/master/CHANGELOG.txt. > Read all release notes and verified noted dependency changes are already in > our build anyway. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13179) Upgrade oauth-oidc-sdk 9.43.4 to 11.10.1
[ https://issues.apache.org/jira/browse/NIFI-13179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-13179: Summary: Upgrade oauth-oidc-sdk 9.43.4 to 11.10.1 (was: Bump oauth-oidc-sdk 9.43.4 to 11.10.1) > Upgrade oauth-oidc-sdk 9.43.4 to 11.10.1 > > > Key: NIFI-13179 > URL: https://issues.apache.org/jira/browse/NIFI-13179 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Joe Witt >Assignee: Joe Witt >Priority: Major > Fix For: 2.0.0-M3 > > Time Spent: 0.5h > Remaining Estimate: 0h > > oauth-oidc-sdk 9.43.4 (sep 22) to 11.10.1 (Mar 24) > Seems relatively safe. > https://bitbucket.org/connect2id/oauth-2.0-sdk-with-openid-connect-extensions/src/master/CHANGELOG.txt. > Read all release notes and verified noted dependency changes are already in > our build anyway. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13113) upgrade Iceberg from 1.5.0 to 1.5.2
[ https://issues.apache.org/jira/browse/NIFI-13113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-13113: Issue Type: Improvement (was: Task) > upgrade Iceberg from 1.5.0 to 1.5.2 > --- > > Key: NIFI-13113 > URL: https://issues.apache.org/jira/browse/NIFI-13113 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Sean Sullivan >Assignee: Joe Witt >Priority: Minor > Fix For: 2.0.0-M3 > > Time Spent: 1h 20m > Remaining Estimate: 0h > > > Iceberg 1.5.2 > > [https://iceberg.apache.org/releases/] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13007) Remove Apache Solr Components
[ https://issues.apache.org/jira/browse/NIFI-13007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-13007: Summary: Remove Apache Solr Components (was: Remove Solr Components in 2x line) > Remove Apache Solr Components > - > > Key: NIFI-13007 > URL: https://issues.apache.org/jira/browse/NIFI-13007 > Project: Apache NiFi > Issue Type: Task >Reporter: Joe Witt >Assignee: Joe Witt >Priority: Major > Fix For: 2.0.0-M3 > > Time Spent: 20m > Remaining Estimate: 0h > > In NIFI-12998 it was found that with basic dependency hygiene our Solr > components cant even build now. And it turns out they already dont work at > runtime. > In NIFi-13006 we deprecate and in this JIRA we remove. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12906) Upgrade ZooKeeper to 3.9.2
[ https://issues.apache.org/jira/browse/NIFI-12906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-12906: Issue Type: Improvement (was: Task) > Upgrade ZooKeeper to 3.9.2 > -- > > Key: NIFI-12906 > URL: https://issues.apache.org/jira/browse/NIFI-12906 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Joe Witt >Assignee: Joe Witt >Priority: Major > Fix For: 1.26.0, 2.0.0-M3 > > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12907) Update to Groovy 4.0.20
[ https://issues.apache.org/jira/browse/NIFI-12907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-12907: Issue Type: Improvement (was: Task) > Update to Groovy 4.0.20 > --- > > Key: NIFI-12907 > URL: https://issues.apache.org/jira/browse/NIFI-12907 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Joe Witt >Assignee: Joe Witt >Priority: Major > Fix For: 2.0.0-M3 > > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-13211: Decoupling other UIs from exiting NiFi UI [nifi]
mcgilman commented on PR #8828: URL: https://github.com/apache/nifi/pull/8828#issuecomment-2110286911 > the following two files were flagged as missing the standard license header Thanks @exceptionfactory. I see I have some `rat` config that needs updating. New commit coming shortly. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-13228) Jolt Transform UI uses incorrect property names
[ https://issues.apache.org/jira/browse/NIFI-13228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-13228: Affects Version/s: 2.0.0-M2 > Jolt Transform UI uses incorrect property names > --- > > Key: NIFI-13228 > URL: https://issues.apache.org/jira/browse/NIFI-13228 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 2.0.0-M2 >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > NIFI-12554 introducing a breaking change with the property names used by > JoltTransformJSON. The existing custom UI saves the transform and > specification into properties under the previous name. As a result, the > values are stored in dynamic properties. This custom UI needs to be updated > to leverage the new/updated Property names. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-13211: Decoupling other UIs from exiting NiFi UI [nifi]
rfellows commented on PR #8828: URL: https://github.com/apache/nifi/pull/8828#issuecomment-2110255600 will review... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-13228) Jolt Transform UI uses incorrect property names
[ https://issues.apache.org/jira/browse/NIFI-13228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman updated NIFI-13228: --- Status: Patch Available (was: In Progress) > Jolt Transform UI uses incorrect property names > --- > > Key: NIFI-13228 > URL: https://issues.apache.org/jira/browse/NIFI-13228 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > NIFI-12554 introducing a breaking change with the property names used by > JoltTransformJSON. The existing custom UI saves the transform and > specification into properties under the previous name. As a result, the > values are stored in dynamic properties. This custom UI needs to be updated > to leverage the new/updated Property names. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13211) Decouple UIs from old NiFi UI
[ https://issues.apache.org/jira/browse/NIFI-13211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman updated NIFI-13211: --- Status: Patch Available (was: In Progress) > Decouple UIs from old NiFi UI > - > > Key: NIFI-13211 > URL: https://issues.apache.org/jira/browse/NIFI-13211 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > NiFi is comprised of numerous UIs that all get deployed at start up. These > UIs piggybacked on some shared stylesheets, components, and dependencies by > directly loading them through a relative path. In preparation for moving away > from the existing UI, these UIs are being updated to bundle these items > directly. In time, these UIs will all be modernized as well so this is just a > transitional step in that process. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13211) Decouple UIs from old NiFi UI
[ https://issues.apache.org/jira/browse/NIFI-13211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman updated NIFI-13211: --- Description: NiFi is comprised of numerous UIs that all get deployed at start up. These UIs piggybacked on some shared stylesheets, components, and dependencies by directly loading them through a relative path. In preparation for moving away from the existing UI, these UIs are being updated to bundle these items directly. In time, these UIs will all be modernized as well so this is just a transitional step in that process. > Decouple UIs from old NiFi UI > - > > Key: NIFI-13211 > URL: https://issues.apache.org/jira/browse/NIFI-13211 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > NiFi is comprised of numerous UIs that all get deployed at start up. These > UIs piggybacked on some shared stylesheets, components, and dependencies by > directly loading them through a relative path. In preparation for moving away > from the existing UI, these UIs are being updated to bundle these items > directly. In time, these UIs will all be modernized as well so this is just a > transitional step in that process. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-13211) Decouple UIs from old NiFi UI
[ https://issues.apache.org/jira/browse/NIFI-13211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman updated NIFI-13211: --- Summary: Decouple UIs from old NiFi UI (was: Remove dataviewer dependencies on old UI) > Decouple UIs from old NiFi UI > - > > Key: NIFI-13211 > URL: https://issues.apache.org/jira/browse/NIFI-13211 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[PR] NIFI-13211: Decoupling other UIs from exiting NiFi UI [nifi]
mcgilman opened a new pull request, #8828: URL: https://github.com/apache/nifi/pull/8828 NIFI-13211: NiFi is comprised of numerous UIs that all get deployed at start up. These UIs piggybacked on some shared stylesheets, components, and dependencies by directly loading them through a relative path. In preparation for moving away from the existing UI, these UIs are being updated to bundle these items directly. In time, these UIs will all be modernized as well so this is just a transitional step in that process. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-13228) Jolt Transform UI uses incorrect property names
Matt Gilman created NIFI-13228: -- Summary: Jolt Transform UI uses incorrect property names Key: NIFI-13228 URL: https://issues.apache.org/jira/browse/NIFI-13228 Project: Apache NiFi Issue Type: Bug Components: Extensions Reporter: Matt Gilman Assignee: Matt Gilman NIFI-12554 introducing a breaking change with the property names used by JoltTransformJSON. The existing custom UI saves the transform and specification into properties under the previous name. As a result, the values are stored in dynamic properties. This custom UI needs to be updated to leverage the new/updated Property names. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] MINIFICPP-2311 Initial support for recordset readers and writers [nifi-minifi-cpp]
martinzink commented on code in PR #1763:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1763#discussion_r1599894534
##
libminifi/include/controllers/RecordSetWriter.h:
##
@@ -0,0 +1,34 @@
+/**
+* Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#pragma once
+
+#include "core/controller/ControllerService.h"
+
+#include "core/FlowFile.h"
+#include "core/ProcessSession.h"
+#include "Record.h"
+
+namespace org::apache::nifi::minifi::core {
+
+class RecordSetWriter : public controller::ControllerService {
+ public:
+ using ControllerService::ControllerService;
+
+ virtual void write(const RecordSet& record_set, const
std::shared_ptr& flow_file, ProcessSession& session) = 0;
Review Comment:
Unfortunetly we'd need to refactor
ProcessSession::read/ProcessSession::write for that
##
libminifi/include/controllers/RecordSetReader.h:
##
@@ -0,0 +1,39 @@
+/**
+* Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#pragma once
+
+
+#include "core/controller/ControllerService.h"
+
+#include "core/FlowFile.h"
+#include "core/ProcessSession.h"
+#include "Record.h"
+#include "utils/Enum.h"
+#include "utils/ProcessorConfigUtils.h"
+
+
+namespace org::apache::nifi::minifi::core {
+
+class RecordSetReader : public controller::ControllerService {
+ public:
+ using ControllerService::ControllerService;
+
+
+ virtual nonstd::expected read(const
std::shared_ptr& flow_file, ProcessSession& session) = 0;
Review Comment:
Unfortunetly we'd need to refactor
ProcessSession::read/ProcessSession::write for that
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Created] (NIFI-13227) Bump aspectj and mockito and various dependencies
Joe Witt created NIFI-13227: --- Summary: Bump aspectj and mockito and various dependencies Key: NIFI-13227 URL: https://issues.apache.org/jira/browse/NIFI-13227 Project: Apache NiFi Issue Type: Task Reporter: Joe Witt Assignee: Joe Witt aspectj 1.9.22.1 mockito 5.12.0 software.amazon.awssdk 2.25.51 com.amazonaws 1.12.721 com.slack.api 1.39.2 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12812) CaptureChangeMySQL consumes binlog backlog
[
https://issues.apache.org/jira/browse/NIFI-12812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17846189#comment-17846189
]
Joe Witt commented on NIFI-12812:
-
User xiyang from apache slack states:
I used CaptureChangeMySQL 1.23.3 for the consumption of binlogs. In the long
running process, we found that there will often be stagnation (we did the
monitoring, extracted the binlog position of the main database and the
comparison of binlog.position), Delayed consumption or backlog occurs; Or in
the case of a large number of updates and inserts, binlog.filename may be
backlogged by 10 or more. Is there any way to optimize this, thanks!
We analyze that setting Events Per FlowFile too small will cause binlog
consumption backlog, setting Events Per FlowFile too small will cause data to
be delayed indefinitely, which is not acceptable in cdc. When the Events Per
FlowFile is set to 150, the data backlog can reach 2 minutes, which is a
serious problem that we hope can be fixed
After nearly two months of observation, we found that there will be a
phenomenon when the consumption binlog is stuck, the running thread identifier
on the CaptureChangeMySQL processor is always 1, theoretically the running
thread identifier should disappear soon, when we find the binlog backlog, the
running thread identifier will always be there, at this time, When we look at
CaptureChangeMySQL consumption capabilities, it's very slow. I'm not sure
what's wrong with it. When we stop the processor, or replace it with another
processor, it runs for a long time, which confuses me.
CPU:
Architecture: x86_64
CPU op-mode(s):32-bit, 64-bit
Byte Order:Little Endian
CPU(s):16
On-line CPU(s) list: 0-15
Thread(s) per core:2
Core(s) per socket:8
Socket(s): 1
NUMA node(s): 1
Vendor ID: AuthenticAMD
CPU family:25
Model: 1
Model name:AMD EPYC 7T83 64-Core Processor
Stepping: 1
CPU MHz: 2545.218
BogoMIPS: 5090.43
Hypervisor vendor: KVM
Virtualization type: full
L1d cache: 32K
L1i cache: 32K
L2 cache: 512K
L3 cache: 32768K
NUMA node0 CPU(s): 0-15
memory
32G
java.arg.2=-Xms1g
java.arg.3=-Xmx16g
disk
200G SSD * 2
--NiFi BinLog monitor alarm --
NiFi data synchronization is abnormal!!
Corresponding error message type: 2
Corresponding error message: binlog Position backlog absolute value exceeds
configuration value :(1000), backlog more than 10 times, please check!
Processor Id: f14230db-ba78-106d-973f-e90bd03f0a28
Processor Name: Advertising link -cdc
mysql: mysql-bin.262489
Corresponding point in the processor: mysql-bin.262489
Backlog value: 99644822
Current error accumulation: 11
Notice time: 2024-04-24 00:18:00
This is yesterday's alarm information, let me explain: the binlog file of the
current processor and the database is the same, the backlog value is 99644822,
it will be compared every 1 minute, if more than 1000, +1, when more than
10 times, send alarm information; We can see that the current backlog is 11
times, which is 11 minutes, and the consumption rate of nifi has not caught up
with the production rate of mysql. But when we stop the processor and change
the value of Events Per FlowFile symbolically (from 5 to 10, or whatever), it
runs normally for a while. Until the next backlog occurs
I initially suspected a thread related problem, but I'm not sure which step,
when a processor will be called to run in NIFI, when the UI shows a thread
identifier on the processor, this processor should be running at this time, but
when the thread identifier in UI is always 1, where will the processor be
stuck? If we solve this problem, we will find the truth. (Forgive me if I don't
know much about how NIFI works, or if it has anything to do with Penalty
Duration and Yield Duration.)
@Joe Witt
The problem is repeated, and the relevant log is obtained in time. Let me
describe the file. The file with restart-after is the log information of the
restart after the problem (restarting the processor, the task is normal), and
the file without restart-after is the log information obtained when the problem
occurs. Problems of the database instance is
pxc-hzr66p3487fwri.polarx.rds.aliyuncs.com, seemingly a lock, but I am not sure
what causes it.(The number after each file, representing minutes, is executed
every minute)
xiyang
19 hours ago
public void onEvent(Event event) {
RawBinlogEvent ep = new RawBinlogEvent(event, client.getBinlogFilename());
try {
while (!stopNow.get()) {
if (queue.offer(ep, QUEUE_OFFER_TIMEOUT_MSEC,
TimeUnit.MILLISECONDS)) {
return;
}
}
logger.info("Stopped while waiting to enqueue event");
} catch
