[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1671: MINIFICPP-1891 Integrate google clang-tidy checks in CI
lordgamez commented on code in PR #1671:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1671#discussion_r1341006225
##
extensions/expression-language/tests/ExpressionLanguageTests.cpp:
##
@@ -1321,13 +1321,13 @@ TEST_CASE("Now Date", "[expressionNowDate]") {
TEST_CASE("Parse RFC3339 with Expression Language toDate") {
using date::sys_days;
using org::apache::nifi::minifi::utils::timeutils::parseRfc3339;
- using namespace date::literals;
+ using namespace date::literals; // NOLINT(google-build-using-namespace)
using namespace std::literals::chrono_literals;
using std::chrono::milliseconds;
- milliseconds expected_second =
std::chrono::floor((sys_days(2023_y / 03 / 01) + 19h + 04min +
55s).time_since_epoch());
- milliseconds expected_tenth_second =
std::chrono::floor((sys_days(2023_y / 03 / 01) + 19h + 04min +
55s + 100ms).time_since_epoch());
- milliseconds expected_milli_second =
std::chrono::floor((sys_days(2023_y / 03 / 01) + 19h + 04min +
55s + 190ms).time_since_epoch());
+ milliseconds expected_second =
std::chrono::floor((sys_days{2023_y / 03 / 01} + 19h + 04min +
55s).time_since_epoch()); // NOLINT(whitespace/braces)
+ milliseconds expected_tenth_second =
std::chrono::floor((sys_days{2023_y / 03 / 01} + 19h + 04min +
55s + 100ms).time_since_epoch()); // NOLINT(whitespace/braces)
+ milliseconds expected_milli_second =
std::chrono::floor((sys_days{2023_y / 03 / 01} + 19h + 04min +
55s + 190ms).time_since_epoch()); // NOLINT(whitespace/braces)
Review Comment:
I think the clang-tidy check returned false positive here as it recognized
this call as a C-style cast and warned for it, this change was meant to avoid
that warning.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1671: MINIFICPP-1891 Integrate google clang-tidy checks in CI
lordgamez commented on code in PR #1671: URL: https://github.com/apache/nifi-minifi-cpp/pull/1671#discussion_r1341008576 ## extensions/standard-processors/processors/PutTCP.cpp: ## @@ -215,7 +215,7 @@ asio::awaitable ConnectionHandler::establishNewConn continue; } if (max_size_of_socket_send_buffer_) - socket.lowest_layer().set_option(TcpSocket::send_buffer_size(*max_size_of_socket_send_buffer_)); + socket.lowest_layer().set_option(TcpSocket::send_buffer_size(*max_size_of_socket_send_buffer_)); // NOLINT(google-readability-casting) Review Comment: Same false positive, clang-tidy recognizing `TcpSocket::send_buffer_size()` call as a C-style cast -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1671: MINIFICPP-1891 Integrate google clang-tidy checks in CI
lordgamez commented on code in PR #1671:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1671#discussion_r1341013782
##
extensions/systemd/tests/ConsumeJournaldTest.cpp:
##
@@ -90,7 +90,7 @@ struct TestJournal final : libwrapper::Journal {
}
int getCursor(gsl::owner* const cursor_out) noexcept override {
-*cursor_out = gsl::owner(strdup(std::to_string(consumed).c_str()));
+*cursor_out = strdup(std::to_string(consumed).c_str()); //
NOLINT(cppcoreguidelines-owning-memory)
Review Comment:
As the `cursor_out` parameter is already a `gsl::owner` type the cast and
according to [these
examples](https://clang.llvm.org/extra/clang-tidy/checks/cppcoreguidelines/owning-memory.html)
that should be enough to mark the returned pointer's onwership. The previous
annotation was marked as a C-style cast, but I think the current
`cppcoreguidelines-owning-memory` warning is also a false positive so it was
suppressed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1671: MINIFICPP-1891 Integrate google clang-tidy checks in CI
lordgamez commented on code in PR #1671:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1671#discussion_r1341017929
##
extensions/sftp/client/SFTPClient.cpp:
##
@@ -83,7 +83,7 @@ static const char* sftp_strerror(unsigned long err) { //
NOLINT(runtime/int) un
}
}
-static SFTPError libssh2_sftp_error_to_sftp_error(unsigned long
libssh2_sftp_error) { // NOLINT(runtime/int) unsigned long comes from libssh2
API
+static SFTPError libssh2_sftp_error_to_sftp_error(unsigned long
libssh2_sftp_error) { // NOLINT(runtime/int,google-runtime-int) unsigned long
comes from libssh2 API
Review Comment:
There were instances for both checks where only one of them gave a warning
for this issue, so I kept both of them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1671: MINIFICPP-1891 Integrate google clang-tidy checks in CI
lordgamez commented on code in PR #1671:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1671#discussion_r1341020326
##
extensions/systemd/libwrapper/DlopenWrapper.cpp:
##
@@ -72,7 +72,7 @@ class DlopenJournal : public Journal {
template
F loadSymbol(const char* const symbol_name) {
// The cast below is supported by POSIX platforms.
https://stackoverflow.com/a/1096349
-F const symbol = (F)dlsym(libhandle_.get(), symbol_name);
+F const symbol = reinterpret_cast(dlsym(libhandle_.get(), symbol_name));
Review Comment:
Good point, updated in 9d15d87a77f8e03e0863f8910fc540d9665b9d8f
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1669: MINIFICPP-2210 Add C2 debug command to MiNiFi Controller
lordgamez commented on code in PR #1669:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1669#discussion_r1341030499
##
libminifi/include/utils/file/ArchiveUtils.h:
##
@@ -0,0 +1,31 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#pragma once
+
+#include
+#include
+#include
+
+#include "io/ArchiveStream.h"
+#include "utils/expected.h"
+#include "io/BufferStream.h"
+
+namespace org::apache::nifi::minifi::utils::archive {
+
+nonstd::expected, std::string>
createArchive(std::map>& files,
const std::shared_ptr& logger);
Review Comment:
I see what you mean, but I don't think it is likely that our utils will be
used by other projects, so I would only think in the scope of our project. If
we think outside the scope of our current requirements that usually leads to
premature generalization which is also a bad practice. But I don't want to go
into philosophical discussions, so I would just leave the current helper in the
C2Utils :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Created] (MINIFICPP-2234) C2JstackTest transiently fails on Windows
Gábor Gyimesi created MINIFICPP-2234: Summary: C2JstackTest transiently fails on Windows Key: MINIFICPP-2234 URL: https://issues.apache.org/jira/browse/MINIFICPP-2234 Project: Apache NiFi MiNiFi C++ Issue Type: Improvement Reporter: Gábor Gyimesi Attachments: C2JstackTest_failure_windows.log Logs in attachment -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2234) C2JstackTest transiently fails on Windows
[ https://issues.apache.org/jira/browse/MINIFICPP-2234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gábor Gyimesi updated MINIFICPP-2234: - Issue Type: Bug (was: Improvement) > C2JstackTest transiently fails on Windows > - > > Key: MINIFICPP-2234 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2234 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Gábor Gyimesi >Priority: Major > Attachments: C2JstackTest_failure_windows.log > > > Logs in attachment -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2234) C2JstackTest transiently fails on Windows
[ https://issues.apache.org/jira/browse/MINIFICPP-2234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gábor Gyimesi updated MINIFICPP-2234: - Epic Link: MINIFICPP-1324 > C2JstackTest transiently fails on Windows > - > > Key: MINIFICPP-2234 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2234 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Gábor Gyimesi >Priority: Major > Attachments: C2JstackTest_failure_windows.log > > > Logs in attachment -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (MINIFICPP-2235) ControllerTests sometimes segfault in CI
Gábor Gyimesi created MINIFICPP-2235: Summary: ControllerTests sometimes segfault in CI Key: MINIFICPP-2235 URL: https://issues.apache.org/jira/browse/MINIFICPP-2235 Project: Apache NiFi MiNiFi C++ Issue Type: Bug Reporter: Gábor Gyimesi Attachments: ControllerTests_segfault.log ControllerTests sometimes segfault on the test case "TestClear". Logs attached with more info. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (MINIFICPP-2236) SchedulingAgentTests sometimes times out in CI
Gábor Gyimesi created MINIFICPP-2236: Summary: SchedulingAgentTests sometimes times out in CI Key: MINIFICPP-2236 URL: https://issues.apache.org/jira/browse/MINIFICPP-2236 Project: Apache NiFi MiNiFi C++ Issue Type: Bug Reporter: Gábor Gyimesi Attachments: SchedulingAgentTests_timeout.log Logs attached for more info. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2236) SchedulingAgentTests sometimes time out in CI
[ https://issues.apache.org/jira/browse/MINIFICPP-2236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gábor Gyimesi updated MINIFICPP-2236: - Summary: SchedulingAgentTests sometimes time out in CI (was: SchedulingAgentTests sometimes times out in CI) > SchedulingAgentTests sometimes time out in CI > - > > Key: MINIFICPP-2236 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2236 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Gábor Gyimesi >Priority: Major > Attachments: SchedulingAgentTests_timeout.log > > > Logs attached for more info. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2237) C2DescribeCoreComponentStateTest sometimes fail ON Windows
[ https://issues.apache.org/jira/browse/MINIFICPP-2237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gábor Gyimesi updated MINIFICPP-2237: - Summary: C2DescribeCoreComponentStateTest sometimes fail ON Windows (was: C2DescribeCoreComponentStateTest sometimes fail in Windows) > C2DescribeCoreComponentStateTest sometimes fail ON Windows > -- > > Key: MINIFICPP-2237 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2237 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Gábor Gyimesi >Priority: Major > Attachments: C2DescribeCoreComponentStateTest_failure.log > > > Logs attached with more info. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2237) C2DescribeCoreComponentStateTest sometimes fail on Windows
[ https://issues.apache.org/jira/browse/MINIFICPP-2237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gábor Gyimesi updated MINIFICPP-2237: - Summary: C2DescribeCoreComponentStateTest sometimes fail on Windows (was: C2DescribeCoreComponentStateTest sometimes fail ON Windows) > C2DescribeCoreComponentStateTest sometimes fail on Windows > -- > > Key: MINIFICPP-2237 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2237 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Gábor Gyimesi >Priority: Major > Attachments: C2DescribeCoreComponentStateTest_failure.log > > > Logs attached with more info. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (MINIFICPP-2237) C2DescribeCoreComponentStateTest sometimes fail in Windows
Gábor Gyimesi created MINIFICPP-2237: Summary: C2DescribeCoreComponentStateTest sometimes fail in Windows Key: MINIFICPP-2237 URL: https://issues.apache.org/jira/browse/MINIFICPP-2237 Project: Apache NiFi MiNiFi C++ Issue Type: Bug Reporter: Gábor Gyimesi Attachments: C2DescribeCoreComponentStateTest_failure.log Logs attached with more info. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2237) C2DescribeCoreComponentStateTest sometimes fail on Windows
[ https://issues.apache.org/jira/browse/MINIFICPP-2237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gábor Gyimesi updated MINIFICPP-2237: - Description: Assertionfailed:root["corecomponentstate"].HasMember(uuid),fileD:\a\nifi-minifi-cpp\nifi-minifi-cpp\extensions\http-curl\tests\C2DescribeCoreComponentStateTest.cpp,line78 Logs attached with more info. was:Logs attached with more info. > C2DescribeCoreComponentStateTest sometimes fail on Windows > -- > > Key: MINIFICPP-2237 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2237 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Gábor Gyimesi >Priority: Major > Attachments: C2DescribeCoreComponentStateTest_failure.log > > > Assertionfailed:root["corecomponentstate"].HasMember(uuid),fileD:\a\nifi-minifi-cpp\nifi-minifi-cpp\extensions\http-curl\tests\C2DescribeCoreComponentStateTest.cpp,line78 > Logs attached with more info. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2234) C2JstackTest transiently fails on Windows
[
https://issues.apache.org/jira/browse/MINIFICPP-2234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gábor Gyimesi updated MINIFICPP-2234:
-
Description:
Assertion failed: root.HasMember("Flowcontroller threadpool #0"), file
D:\a\nifi-minifi-cpp\nifi-minifi-cpp\extensions\http-curl\tests\C2JstackTest.cpp,
line 39
Logs in attachment
was:Logs in attachment
> C2JstackTest transiently fails on Windows
> -
>
> Key: MINIFICPP-2234
> URL: https://issues.apache.org/jira/browse/MINIFICPP-2234
> Project: Apache NiFi MiNiFi C++
> Issue Type: Bug
>Reporter: Gábor Gyimesi
>Priority: Major
> Attachments: C2JstackTest_failure_windows.log
>
>
> Assertion failed: root.HasMember("Flowcontroller threadpool #0"), file
> D:\a\nifi-minifi-cpp\nifi-minifi-cpp\extensions\http-curl\tests\C2JstackTest.cpp,
> line 39
> Logs in attachment
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2237) C2DescribeCoreComponentStateTest sometimes fail on Windows
[ https://issues.apache.org/jira/browse/MINIFICPP-2237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gábor Gyimesi updated MINIFICPP-2237: - Description: Assertion failed: root["corecomponentstate"].HasMember(uuid), file D:\a\nifi-minifi-cpp\nifi-minifi-cpp\extensions\http-curl\tests\C2DescribeCoreComponentStateTest.cpp, line 78 Logs attached with more info. was: Assertionfailed:root["corecomponentstate"].HasMember(uuid),fileD:\a\nifi-minifi-cpp\nifi-minifi-cpp\extensions\http-curl\tests\C2DescribeCoreComponentStateTest.cpp,line78 Logs attached with more info. > C2DescribeCoreComponentStateTest sometimes fail on Windows > -- > > Key: MINIFICPP-2237 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2237 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Gábor Gyimesi >Priority: Major > Attachments: C2DescribeCoreComponentStateTest_failure.log > > > Assertion failed: root["corecomponentstate"].HasMember(uuid), file > D:\a\nifi-minifi-cpp\nifi-minifi-cpp\extensions\http-curl\tests\C2DescribeCoreComponentStateTest.cpp, > line 78 > Logs attached with more info. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi-minifi-cpp] martinzink commented on a diff in pull request #1670: MINIFICPP-2220 Change core::logging::Logger to use fmt format
martinzink commented on code in PR #1670:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1670#discussion_r1341084547
##
libminifi/src/sitetosite/RawSocketProtocol.cpp:
##
@@ -435,8 +431,8 @@ int RawSiteToSiteClient::readRespond(const
std::shared_ptr &transac
return readResponse(transaction, code, message);
}
-int RawSiteToSiteClient::writeRespond(const std::shared_ptr
&transaction, RespondCode code, std::string message) {
- return writeResponse(transaction, code, std::move(message));
+int RawSiteToSiteClient::writeRespond(const std::shared_ptr
&transaction, RespondCode code, const std::string& message) {
+ return writeResponse(transaction, code, message);
Review Comment:
Changing that properly would lead further than I would like in this PR (this
leads to a call to `size_t OutputStream::write(const std::string& str, bool
widen = false);`)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] simonbence commented on a diff in pull request #7661: NIFI-11772 Removing flow.xml support
simonbence commented on code in PR #7661:
URL: https://github.com/apache/nifi/pull/7661#discussion_r1341127238
##
nifi-system-tests/nifi-system-test-suite/src/test/java/org/apache/nifi/tests/system/clustering/JoinClusterWithDifferentFlow.java:
##
@@ -106,100 +95,57 @@ public void testStartupWithDifferentFlow() throws
IOException, NiFiClientExcepti
node2.stop();
final File node2ConfDir = new File(node2.getInstanceDirectory(),
"conf");
-final File flowXmlFile = new File(node2ConfDir, "flow.xml.gz");
-Files.deleteIfExists(flowXmlFile.toPath());
-
Files.copy(Paths.get("src/test/resources/flows/mismatched-flows/flow2.xml.gz"),
flowXmlFile.toPath());
+final File flowJsonFile = new File(node2ConfDir, "flow.json.gz");
+Files.deleteIfExists(flowJsonFile.toPath());
+
Files.copy(Paths.get("src/test/resources/flows/mismatched-flows/flow2.json.gz"),
flowJsonFile.toPath());
node2.start(true);
-final File backupFile = getBackupFile(node2ConfDir);
-final NodeDTO node2Dto = getNodeDtoByApiPort(5672);
+while
(getNifiClient().getControllerClient().getNodes().getCluster().getNodes().stream().filter(n
-> n.getStatus().equals("CONNECTED")).count() != 2) {
+Thread.sleep(3000);
+}
+final File backupFile = getBackupFile(node2ConfDir);
verifyFlowContentsOnDisk(backupFile);
-disconnectNode(node2Dto);
verifyInMemoryFlowContents();
-
-// Reconnect the node so that we can properly shutdown
-reconnectNode(node2Dto);
Review Comment:
Thanks! I would believe the previous logic did not work properly as it was
before remoeval (it did a REST call against the endpoint
`cluster/nodes/${nodeId}` which behaves based on the accompanying `NodeEntity`
/ `NodeDTO`, but the status was not set to be DISCONNECTING, so in practice I
think it did not disconnect), but thanks for the context! I will look how it
behaves this way
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] martinzink opened a new pull request, #1672: MINIFICPP-2233 Upgrade bustache to fix gcc13 compatibility
martinzink opened a new pull request, #1672: URL: https://github.com/apache/nifi-minifi-cpp/pull/1672 We should use the latest commit which includes https://github.com/jamboree/bustache/pull/35 This fixes gcc13 compatibility https://github.com/apache/nifi-minifi-cpp/actions/runs/6327016378 vs https://github.com/martinzink/nifi-minifi-cpp/actions/runs/6348941850 --- Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFICPP- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically main)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check GitHub Actions CI results for build issues and submit an update to your PR as soon as possible. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] dan-s1 commented on pull request #7782: NIFI-12116 Configured ObjectMapper to not allow multiline JSON.
dan-s1 commented on PR #7782: URL: https://github.com/apache/nifi/pull/7782#issuecomment-1740814657 @r-vandenbos I would like to go with @ChrisSamo632 suggestion of two processors. Can you please submit a ticket for your work but creating a new processor that can handle records and I will fix ValidateJson with @markap14 original suggestion of a property. As I mentioned before, this seems to align with what NIFI already has for Jolt `JoltTransformJSON` which handles single documents and `JoltTransformRecord` which can handle multiple documents. Hopefully now there will be ValidateJson and ValidateJsonRecord. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Comment Edited] (NIFI-11084) Character/text data "mis-identified" by IdentifyMimeType processor
[
https://issues.apache.org/jira/browse/NIFI-11084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17768114#comment-17768114
]
Nissim Shiman edited comment on NIFI-11084 at 9/29/23 12:36 PM:
Once NIFI-11463 is complete, a custom rule can be made for this [1], which will
hit before reaching the tika rules. This would elevate csv rule processing
above image/x-portable-graymap (Thank you [~dstieg1] for pointing this out to
me].
[1] Config Strategy would be set to "Add"
and Config Body would be:
{code:java}
{code}
was (Author: nissim shiman):
Once NIFI-11463 is complete, a custom rule can be made for this, which will hit
before reaching the tika rules. (Thank you [~dstieg1] for pointing this out to
me].
> Character/text data "mis-identified" by IdentifyMimeType processor
> --
>
> Key: NIFI-11084
> URL: https://issues.apache.org/jira/browse/NIFI-11084
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.15.2, 1.19.1
> Environment: Windows Server 2019, Java 11.0.17
>Reporter: Mark Ward
>Priority: Minor
> Attachments: mime_type_mis-id_file.csv
>
>
> When *IdentifyMimeType* is presented with a text file with a `.csv` extension
> and the first two characters of the content as `P2`, the processor
> mis-identifies the mime.extension as `pgm` and mime.type as
> `image/x-portable-graymap`.
> The processor's *Use Filename In Detection* property is set to `true`.
> An example file is attached and the following flow and be used to reproduce:
> *GetFile* > *IdentifyMimeType* where the outputted flowfile's attributes can
> be inspected.
> This has been tested on NiFi versions `1.15.2` and `1.19.1` both running on a
> Window's Server 2019 instance.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-12095) Increase default Xmx to 1g
[ https://issues.apache.org/jira/browse/NIFI-12095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17770422#comment-17770422 ] ASF subversion and git services commented on NIFI-12095: Commit 5c1ca14d330c928703326e46a1e24eaa6ae9022b in nifi's branch refs/heads/support/nifi-1.x from Nandor Soma Abonyi [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=5c1ca14d33 ] NIFI-12095 This closes #7762. Increase default Xmx to 1g Signed-off-by: Joseph Witt > Increase default Xmx to 1g > -- > > Key: NIFI-12095 > URL: https://issues.apache.org/jira/browse/NIFI-12095 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Nandor Soma Abonyi >Assignee: Nandor Soma Abonyi >Priority: Minor > Fix For: 2.0.0 > > Time Spent: 1h > Remaining Estimate: 0h > > The default 512m dates back many years, and sometimes, a simple flow can > cause OutOfMemoryError. This should be fine-tuned for production for sure, > but increasing the default to 1g can reduce the inconvenience new users might > experience. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-12148) NiFi Hive3Streaming memory leak
Dmytro Firsov created NIFI-12148: Summary: NiFi Hive3Streaming memory leak Key: NIFI-12148 URL: https://issues.apache.org/jira/browse/NIFI-12148 Project: Apache NiFi Issue Type: Bug Affects Versions: 1.23.2, 1.19.1 Reporter: Dmytro Firsov Attachments: image-2023-09-29-15-46-17-207.png, image-2023-09-29-15-48-15-931.png JVM heap size is constantly grows when using Hive3Streaming processor, depending on flowfile rate (higher rate -> faster growing). Even when processor is stopped (or deleted) memory is not freed. Without Hive3Streaming processor - no leaks. Hadoop: 3.3.6, Hive 4.0.0-beta-1, NiFi - 1.23.2 Also tried: Hadoop 3.3.6, Hive 3.1.2, NiFi - 1.19.1 - same issue. I create jvm memory dump with Jmap and with Eclipse Memory Analyzer these results: !image-2023-09-29-15-46-17-207.png! !image-2023-09-29-15-48-15-931.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] bbende merged pull request #7807: NIFI-12146: Updating nifi cli command list in documentation
bbende merged PR #7807: URL: https://github.com/apache/nifi/pull/7807 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-12146) Update nifi cli command list
[ https://issues.apache.org/jira/browse/NIFI-12146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17770429#comment-17770429 ] ASF subversion and git services commented on NIFI-12146: Commit 71adc196b38c701b4015afc61106a4986c6d18f0 in nifi's branch refs/heads/main from Joe Gresock [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=71adc196b3 ] NIFI-12146: Updating nifi cli command list in documentation (#7807) > Update nifi cli command list > > > Key: NIFI-12146 > URL: https://issues.apache.org/jira/browse/NIFI-12146 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Joe Gresock >Assignee: Joe Gresock >Priority: Trivial > Time Spent: 20m > Remaining Estimate: 0h > > A couple of the commands in the listing of the toolkit documentation don't > align with actual commands. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12146) Update nifi cli command list
[ https://issues.apache.org/jira/browse/NIFI-12146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17770431#comment-17770431 ] ASF subversion and git services commented on NIFI-12146: Commit 74d1040cadfeb2debe2d8a7199f182a609f7b730 in nifi's branch refs/heads/support/nifi-1.x from Joe Gresock [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=74d1040cad ] NIFI-12146: Updating nifi cli command list in documentation (#7807) > Update nifi cli command list > > > Key: NIFI-12146 > URL: https://issues.apache.org/jira/browse/NIFI-12146 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Joe Gresock >Assignee: Joe Gresock >Priority: Trivial > Time Spent: 20m > Remaining Estimate: 0h > > A couple of the commands in the listing of the toolkit documentation don't > align with actual commands. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-12146) Update nifi cli command list
[ https://issues.apache.org/jira/browse/NIFI-12146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Bende resolved NIFI-12146. Fix Version/s: 2.0.0 1.24.0 Resolution: Fixed > Update nifi cli command list > > > Key: NIFI-12146 > URL: https://issues.apache.org/jira/browse/NIFI-12146 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Joe Gresock >Assignee: Joe Gresock >Priority: Trivial > Fix For: 2.0.0, 1.24.0 > > Time Spent: 20m > Remaining Estimate: 0h > > A couple of the commands in the listing of the toolkit documentation don't > align with actual commands. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11841) Nifi in cluster setup doesn't stop within the graceful shutdown period
[
https://issues.apache.org/jira/browse/NIFI-11841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17770444#comment-17770444
]
Michael W Moser commented on NIFI-11841:
[~dheerajjoshim] It looks like one of your three nodes (nifi-2) did shutdown
after 10 seconds and did not have to be killed after 20 seconds. This is
suspicious, because if there was a software problem I expect all three to
behave the same. On what kind of environment are you running NiFi? 3 separate
hardware machines? 3 VMs on a cloud provider?
> Nifi in cluster setup doesn't stop within the graceful shutdown period
> --
>
> Key: NIFI-11841
> URL: https://issues.apache.org/jira/browse/NIFI-11841
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.22.0
>Reporter: Dheeraj Joshi
>Priority: Major
> Attachments: Logs.zip, bootstrap.conf, nifi-app.log,
> nifi-bootstrap.log, nifi.properties, zookeeper.properties
>
>
> Bootstrap logs show the following logs when one of the nodes in a 3-node
> cluster is stopped
> {code:java}
> 2023-07-20 11:10:49,847 INFO [main] org.apache.nifi.bootstrap.Command
> Launched Apache NiFi with Process ID 18579
> 2023-07-20 11:10:50,475 INFO [NiFi Bootstrap Command Listener]
> org.apache.nifi.bootstrap.RunNiFi Apache NiFi now running and listening for
> Bootstrap requests on port 64739
> 2023-07-20 11:12:15,723 INFO [main] o.a.n.b.NotificationServiceManager
> Successfully loaded the following 0 services: []
> 2023-07-20 11:12:15,725 INFO [main] org.apache.nifi.bootstrap.RunNiFi
> Registered no Notification Services for Notification Type NIFI_STARTED
> 2023-07-20 11:12:15,725 INFO [main] org.apache.nifi.bootstrap.RunNiFi
> Registered no Notification Services for Notification Type NIFI_STOPPED
> 2023-07-20 11:12:15,725 INFO [main] org.apache.nifi.bootstrap.RunNiFi
> Registered no Notification Services for Notification Type NIFI_DIED
> 2023-07-20 11:12:15,750 INFO [main] org.apache.nifi.bootstrap.Command Apache
> NiFi has accepted the Shutdown Command and is shutting down now
> 2023-07-20 11:12:15,859 INFO [main] org.apache.nifi.bootstrap.Command NiFi
> PID [18579] shutdown in progress...
> 2023-07-20 11:12:17,896 INFO [main] org.apache.nifi.bootstrap.Command NiFi
> PID [18579] shutdown in progress...
> 2023-07-20 11:12:19,919 INFO [main] org.apache.nifi.bootstrap.Command NiFi
> PID [18579] shutdown in progress...{code}
> The shutdown in progress message gets repeated till
> {code:java}
> graceful.shutdown.seconds{code}
> is elapsed.
> Nifi app logs print socket exception during the shutdown but eventually shuts
> down the application server server
> {code:java}
> 2023-07-20 11:12:15,750 INFO [pool-2-thread-2]
> org.apache.nifi.BootstrapListener Received SHUTDOWN request from Bootstrap
> 2023-07-20 11:12:15,750 INFO [pool-2-thread-2] org.apache.nifi.NiFi
> Application Server shutdown started
> 2023-07-20 11:12:15,755 INFO [pool-2-thread-2]
> o.eclipse.jetty.server.AbstractConnector Stopped
> ServerConnector@2407a36c{HTTP/1.1, (http/1.1)}
> {127.0.0.1:8080}
> 2023-07-20 11:12:15,756 INFO [pool-2-thread-2]
> org.eclipse.jetty.server.session node0 Stopped scavenging
> 2023-07-20 11:12:15,769 INFO [pool-2-thread-2]
> o.e.jetty.server.handler.ContextHandler Stopped
> o.e.j.w.WebAppContext@7cfb8e98{nifi-error,/,null,STOPPED}
> {./work/nar/extensions/nifi-server-nar-1.22.0.nar-unpacked/NAR-INF/bundled-dependencies/nifi-web-error-1.22.0.war}
> 2023-07-20 11:12:15,782 INFO [pool-2-thread-2]
> o.e.jetty.server.handler.ContextHandler Stopped
> o.e.j.w.WebAppContext@2aa5bd48{nifi-docs,/nifi-docs,null,STOPPED}
> {./work/nar/extensions/nifi-server-nar-1.22.0.nar-unpacked/NAR-INF/bundled-dependencies/nifi-web-docs-1.22.0.war}
> 2023-07-20 11:12:15,798 INFO [pool-2-thread-2]
> o.e.jetty.server.handler.ContextHandler Stopped
> o.e.j.w.WebAppContext@133aacbe{nifi-content-viewer,/nifi-content-viewer,null,STOPPED}
> {./work/nar/extensions/nifi-server-nar-1.22.0.nar-unpacked/NAR-INF/bundled-dependencies/nifi-web-content-viewer-1.22.0.war}
> 2023-07-20 11:12:15,803 INFO [pool-2-thread-2]
> o.a.n.w.c.ApplicationStartupContextListener Flow Service shutdown started
> 2023-07-20 11:12:15,803 INFO [pool-2-thread-2]
> o.a.n.c.c.node.NodeClusterCoordinator Status of 127.0.0.1:8080 changed from
> NodeConnectionStatus[nodeId=127.0.0.1:8080, state=CONNECTED, updateId=50] to
> NodeConnectionStatus[nodeId=127.0.0.1:8080, state=DISCONNECTED, Disconnect
> Code=Node was Shutdown, Disconnect Reason=Node was Shutdown, updateId=50]
> 2023-07-20 11:12:15,809 INFO [pool-2-thread-2]
> o.a.n.c.c.node.NodeClusterCoordinator Successfully notified other nodes that
> I am shutting down
> 2023-07-20 11:12:15,811 INFO [Curator-Framework-0]
[jira] [Created] (NIFI-12149) Separate RedisUtils into a separate module
Bryan Bende created NIFI-12149: -- Summary: Separate RedisUtils into a separate module Key: NIFI-12149 URL: https://issues.apache.org/jira/browse/NIFI-12149 Project: Apache NiFi Issue Type: Improvement Affects Versions: 1.23.2 Reporter: Bryan Bende Assignee: Bryan Bende Currently the RedisUtils class and some other utility code is embedded into the nifi-redis-extensions module which makes it not easy to reuse anywhere since depending on it would bring in extensions. It should be in a separate module like nifi-redis-utils. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (NIFI-11782) NPE when adding Snippet with label into Process Group
[
https://issues.apache.org/jira/browse/NIFI-11782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nissim Shiman reassigned NIFI-11782:
Assignee: Nissim Shiman
> NPE when adding Snippet with label into Process Group
> -
>
> Key: NIFI-11782
> URL: https://issues.apache.org/jira/browse/NIFI-11782
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.21.0, 1.22.0
>Reporter: Pierre Villard
>Assignee: Nissim Shiman
>Priority: Major
> Fix For: 1.latest, 2.latest
>
>
> When adding to a Process Group a snippet that contains a label
> {code:java}
> 2023-07-06 15:35:47,575 ERROR [NiFi Web Server-222]
> o.a.nifi.web.api.config.ThrowableMapper An unexpected error has occurred:
> java.lang.NullPointerException: Cannot invoke
> "org.apache.nifi.web.dao.LabelDAO.getLabel(String)" because "this.labelDAO"
> is null. Returning Internal Server Error response.
> java.lang.NullPointerException: Cannot invoke
> "org.apache.nifi.web.dao.LabelDAO.getLabel(String)" because "this.labelDAO"
> is null
> at
> org.apache.nifi.audit.SnippetAuditor.updateSnippetAdvice(SnippetAuditor.java:336)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:568)
> at
> org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:634)
> at
> org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:624)
> at
> org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:72)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
> at
> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
> {code}
> This seems to be caused by NIFI-11134.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi] exceptionfactory commented on pull request #7803: NIFI-12100_deprecate Added DeprecationNotice annotation to the ConvertExcelToCSVProcessor
exceptionfactory commented on PR #7803: URL: https://github.com/apache/nifi/pull/7803#issuecomment-1740913472 > @exceptionfactory Does that mean I have to create a branch off of support/nifi-1.x with my change and then create a new PR or does it just mean editing this PR with the new target branch? @dan-s1 Changing the base branch of this pull request to `support/nifi-1.x` should be sufficient provided that is does not result in bringing in any additional commits. If it does result in additional commits, then you will need to create a new pull request with this specific commit, from the `support/nifi-1.x` branch. Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-11841) Nifi in cluster setup doesn't stop within the graceful shutdown period
[
https://issues.apache.org/jira/browse/NIFI-11841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17770452#comment-17770452
]
Dheeraj Joshi commented on NIFI-11841:
--
The logs are from 3 instances running on my local development box. But we saw
the same issue in the K8S cluster where we have 3 replicas forming a cluster.
Our current understanding is that the leader node gets terminated within the
timeout, but the other two nodes do not terminate within the configured timeout.
> Nifi in cluster setup doesn't stop within the graceful shutdown period
> --
>
> Key: NIFI-11841
> URL: https://issues.apache.org/jira/browse/NIFI-11841
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.22.0
>Reporter: Dheeraj Joshi
>Priority: Major
> Attachments: Logs.zip, bootstrap.conf, nifi-app.log,
> nifi-bootstrap.log, nifi.properties, zookeeper.properties
>
>
> Bootstrap logs show the following logs when one of the nodes in a 3-node
> cluster is stopped
> {code:java}
> 2023-07-20 11:10:49,847 INFO [main] org.apache.nifi.bootstrap.Command
> Launched Apache NiFi with Process ID 18579
> 2023-07-20 11:10:50,475 INFO [NiFi Bootstrap Command Listener]
> org.apache.nifi.bootstrap.RunNiFi Apache NiFi now running and listening for
> Bootstrap requests on port 64739
> 2023-07-20 11:12:15,723 INFO [main] o.a.n.b.NotificationServiceManager
> Successfully loaded the following 0 services: []
> 2023-07-20 11:12:15,725 INFO [main] org.apache.nifi.bootstrap.RunNiFi
> Registered no Notification Services for Notification Type NIFI_STARTED
> 2023-07-20 11:12:15,725 INFO [main] org.apache.nifi.bootstrap.RunNiFi
> Registered no Notification Services for Notification Type NIFI_STOPPED
> 2023-07-20 11:12:15,725 INFO [main] org.apache.nifi.bootstrap.RunNiFi
> Registered no Notification Services for Notification Type NIFI_DIED
> 2023-07-20 11:12:15,750 INFO [main] org.apache.nifi.bootstrap.Command Apache
> NiFi has accepted the Shutdown Command and is shutting down now
> 2023-07-20 11:12:15,859 INFO [main] org.apache.nifi.bootstrap.Command NiFi
> PID [18579] shutdown in progress...
> 2023-07-20 11:12:17,896 INFO [main] org.apache.nifi.bootstrap.Command NiFi
> PID [18579] shutdown in progress...
> 2023-07-20 11:12:19,919 INFO [main] org.apache.nifi.bootstrap.Command NiFi
> PID [18579] shutdown in progress...{code}
> The shutdown in progress message gets repeated till
> {code:java}
> graceful.shutdown.seconds{code}
> is elapsed.
> Nifi app logs print socket exception during the shutdown but eventually shuts
> down the application server server
> {code:java}
> 2023-07-20 11:12:15,750 INFO [pool-2-thread-2]
> org.apache.nifi.BootstrapListener Received SHUTDOWN request from Bootstrap
> 2023-07-20 11:12:15,750 INFO [pool-2-thread-2] org.apache.nifi.NiFi
> Application Server shutdown started
> 2023-07-20 11:12:15,755 INFO [pool-2-thread-2]
> o.eclipse.jetty.server.AbstractConnector Stopped
> ServerConnector@2407a36c{HTTP/1.1, (http/1.1)}
> {127.0.0.1:8080}
> 2023-07-20 11:12:15,756 INFO [pool-2-thread-2]
> org.eclipse.jetty.server.session node0 Stopped scavenging
> 2023-07-20 11:12:15,769 INFO [pool-2-thread-2]
> o.e.jetty.server.handler.ContextHandler Stopped
> o.e.j.w.WebAppContext@7cfb8e98{nifi-error,/,null,STOPPED}
> {./work/nar/extensions/nifi-server-nar-1.22.0.nar-unpacked/NAR-INF/bundled-dependencies/nifi-web-error-1.22.0.war}
> 2023-07-20 11:12:15,782 INFO [pool-2-thread-2]
> o.e.jetty.server.handler.ContextHandler Stopped
> o.e.j.w.WebAppContext@2aa5bd48{nifi-docs,/nifi-docs,null,STOPPED}
> {./work/nar/extensions/nifi-server-nar-1.22.0.nar-unpacked/NAR-INF/bundled-dependencies/nifi-web-docs-1.22.0.war}
> 2023-07-20 11:12:15,798 INFO [pool-2-thread-2]
> o.e.jetty.server.handler.ContextHandler Stopped
> o.e.j.w.WebAppContext@133aacbe{nifi-content-viewer,/nifi-content-viewer,null,STOPPED}
> {./work/nar/extensions/nifi-server-nar-1.22.0.nar-unpacked/NAR-INF/bundled-dependencies/nifi-web-content-viewer-1.22.0.war}
> 2023-07-20 11:12:15,803 INFO [pool-2-thread-2]
> o.a.n.w.c.ApplicationStartupContextListener Flow Service shutdown started
> 2023-07-20 11:12:15,803 INFO [pool-2-thread-2]
> o.a.n.c.c.node.NodeClusterCoordinator Status of 127.0.0.1:8080 changed from
> NodeConnectionStatus[nodeId=127.0.0.1:8080, state=CONNECTED, updateId=50] to
> NodeConnectionStatus[nodeId=127.0.0.1:8080, state=DISCONNECTED, Disconnect
> Code=Node was Shutdown, Disconnect Reason=Node was Shutdown, updateId=50]
> 2023-07-20 11:12:15,809 INFO [pool-2-thread-2]
> o.a.n.c.c.node.NodeClusterCoordinator Successfully notified other nodes that
> I am shutting down
> 2023-07-20 11:12:15,811 INFO [Curator-Framework-0]
> o.a.c.f.imps.CuratorFrameworkImpl backgr
[GitHub] [nifi] dan-s1 commented on pull request #7803: NIFI-12100_deprecate Added DeprecationNotice annotation to the ConvertExcelToCSVProcessor
dan-s1 commented on PR #7803: URL: https://github.com/apache/nifi/pull/7803#issuecomment-1740922914 @exceptionfactory It resulted in other commits so I created a new branch off of support/nifi-1.x and I will create another PR. Interesting I am doing the build with the contrib profile and there was a Checkstyle violation on another piece of code SplitJson -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Comment Edited] (NIFI-11841) Nifi in cluster setup doesn't stop within the graceful shutdown period
[
https://issues.apache.org/jira/browse/NIFI-11841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17770452#comment-17770452
]
Dheeraj Joshi edited comment on NIFI-11841 at 9/29/23 1:48 PM:
---
The logs are from 3 instances running on my local development box. But we saw
the same issue in the K8S cluster where we have 3 replicas forming a cluster.
Our current understanding is that the leader node gets terminated within the
timeout, but the other two nodes do not terminate within the configured timeout.
We are using embedded zookeeper in both the deployments
was (Author: dheerajjoshim):
The logs are from 3 instances running on my local development box. But we saw
the same issue in the K8S cluster where we have 3 replicas forming a cluster.
Our current understanding is that the leader node gets terminated within the
timeout, but the other two nodes do not terminate within the configured timeout.
> Nifi in cluster setup doesn't stop within the graceful shutdown period
> --
>
> Key: NIFI-11841
> URL: https://issues.apache.org/jira/browse/NIFI-11841
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.22.0
>Reporter: Dheeraj Joshi
>Priority: Major
> Attachments: Logs.zip, bootstrap.conf, nifi-app.log,
> nifi-bootstrap.log, nifi.properties, zookeeper.properties
>
>
> Bootstrap logs show the following logs when one of the nodes in a 3-node
> cluster is stopped
> {code:java}
> 2023-07-20 11:10:49,847 INFO [main] org.apache.nifi.bootstrap.Command
> Launched Apache NiFi with Process ID 18579
> 2023-07-20 11:10:50,475 INFO [NiFi Bootstrap Command Listener]
> org.apache.nifi.bootstrap.RunNiFi Apache NiFi now running and listening for
> Bootstrap requests on port 64739
> 2023-07-20 11:12:15,723 INFO [main] o.a.n.b.NotificationServiceManager
> Successfully loaded the following 0 services: []
> 2023-07-20 11:12:15,725 INFO [main] org.apache.nifi.bootstrap.RunNiFi
> Registered no Notification Services for Notification Type NIFI_STARTED
> 2023-07-20 11:12:15,725 INFO [main] org.apache.nifi.bootstrap.RunNiFi
> Registered no Notification Services for Notification Type NIFI_STOPPED
> 2023-07-20 11:12:15,725 INFO [main] org.apache.nifi.bootstrap.RunNiFi
> Registered no Notification Services for Notification Type NIFI_DIED
> 2023-07-20 11:12:15,750 INFO [main] org.apache.nifi.bootstrap.Command Apache
> NiFi has accepted the Shutdown Command and is shutting down now
> 2023-07-20 11:12:15,859 INFO [main] org.apache.nifi.bootstrap.Command NiFi
> PID [18579] shutdown in progress...
> 2023-07-20 11:12:17,896 INFO [main] org.apache.nifi.bootstrap.Command NiFi
> PID [18579] shutdown in progress...
> 2023-07-20 11:12:19,919 INFO [main] org.apache.nifi.bootstrap.Command NiFi
> PID [18579] shutdown in progress...{code}
> The shutdown in progress message gets repeated till
> {code:java}
> graceful.shutdown.seconds{code}
> is elapsed.
> Nifi app logs print socket exception during the shutdown but eventually shuts
> down the application server server
> {code:java}
> 2023-07-20 11:12:15,750 INFO [pool-2-thread-2]
> org.apache.nifi.BootstrapListener Received SHUTDOWN request from Bootstrap
> 2023-07-20 11:12:15,750 INFO [pool-2-thread-2] org.apache.nifi.NiFi
> Application Server shutdown started
> 2023-07-20 11:12:15,755 INFO [pool-2-thread-2]
> o.eclipse.jetty.server.AbstractConnector Stopped
> ServerConnector@2407a36c{HTTP/1.1, (http/1.1)}
> {127.0.0.1:8080}
> 2023-07-20 11:12:15,756 INFO [pool-2-thread-2]
> org.eclipse.jetty.server.session node0 Stopped scavenging
> 2023-07-20 11:12:15,769 INFO [pool-2-thread-2]
> o.e.jetty.server.handler.ContextHandler Stopped
> o.e.j.w.WebAppContext@7cfb8e98{nifi-error,/,null,STOPPED}
> {./work/nar/extensions/nifi-server-nar-1.22.0.nar-unpacked/NAR-INF/bundled-dependencies/nifi-web-error-1.22.0.war}
> 2023-07-20 11:12:15,782 INFO [pool-2-thread-2]
> o.e.jetty.server.handler.ContextHandler Stopped
> o.e.j.w.WebAppContext@2aa5bd48{nifi-docs,/nifi-docs,null,STOPPED}
> {./work/nar/extensions/nifi-server-nar-1.22.0.nar-unpacked/NAR-INF/bundled-dependencies/nifi-web-docs-1.22.0.war}
> 2023-07-20 11:12:15,798 INFO [pool-2-thread-2]
> o.e.jetty.server.handler.ContextHandler Stopped
> o.e.j.w.WebAppContext@133aacbe{nifi-content-viewer,/nifi-content-viewer,null,STOPPED}
> {./work/nar/extensions/nifi-server-nar-1.22.0.nar-unpacked/NAR-INF/bundled-dependencies/nifi-web-content-viewer-1.22.0.war}
> 2023-07-20 11:12:15,803 INFO [pool-2-thread-2]
> o.a.n.w.c.ApplicationStartupContextListener Flow Service shutdown started
> 2023-07-20 11:12:15,803 INFO [pool-2-thread-2]
> o.a.n.c.c.node.NodeClusterCoordinator Status of 127.0.0.1:8080 changed from
> NodeConnectionStatus[node
[GitHub] [nifi] exceptionfactory commented on pull request #7803: NIFI-12100_deprecate Added DeprecationNotice annotation to the ConvertExcelToCSVProcessor
exceptionfactory commented on PR #7803: URL: https://github.com/apache/nifi/pull/7803#issuecomment-1740927375 > @exceptionfactory It resulted in other commits so I created a new branch off of support/nifi-1.x and I will create another PR. Interesting I am doing the build with the contrib profile and there was a Checkstyle violation on another piece of code SplitJson Thanks @dan-s1. Feel free to close this pull request when you have the new one submitted. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] elcsiga opened a new pull request, #7811: NIFI-12099 Repurpose Template icon in top left bar
elcsiga opened a new pull request, #7811: URL: https://github.com/apache/nifi/pull/7811 # Summary [NIFI-12099](https://issues.apache.org/jira/browse/NIFI-12099) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [x] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [x] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [x] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [x] Pull Request based on current revision of the `main` branch - [x] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [x] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files https://github.com/apache/nifi/assets/1295926/364ef977-26b8-4eaf-ae19-29ed32ef2dc7 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] dan-s1 closed pull request #7803: NIFI-12100_deprecate Added DeprecationNotice annotation to the ConvertExcelToCSVProcessor
dan-s1 closed pull request #7803: NIFI-12100_deprecate Added DeprecationNotice annotation to the ConvertExcelToCSVProcessor URL: https://github.com/apache/nifi/pull/7803 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm commented on a diff in pull request #1671: MINIFICPP-1891 Integrate google clang-tidy checks in CI
szaszm commented on code in PR #1671: URL: https://github.com/apache/nifi-minifi-cpp/pull/1671#discussion_r1341418984 ## extensions/standard-processors/processors/PutTCP.cpp: ## @@ -215,7 +215,7 @@ asio::awaitable ConnectionHandler::establishNewConn continue; } if (max_size_of_socket_send_buffer_) - socket.lowest_layer().set_option(TcpSocket::send_buffer_size(*max_size_of_socket_send_buffer_)); + socket.lowest_layer().set_option(TcpSocket::send_buffer_size(*max_size_of_socket_send_buffer_)); // NOLINT(google-readability-casting) Review Comment: I think casts are pretty well covered by other checks. I would just disable this one due to the false positives, instead of littering the code with NOLINTs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm commented on a diff in pull request #1671: MINIFICPP-1891 Integrate google clang-tidy checks in CI
szaszm commented on code in PR #1671:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1671#discussion_r1341420013
##
extensions/expression-language/tests/ExpressionLanguageTests.cpp:
##
@@ -1321,13 +1321,13 @@ TEST_CASE("Now Date", "[expressionNowDate]") {
TEST_CASE("Parse RFC3339 with Expression Language toDate") {
using date::sys_days;
using org::apache::nifi::minifi::utils::timeutils::parseRfc3339;
- using namespace date::literals;
+ using namespace date::literals; // NOLINT(google-build-using-namespace)
using namespace std::literals::chrono_literals;
using std::chrono::milliseconds;
- milliseconds expected_second =
std::chrono::floor((sys_days(2023_y / 03 / 01) + 19h + 04min +
55s).time_since_epoch());
- milliseconds expected_tenth_second =
std::chrono::floor((sys_days(2023_y / 03 / 01) + 19h + 04min +
55s + 100ms).time_since_epoch());
- milliseconds expected_milli_second =
std::chrono::floor((sys_days(2023_y / 03 / 01) + 19h + 04min +
55s + 190ms).time_since_epoch());
+ milliseconds expected_second =
std::chrono::floor((sys_days{2023_y / 03 / 01} + 19h + 04min +
55s).time_since_epoch()); // NOLINT(whitespace/braces)
+ milliseconds expected_tenth_second =
std::chrono::floor((sys_days{2023_y / 03 / 01} + 19h + 04min +
55s + 100ms).time_since_epoch()); // NOLINT(whitespace/braces)
+ milliseconds expected_milli_second =
std::chrono::floor((sys_days{2023_y / 03 / 01} + 19h + 04min +
55s + 190ms).time_since_epoch()); // NOLINT(whitespace/braces)
Review Comment:
And what necessitated `// NOLINT(whitespace/braces)`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] bbende opened a new pull request, #7812: NIFI-12149 Create nifi-redis-utils and minor improvements to util met…
bbende opened a new pull request, #7812: URL: https://github.com/apache/nifi/pull/7812 …hods # Summary [NIFI-12149](https://issues.apache.org/jira/browse/NIFI-12149) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [X] Build completed using `mvn clean install -P contrib-check` - [X] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] dan-s1 opened a new pull request, #7813: Nifi 12100 deprecate v2
dan-s1 opened a new pull request, #7813: URL: https://github.com/apache/nifi/pull/7813 # Summary [ NIFI-12100](https://issues.apache.org/jira/browse/NIFI-12100) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-12149) Separate RedisUtils into a separate module
[ https://issues.apache.org/jira/browse/NIFI-12149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Bende updated NIFI-12149: --- Status: Patch Available (was: Open) > Separate RedisUtils into a separate module > -- > > Key: NIFI-12149 > URL: https://issues.apache.org/jira/browse/NIFI-12149 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.23.2 >Reporter: Bryan Bende >Assignee: Bryan Bende >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Currently the RedisUtils class and some other utility code is embedded into > the nifi-redis-extensions module which makes it not easy to reuse anywhere > since depending on it would bring in extensions. It should be in a separate > module like nifi-redis-utils. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] dan-s1 closed pull request #7813: Nifi 12100 deprecate v2
dan-s1 closed pull request #7813: Nifi 12100 deprecate v2 URL: https://github.com/apache/nifi/pull/7813 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm commented on a diff in pull request #1671: MINIFICPP-1891 Integrate google clang-tidy checks in CI
szaszm commented on code in PR #1671:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1671#discussion_r1341429719
##
extensions/systemd/tests/ConsumeJournaldTest.cpp:
##
@@ -90,7 +90,7 @@ struct TestJournal final : libwrapper::Journal {
}
int getCursor(gsl::owner* const cursor_out) noexcept override {
-*cursor_out = gsl::owner(strdup(std::to_string(consumed).c_str()));
+*cursor_out = strdup(std::to_string(consumed).c_str()); //
NOLINT(cppcoreguidelines-owning-memory)
Review Comment:
There was no example that assigned owner through a pointer parameter. The
examples don't have an additional layer of indirection over the owner.
Could you double check that leaving the gsl::owner annotation in place
doesn't solve the warning? The C-style cast one should probably be just
disabled, due to the high number of false positives.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] dan-s1 opened a new pull request, #7814: Annotated the ConvertExcelToCSVProcessor with the DeprecationNotice annotation and fixed the Checkstyle violation in SplitJson.
dan-s1 opened a new pull request, #7814: URL: https://github.com/apache/nifi/pull/7814 …e DeprecationNotice annotation and fixed the Checkstyle violation in SplitJson. # Summary [NIFI-12100](https://issues.apache.org/jira/browse/NIFI-12100) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm commented on a diff in pull request #1671: MINIFICPP-1891 Integrate google clang-tidy checks in CI
szaszm commented on code in PR #1671: URL: https://github.com/apache/nifi-minifi-cpp/pull/1671#discussion_r1341433299 ## .clang-tidy: ## Review Comment: Just a general comment: If a check produces many false positives, then we shouldn't enable it at all costs. The static analyzer is meant to help development, not hinder it. If a particular check hurts more than it helps, then we shouldn't use it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-12100) Remove ConvertExcelToCSVProcessor
[ https://issues.apache.org/jira/browse/NIFI-12100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17770461#comment-17770461 ] Daniel Stieglitz commented on NIFI-12100: - [~exceptionfactory] I branched off of support/nifi-1.x as you had instructed, closed PR #7803 and I created a new one [#7814|https://github.com/apache/nifi/pull/7814] > Remove ConvertExcelToCSVProcessor > --- > > Key: NIFI-12100 > URL: https://issues.apache.org/jira/browse/NIFI-12100 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Daniel Stieglitz >Assignee: Daniel Stieglitz >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > ConvertExcelToCSVProcessor is no longer needed since there is now the > ExcelReader which along with CSVRecordSetWriter can be used in ConvertRecord > to achieve the same thing. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi-minifi-cpp] szaszm commented on a diff in pull request #1669: MINIFICPP-2210 Add C2 debug command to MiNiFi Controller
szaszm commented on code in PR #1669:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1669#discussion_r1341459182
##
libminifi/include/utils/file/ArchiveUtils.h:
##
@@ -0,0 +1,31 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#pragma once
+
+#include
+#include
+#include
+
+#include "io/ArchiveStream.h"
+#include "utils/expected.h"
+#include "io/BufferStream.h"
+
+namespace org::apache::nifi::minifi::utils::archive {
+
+nonstd::expected, std::string>
createArchive(std::map>& files,
const std::shared_ptr& logger);
Review Comment:
I want to go into philosophical discussions, because debate helps us find
the common truth if it exists. Please don't take any criticism or disagreement
personally: I like you and your work, but there are parts I don't like, and
they make for an interesting discussion.
If the utility is not general enough to at least in theory be usable by
others, then it's most likely also not general enough to be reused elsewhere in
the code (except if it was copy-pasted from the first usage site).
Utilities that are specific to implementation details are fine, but they
should be closer to the usage site, and not extracted to this place, where
reusable utilities live.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1671: MINIFICPP-1891 Integrate google clang-tidy checks in CI
lordgamez commented on code in PR #1671:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1671#discussion_r1341463466
##
extensions/systemd/tests/ConsumeJournaldTest.cpp:
##
@@ -90,7 +90,7 @@ struct TestJournal final : libwrapper::Journal {
}
int getCursor(gsl::owner* const cursor_out) noexcept override {
-*cursor_out = gsl::owner(strdup(std::to_string(consumed).c_str()));
+*cursor_out = strdup(std::to_string(consumed).c_str()); //
NOLINT(cppcoreguidelines-owning-memory)
Review Comment:
I was referring to the example assigning the returned value of a function to
a `gsl::owner<>` type, as we are assigning to the pointed value I don't think
the pointer parameter is relevant in this case, the warning would be the same
if we only had a `gsl::owner` type parameter. I can revert it as the
C-style cast check will be removed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (NIFI-12100) Remove ConvertExcelToCSVProcessor
[ https://issues.apache.org/jira/browse/NIFI-12100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17770471#comment-17770471 ] David Handermann commented on NIFI-12100: - Thanks [~dstiegli1]! > Remove ConvertExcelToCSVProcessor > --- > > Key: NIFI-12100 > URL: https://issues.apache.org/jira/browse/NIFI-12100 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Daniel Stieglitz >Assignee: Daniel Stieglitz >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > ConvertExcelToCSVProcessor is no longer needed since there is now the > ExcelReader which along with CSVRecordSetWriter can be used in ConvertRecord > to achieve the same thing. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] asfgit closed pull request #7796: NIFI-12104 Separate a non-atomic Redis DMC implementation from the ex…
asfgit closed pull request #7796: NIFI-12104 Separate a non-atomic Redis DMC implementation from the ex… URL: https://github.com/apache/nifi/pull/7796 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1671: MINIFICPP-1891 Integrate google clang-tidy checks in CI
lordgamez commented on code in PR #1671:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1671#discussion_r1341466297
##
extensions/expression-language/tests/ExpressionLanguageTests.cpp:
##
@@ -1321,13 +1321,13 @@ TEST_CASE("Now Date", "[expressionNowDate]") {
TEST_CASE("Parse RFC3339 with Expression Language toDate") {
using date::sys_days;
using org::apache::nifi::minifi::utils::timeutils::parseRfc3339;
- using namespace date::literals;
+ using namespace date::literals; // NOLINT(google-build-using-namespace)
using namespace std::literals::chrono_literals;
using std::chrono::milliseconds;
- milliseconds expected_second =
std::chrono::floor((sys_days(2023_y / 03 / 01) + 19h + 04min +
55s).time_since_epoch());
- milliseconds expected_tenth_second =
std::chrono::floor((sys_days(2023_y / 03 / 01) + 19h + 04min +
55s + 100ms).time_since_epoch());
- milliseconds expected_milli_second =
std::chrono::floor((sys_days(2023_y / 03 / 01) + 19h + 04min +
55s + 190ms).time_since_epoch());
+ milliseconds expected_second =
std::chrono::floor((sys_days{2023_y / 03 / 01} + 19h + 04min +
55s).time_since_epoch()); // NOLINT(whitespace/braces)
+ milliseconds expected_tenth_second =
std::chrono::floor((sys_days{2023_y / 03 / 01} + 19h + 04min +
55s + 100ms).time_since_epoch()); // NOLINT(whitespace/braces)
+ milliseconds expected_milli_second =
std::chrono::floor((sys_days{2023_y / 03 / 01} + 19h + 04min +
55s + 190ms).time_since_epoch()); // NOLINT(whitespace/braces)
Review Comment:
That's also a false positive as it wants to add a space after the brace in
`sys_days{`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (NIFI-12104) Allow Redis DMC to be used by Put/Fetch DMC when Redis is clustered
[ https://issues.apache.org/jira/browse/NIFI-12104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17770472#comment-17770472 ] ASF subversion and git services commented on NIFI-12104: Commit a74c411079efe7d886ddae58314d1f2cee74f58e in nifi's branch refs/heads/main from Bryan Bende [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=a74c411079 ] NIFI-12104 Separate a non-atomic Redis DMC implementation from the existing one for use in Put/Fetch DMC when Redis is clustered This closes #7796 Signed-off-by: Mike Thomsen > Allow Redis DMC to be used by Put/Fetch DMC when Redis is clustered > --- > > Key: NIFI-12104 > URL: https://issues.apache.org/jira/browse/NIFI-12104 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.23.2 >Reporter: Bryan Bende >Assignee: Bryan Bende >Priority: Minor > Time Spent: 40m > Remaining Estimate: 0h > > The Redis DMC directly implements Atomic DMC Client and because it implements > the atomic updates using Redis transactions which don't work in a Redis > cluster, it imposes a constraint that the Redis Mode cannot be set to > Cluster, otherwise it is invalid. The downside is that then the Redis DMC > can't be used with a Redis cluster, even in places that didn't require atomic > updates like Put/Fetch DMC. We should be able to improve this. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1671: MINIFICPP-1891 Integrate google clang-tidy checks in CI
lordgamez commented on code in PR #1671: URL: https://github.com/apache/nifi-minifi-cpp/pull/1671#discussion_r1341470484 ## .clang-tidy: ## Review Comment: You are right, I was thinking about it, but didn't want to remove it before discussing it. Even though there were a lot of false positives most of them were in a single file, and the check also found some valid cases, so I was not sure about removing it, but now I think it's better as this check does not add enough value to keep it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] MikeThomsen commented on pull request #7796: NIFI-12104 Separate a non-atomic Redis DMC implementation from the ex…
MikeThomsen commented on PR #7796: URL: https://github.com/apache/nifi/pull/7796#issuecomment-1741032158 The cherry-pick to 1.x was kinda messy. Looks like `CONNECTION_POOL` was moved to `RedisUtils` in another commit. I'll try to chase that down later. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-12104) Allow Redis DMC to be used by Put/Fetch DMC when Redis is clustered
[ https://issues.apache.org/jira/browse/NIFI-12104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Thomsen updated NIFI-12104: Fix Version/s: 2.latest Resolution: Fixed Status: Resolved (was: Patch Available) > Allow Redis DMC to be used by Put/Fetch DMC when Redis is clustered > --- > > Key: NIFI-12104 > URL: https://issues.apache.org/jira/browse/NIFI-12104 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.23.2 >Reporter: Bryan Bende >Assignee: Bryan Bende >Priority: Minor > Fix For: 2.latest > > Time Spent: 1h > Remaining Estimate: 0h > > The Redis DMC directly implements Atomic DMC Client and because it implements > the atomic updates using Redis transactions which don't work in a Redis > cluster, it imposes a constraint that the Redis Mode cannot be set to > Cluster, otherwise it is invalid. The downside is that then the Redis DMC > can't be used with a Redis cluster, even in places that didn't require atomic > updates like Put/Fetch DMC. We should be able to improve this. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1671: MINIFICPP-1891 Integrate google clang-tidy checks in CI
lordgamez commented on code in PR #1671:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1671#discussion_r1341498042
##
extensions/expression-language/tests/ExpressionLanguageTests.cpp:
##
@@ -1321,13 +1321,13 @@ TEST_CASE("Now Date", "[expressionNowDate]") {
TEST_CASE("Parse RFC3339 with Expression Language toDate") {
using date::sys_days;
using org::apache::nifi::minifi::utils::timeutils::parseRfc3339;
- using namespace date::literals;
+ using namespace date::literals; // NOLINT(google-build-using-namespace)
using namespace std::literals::chrono_literals;
using std::chrono::milliseconds;
- milliseconds expected_second =
std::chrono::floor((sys_days(2023_y / 03 / 01) + 19h + 04min +
55s).time_since_epoch());
- milliseconds expected_tenth_second =
std::chrono::floor((sys_days(2023_y / 03 / 01) + 19h + 04min +
55s + 100ms).time_since_epoch());
- milliseconds expected_milli_second =
std::chrono::floor((sys_days(2023_y / 03 / 01) + 19h + 04min +
55s + 190ms).time_since_epoch());
+ milliseconds expected_second =
std::chrono::floor((sys_days{2023_y / 03 / 01} + 19h + 04min +
55s).time_since_epoch()); // NOLINT(whitespace/braces)
+ milliseconds expected_tenth_second =
std::chrono::floor((sys_days{2023_y / 03 / 01} + 19h + 04min +
55s + 100ms).time_since_epoch()); // NOLINT(whitespace/braces)
+ milliseconds expected_milli_second =
std::chrono::floor((sys_days{2023_y / 03 / 01} + 19h + 04min +
55s + 190ms).time_since_epoch()); // NOLINT(whitespace/braces)
Review Comment:
Reverted in b1d4a35c9a2021f2bd8d0fa750e2f2971f635cae
##
extensions/standard-processors/processors/PutTCP.cpp:
##
@@ -215,7 +215,7 @@ asio::awaitable
ConnectionHandler::establishNewConn
continue;
}
if (max_size_of_socket_send_buffer_)
-
socket.lowest_layer().set_option(TcpSocket::send_buffer_size(*max_size_of_socket_send_buffer_));
+
socket.lowest_layer().set_option(TcpSocket::send_buffer_size(*max_size_of_socket_send_buffer_));
// NOLINT(google-readability-casting)
Review Comment:
Removed in b1d4a35c9a2021f2bd8d0fa750e2f2971f635cae
##
extensions/systemd/tests/ConsumeJournaldTest.cpp:
##
@@ -90,7 +90,7 @@ struct TestJournal final : libwrapper::Journal {
}
int getCursor(gsl::owner* const cursor_out) noexcept override {
-*cursor_out = gsl::owner(strdup(std::to_string(consumed).c_str()));
+*cursor_out = strdup(std::to_string(consumed).c_str()); //
NOLINT(cppcoreguidelines-owning-memory)
Review Comment:
Reverted in b1d4a35c9a2021f2bd8d0fa750e2f2971f635cae
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1669: MINIFICPP-2210 Add C2 debug command to MiNiFi Controller
lordgamez commented on code in PR #1669:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1669#discussion_r1341548877
##
libminifi/include/utils/file/ArchiveUtils.h:
##
@@ -0,0 +1,31 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#pragma once
+
+#include
+#include
+#include
+
+#include "io/ArchiveStream.h"
+#include "utils/expected.h"
+#include "io/BufferStream.h"
+
+namespace org::apache::nifi::minifi::utils::archive {
+
+nonstd::expected, std::string>
createArchive(std::map>& files,
const std::shared_ptr& logger);
Review Comment:
Sorry, what I actually meant is that I don't want to go into philosophical
discussions _here_ as it is not closely related to this PR, but I agree it's
good to have debates (in a separate thread :))
I think in this specific case having a function that takes a map of files,
returns a stream, and used two different places in the codebase can be
extracted and called as a utility. It is a utility function that has a general
purpose for two modules outside of the main domain of those modules. I would
not overgeneralize the interface until we would need third use case that
requires this interface to be refined. I would rather have our use cases refine
our interfaces and only develop our code for the current requirements than
trying to predict what future requirements will be.
There was a talk about this years ago that optimizing for code reuse is
overrated as most `generic` purpose classes and functions are used 2-3 times in
their lifetime and making them too generic usually come with unneeded
compexity. Unfortunately I can't find that talk, but I think [this blog
post](https://medium.com/@scott.boring.sb/dont-write-reusable-code-a857e925b683)
summarizes it well. "Reuse is an outcome, not a design pattern."
On the other hand maybe I was wrong calling it a "util", and of course we
have different concepts of what a "util" is. In the project we also have some
naming issues as we have too many "Utils" in the code already. We may have to
think of another way to group our (somewhat) general functionalities that may
not have to be globally generic, but only generic in the domain of a specific
module. [Kevlin
Henney](https://twitter.com/KevlinHenney/status/715093313301454849) had a whole
talk about this that the problem with `util` modules that usually people put
classes, functions in `utils` when they cannot really be categorized and it is
mostly just a dumping ground for everything. Maybe we could think about
reorganizing `libminifi/utils` to separate encapsulated libraries along their
functionalities. (It may solve the problem of having to rebuild everything in
libminifi when a `util` is changed)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Created] (NIFI-12150) Restore any deleted classes that are still required such as NiFiRegistrySecurityConfig
Joe Witt created NIFI-12150: --- Summary: Restore any deleted classes that are still required such as NiFiRegistrySecurityConfig Key: NIFI-12150 URL: https://issues.apache.org/jira/browse/NIFI-12150 Project: Apache NiFi Issue Type: Bug Affects Versions: 2.0.0 Reporter: Joe Witt Assignee: Joe Witt Fix For: 2.0.0 In NIFI-12142 we removed too much. Notable the spring bean necessary for security config for the registry. Will search for any other deleted classes. The method used to find unreadable/unused classes was too aggressive. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] joewitt opened a new pull request, #7815: NIFI-12150 restoring NiFiRegistrySecurityConfig as it is a key spring…
joewitt opened a new pull request, #7815: URL: https://github.com/apache/nifi/pull/7815 … bean needed for the registry and was removed mistakenly in NIFI-12147 and updating extractor to match changes in NIFI-12147 for dn extraction # Summary [NIFI-0](https://issues.apache.org/jira/browse/NIFI-0) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-12150) Restore any deleted classes that are still required such as NiFiRegistrySecurityConfig
[ https://issues.apache.org/jira/browse/NIFI-12150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt updated NIFI-12150: Status: Patch Available (was: Open) > Restore any deleted classes that are still required such as > NiFiRegistrySecurityConfig > -- > > Key: NIFI-12150 > URL: https://issues.apache.org/jira/browse/NIFI-12150 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Joe Witt >Assignee: Joe Witt >Priority: Major > Fix For: 2.0.0 > > Time Spent: 10m > Remaining Estimate: 0h > > In NIFI-12142 we removed too much. Notable the spring bean necessary for > security config for the registry. Will search for any other deleted classes. > The method used to find unreadable/unused classes was too aggressive. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] mattyb149 commented on pull request #2275: NIFI-4612: Allow AvroSchemaRegistry to disable name validation
mattyb149 commented on PR #2275: URL: https://github.com/apache/nifi/pull/2275#issuecomment-1741236344 How do the other apps work with an invalid schema? If you need to "normalize" the names so the schema is valid (and the data is not already in Avro format), I wrote a [blog post ](https://funnifi.blogspot.com/2023/08/change-record-field-names-with.html)showing how to do that. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] joewitt commented on pull request #7815: NIFI-12150 restoring NiFiRegistrySecurityConfig as it is a key spring…
joewitt commented on PR #7815: URL: https://github.com/apache/nifi/pull/7815#issuecomment-1741244224 commit message should reference NIFI-12142 not NIFI-12147 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] asfgit closed pull request #7802: NIFI-12100 Removed the ConvertExcelToCSVProcessor
asfgit closed pull request #7802: NIFI-12100 Removed the ConvertExcelToCSVProcessor URL: https://github.com/apache/nifi/pull/7802 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-12100) Remove ConvertExcelToCSVProcessor
[ https://issues.apache.org/jira/browse/NIFI-12100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Thomsen updated NIFI-12100: Fix Version/s: 2.latest Resolution: Fixed Status: Resolved (was: Patch Available) > Remove ConvertExcelToCSVProcessor > --- > > Key: NIFI-12100 > URL: https://issues.apache.org/jira/browse/NIFI-12100 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Daniel Stieglitz >Assignee: Daniel Stieglitz >Priority: Major > Fix For: 2.latest > > Time Spent: 20m > Remaining Estimate: 0h > > ConvertExcelToCSVProcessor is no longer needed since there is now the > ExcelReader which along with CSVRecordSetWriter can be used in ConvertRecord > to achieve the same thing. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12100) Remove ConvertExcelToCSVProcessor
[ https://issues.apache.org/jira/browse/NIFI-12100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17770554#comment-17770554 ] ASF subversion and git services commented on NIFI-12100: Commit e9b532bd3237fb23b351471586d0f8aebab16d6b in nifi's branch refs/heads/main from dan-s1 [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=e9b532bd32 ] NIFI-12100 Removed the ConvertExcelToCSVProcessor This closes #7802 Signed-off-by: Mike Thomsen > Remove ConvertExcelToCSVProcessor > --- > > Key: NIFI-12100 > URL: https://issues.apache.org/jira/browse/NIFI-12100 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Daniel Stieglitz >Assignee: Daniel Stieglitz >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > ConvertExcelToCSVProcessor is no longer needed since there is now the > ExcelReader which along with CSVRecordSetWriter can be used in ConvertRecord > to achieve the same thing. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (NIFI-12143) Update all calcite tables to new API
[ https://issues.apache.org/jira/browse/NIFI-12143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne reassigned NIFI-12143: - Assignee: Mark Payne > Update all calcite tables to new API > > > Key: NIFI-12143 > URL: https://issues.apache.org/jira/browse/NIFI-12143 > Project: Apache NiFi > Issue Type: Task >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Labels: deprecation > Fix For: 2.latest > > > We have several Calcite tables that have been implemented in the > nifi-standard-processors for QueryRecord, and in the nifi-sql-reporting-tasks > module. > These are using the deprecated API. We should update to the new API. > Additionally, it appears that what is in nifi-sql-reporting-tasks are the > same classes copied & pasted over & over. We should look to consolidate them > into a single implementation. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12100) Remove ConvertExcelToCSVProcessor
[ https://issues.apache.org/jira/browse/NIFI-12100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17770555#comment-17770555 ] ASF subversion and git services commented on NIFI-12100: Commit 6288470504aa46d9328d4c97e07212f238151503 in nifi's branch refs/heads/support/nifi-1.x from dan-s1 [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=6288470504 ] NIFI-12100_deprecate Annotated the ConvertExcelToCSVProcessor with the DeprecationNotice annotation and fixed the Checkstyle violation in SplitJson. This closes #7814 Signed-off-by: Mike Thomsen > Remove ConvertExcelToCSVProcessor > --- > > Key: NIFI-12100 > URL: https://issues.apache.org/jira/browse/NIFI-12100 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Daniel Stieglitz >Assignee: Daniel Stieglitz >Priority: Major > Fix For: 2.latest > > Time Spent: 20m > Remaining Estimate: 0h > > ConvertExcelToCSVProcessor is no longer needed since there is now the > ExcelReader which along with CSVRecordSetWriter can be used in ConvertRecord > to achieve the same thing. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] MikeThomsen closed pull request #7814: NIFI-12100 Annotated the ConvertExcelToCSVProcessor with the DeprecationNotice annotation and fixed the Checkstyle violation in SplitJson.
MikeThomsen closed pull request #7814: NIFI-12100 Annotated the ConvertExcelToCSVProcessor with the DeprecationNotice annotation and fixed the Checkstyle violation in SplitJson. URL: https://github.com/apache/nifi/pull/7814 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] MikeThomsen commented on pull request #7814: NIFI-12100 Annotated the ConvertExcelToCSVProcessor with the DeprecationNotice annotation and fixed the Checkstyle violation in SplitJson.
MikeThomsen commented on PR #7814: URL: https://github.com/apache/nifi/pull/7814#issuecomment-1741305081 (Manually closed, github isn't tracking support/nifi-1.x) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on a diff in pull request #7740: NIFI-6240 Proxy Support for AzureEventHub processor via Websockets
exceptionfactory commented on code in PR #7740: URL: https://github.com/apache/nifi/pull/7740#discussion_r1341690686 ## nifi-nar-bundles/nifi-azure-bundle/nifi-azure-processors/src/main/java/org/apache/nifi/processors/azure/eventhub/utils/AzureEventHubUtils.java: ## @@ -103,4 +111,42 @@ public static Map getApplicationProperties(final Map
[jira] [Updated] (NIFI-12100) Remove ConvertExcelToCSVProcessor
[ https://issues.apache.org/jira/browse/NIFI-12100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-12100: Fix Version/s: 2.0.0 (was: 2.latest) > Remove ConvertExcelToCSVProcessor > --- > > Key: NIFI-12100 > URL: https://issues.apache.org/jira/browse/NIFI-12100 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Daniel Stieglitz >Assignee: Daniel Stieglitz >Priority: Major > Fix For: 2.0.0 > > Time Spent: 40m > Remaining Estimate: 0h > > ConvertExcelToCSVProcessor is no longer needed since there is now the > ExcelReader which along with CSVRecordSetWriter can be used in ConvertRecord > to achieve the same thing. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12100) Remove ConvertExcelToCSVProcessor
[ https://issues.apache.org/jira/browse/NIFI-12100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17770561#comment-17770561 ] David Handermann commented on NIFI-12100: - Thanks for merging the changes [~mthomsen]! Just a note to remember to set the exact Fix Version field after merging. I've changed this one to 2.0.0. > Remove ConvertExcelToCSVProcessor > --- > > Key: NIFI-12100 > URL: https://issues.apache.org/jira/browse/NIFI-12100 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Daniel Stieglitz >Assignee: Daniel Stieglitz >Priority: Major > Fix For: 2.latest > > Time Spent: 40m > Remaining Estimate: 0h > > ConvertExcelToCSVProcessor is no longer needed since there is now the > ExcelReader which along with CSVRecordSetWriter can be used in ConvertRecord > to achieve the same thing. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11172) Remove Duplicative Features for 2.0.0
[
https://issues.apache.org/jira/browse/NIFI-11172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17770566#comment-17770566
]
David Handermann commented on NIFI-11172:
-
[~dstiegli1] We should retain both JoltTransformJSON and JoltTransformRecord
for now. JoltTransformJSON is very popular and widely used. JoltTransformRecord
could also work with the right settings, but I recommend avoiding consolidation
attempts for now.
> Remove Duplicative Features for 2.0.0
> -
>
> Key: NIFI-11172
> URL: https://issues.apache.org/jira/browse/NIFI-11172
> Project: Apache NiFi
> Issue Type: Epic
>Reporter: David Handermann
>Priority: Major
> Fix For: 2.0.0
>
>
> Duplicative features should be removed as highlighted in the [NiFi 2.0
> Release
> Goals|https://cwiki.apache.org/confluence/display/NIFI/NiFi+2.0+Release+Goals].
> Removal should include eliminating support for XML Templates and Variable
> Registry features.
> Separate tasks should be created for removing support for {{flow.xml.gz}} as
> the fallback flow configuration in absence of {{{}flow.json.gz{}}}.
> Individual feature removal efforts should be tracked separately in logical
> groups to encourage sufficient review.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-11627) Add Dynamic Schema References to ValidateJSON Processor
[
https://issues.apache.org/jira/browse/NIFI-11627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17770567#comment-17770567
]
David Handermann commented on NIFI-11627:
-
[~dstiegli1] Can you describe the use case for providing the Schema in an
attribute? That would require a the FlowFile to include its own Schema, which
has some utility, but could be problematic and a JSON Schema can be large,
resulting in potential memory issues related to FlowFile attribute sizing.
> Add Dynamic Schema References to ValidateJSON Processor
> ---
>
> Key: NIFI-11627
> URL: https://issues.apache.org/jira/browse/NIFI-11627
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Affects Versions: 1.19.1
>Reporter: Chuck Tilly
>Assignee: Daniel Stieglitz
>Priority: Major
>
> For the ValidateJSON processor, add support for flowfile attribute references
> that will allow for a JSON schema located in the Parameter Contexts, to be
> referenced dynamically based on a flowfile attribute. e.g.
> {code:java}
> #{${schema.name}} {code}
>
> The benefits of adding support for attribute references are significant.
> Adding this capability will allow a single processor to be used for all JSON
> schema validation. Unfortunately, the current version of this processor
> requires a dedicated processor for every schema, i.e. 12 schemas requires 12
> ValidateJSON processors. This is very laborious to construct and maintain,
> and resource expensive.
> ValidateJSON processor (https://issues.apache.org/jira/browse/NIFI-7392)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-11172) Remove Duplicative Features for 2.0.0
[
https://issues.apache.org/jira/browse/NIFI-11172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17770568#comment-17770568
]
Joe Witt commented on NIFI-11172:
-
yep agreed with Handermann's response
> Remove Duplicative Features for 2.0.0
> -
>
> Key: NIFI-11172
> URL: https://issues.apache.org/jira/browse/NIFI-11172
> Project: Apache NiFi
> Issue Type: Epic
>Reporter: David Handermann
>Priority: Major
> Fix For: 2.0.0
>
>
> Duplicative features should be removed as highlighted in the [NiFi 2.0
> Release
> Goals|https://cwiki.apache.org/confluence/display/NIFI/NiFi+2.0+Release+Goals].
> Removal should include eliminating support for XML Templates and Variable
> Registry features.
> Separate tasks should be created for removing support for {{flow.xml.gz}} as
> the fallback flow configuration in absence of {{{}flow.json.gz{}}}.
> Individual feature removal efforts should be tracked separately in logical
> groups to encourage sufficient review.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-12104) Allow Redis DMC to be used by Put/Fetch DMC when Redis is clustered
[ https://issues.apache.org/jira/browse/NIFI-12104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17770576#comment-17770576 ] ASF subversion and git services commented on NIFI-12104: Commit 72c541fb877dc2d730c1b360cd8c2631e0a6c53e in nifi's branch refs/heads/support/nifi-1.x from Bryan Bende [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=72c541fb87 ] NIFI-12104 Separate a non-atomic Redis DMC implementation from the existing one for use in Put/Fetch DMC when Redis is clustered This closes #7796 Signed-off-by: Mike Thomsen > Allow Redis DMC to be used by Put/Fetch DMC when Redis is clustered > --- > > Key: NIFI-12104 > URL: https://issues.apache.org/jira/browse/NIFI-12104 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.23.2 >Reporter: Bryan Bende >Assignee: Bryan Bende >Priority: Minor > Fix For: 2.latest > > Time Spent: 1h > Remaining Estimate: 0h > > The Redis DMC directly implements Atomic DMC Client and because it implements > the atomic updates using Redis transactions which don't work in a Redis > cluster, it imposes a constraint that the Redis Mode cannot be set to > Cluster, otherwise it is invalid. The downside is that then the Redis DMC > can't be used with a Redis cluster, even in places that didn't require atomic > updates like Put/Fetch DMC. We should be able to improve this. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] bbende commented on pull request #7796: NIFI-12104 Separate a non-atomic Redis DMC implementation from the ex…
bbende commented on PR #7796: URL: https://github.com/apache/nifi/pull/7796#issuecomment-1741389094 Thanks @MikeThomsen and @gresockj I was able to resolve the cherry-pick to `support/nifi-1.x`, it was because there is one change for a new redis processor that was not back-ported there, so worked around that. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-12104) Allow Redis DMC to be used by Put/Fetch DMC when Redis is clustered
[ https://issues.apache.org/jira/browse/NIFI-12104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Bende updated NIFI-12104: --- Fix Version/s: 2.0.0 1.24.0 (was: 2.latest) > Allow Redis DMC to be used by Put/Fetch DMC when Redis is clustered > --- > > Key: NIFI-12104 > URL: https://issues.apache.org/jira/browse/NIFI-12104 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.23.2 >Reporter: Bryan Bende >Assignee: Bryan Bende >Priority: Minor > Fix For: 2.0.0, 1.24.0 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > The Redis DMC directly implements Atomic DMC Client and because it implements > the atomic updates using Redis transactions which don't work in a Redis > cluster, it imposes a constraint that the Redis Mode cannot be set to > Cluster, otherwise it is invalid. The downside is that then the Redis DMC > can't be used with a Redis cluster, even in places that didn't require atomic > updates like Put/Fetch DMC. We should be able to improve this. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] exceptionfactory closed pull request #7815: NIFI-12150 restoring NiFiRegistrySecurityConfig as it is a key spring…
exceptionfactory closed pull request #7815: NIFI-12150 restoring NiFiRegistrySecurityConfig as it is a key spring… URL: https://github.com/apache/nifi/pull/7815 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-12147) Update NiFi docker images to use JRE instead of JDK and make NiFi Registry image use Java 21
[ https://issues.apache.org/jira/browse/NIFI-12147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17770578#comment-17770578 ] ASF subversion and git services commented on NIFI-12147: Commit 67ba13796e3d34b43daec25a28f93febabd88aec in nifi's branch refs/heads/main from Joe Witt [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=67ba13796e ] NIFI-12150 Restored NiFiRegistrySecurityConfig required for Registry - NiFiRegistrySecurityConfig was removed mistakenly in NIFI-12147 and updating extractor to match changes in NIFI-12147 for DN extraction This closes #7815 Signed-off-by: David Handermann > Update NiFi docker images to use JRE instead of JDK and make NiFi Registry > image use Java 21 > > > Key: NIFI-12147 > URL: https://issues.apache.org/jira/browse/NIFI-12147 > Project: Apache NiFi > Issue Type: Task >Reporter: Joe Witt >Assignee: Joe Witt >Priority: Major > Fix For: 2.latest > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12150) Restore any deleted classes that are still required such as NiFiRegistrySecurityConfig
[ https://issues.apache.org/jira/browse/NIFI-12150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17770577#comment-17770577 ] ASF subversion and git services commented on NIFI-12150: Commit 67ba13796e3d34b43daec25a28f93febabd88aec in nifi's branch refs/heads/main from Joe Witt [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=67ba13796e ] NIFI-12150 Restored NiFiRegistrySecurityConfig required for Registry - NiFiRegistrySecurityConfig was removed mistakenly in NIFI-12147 and updating extractor to match changes in NIFI-12147 for DN extraction This closes #7815 Signed-off-by: David Handermann > Restore any deleted classes that are still required such as > NiFiRegistrySecurityConfig > -- > > Key: NIFI-12150 > URL: https://issues.apache.org/jira/browse/NIFI-12150 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Joe Witt >Assignee: Joe Witt >Priority: Major > Fix For: 2.0.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > In NIFI-12142 we removed too much. Notable the spring bean necessary for > security config for the registry. Will search for any other deleted classes. > The method used to find unreadable/unused classes was too aggressive. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12150) Restore any deleted classes that are still required such as NiFiRegistrySecurityConfig
[ https://issues.apache.org/jira/browse/NIFI-12150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-12150: Resolution: Fixed Status: Resolved (was: Patch Available) > Restore any deleted classes that are still required such as > NiFiRegistrySecurityConfig > -- > > Key: NIFI-12150 > URL: https://issues.apache.org/jira/browse/NIFI-12150 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Joe Witt >Assignee: Joe Witt >Priority: Major > Fix For: 2.0.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > In NIFI-12142 we removed too much. Notable the spring bean necessary for > security config for the registry. Will search for any other deleted classes. > The method used to find unreadable/unused classes was too aggressive. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11627) Add Dynamic Schema References to ValidateJSON Processor
[
https://issues.apache.org/jira/browse/NIFI-11627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17770581#comment-17770581
]
Daniel Stieglitz commented on NIFI-11627:
-
[~exceptionfactory] I do not have a specific use case for a schema in an
attribute but I thought to align ValidateJson with JoltTransformJSON.
> Add Dynamic Schema References to ValidateJSON Processor
> ---
>
> Key: NIFI-11627
> URL: https://issues.apache.org/jira/browse/NIFI-11627
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Affects Versions: 1.19.1
>Reporter: Chuck Tilly
>Assignee: Daniel Stieglitz
>Priority: Major
>
> For the ValidateJSON processor, add support for flowfile attribute references
> that will allow for a JSON schema located in the Parameter Contexts, to be
> referenced dynamically based on a flowfile attribute. e.g.
> {code:java}
> #{${schema.name}} {code}
>
> The benefits of adding support for attribute references are significant.
> Adding this capability will allow a single processor to be used for all JSON
> schema validation. Unfortunately, the current version of this processor
> requires a dedicated processor for every schema, i.e. 12 schemas requires 12
> ValidateJSON processors. This is very laborious to construct and maintain,
> and resource expensive.
> ValidateJSON processor (https://issues.apache.org/jira/browse/NIFI-7392)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (NIFI-12151) StandardPrivateKeyService fails due to missing BouncyCastleProvider
Peter Turcsanyi created NIFI-12151:
--
Summary: StandardPrivateKeyService fails due to missing
BouncyCastleProvider
Key: NIFI-12151
URL: https://issues.apache.org/jira/browse/NIFI-12151
Project: Apache NiFi
Issue Type: Bug
Reporter: Peter Turcsanyi
Assignee: Peter Turcsanyi
{{StandardPrivateKeyService}} depends on {{BouncyCastleProvider}} but it does
not register it or pass it to BC methods calls. The service was relying on
other components registering BC. Like {{EncryptContent}} which was removed
recently.
{{StandardPrivateKeyService}} should handle {{BouncyCastleProvider}} on its
own. It can be done via passing {{BouncyCastleProvider}} instance to the key
decryptor classes.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-11627) Add Dynamic Schema References to ValidateJSON Processor
[
https://issues.apache.org/jira/browse/NIFI-11627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17770584#comment-17770584
]
David Handermann commented on NIFI-11627:
-
Thanks for the reply [~dstiegli1]. Supporting FlowFile attributes works well
for JoltTransformJSON as it allows for minor adjustments in transformation.
However, JSON schema validation is a different use case where the schema should
not change for each FlowFile. For this reasons, supporting FlowFile attributes
in ValidateJson does not appear to be the best way forward.
> Add Dynamic Schema References to ValidateJSON Processor
> ---
>
> Key: NIFI-11627
> URL: https://issues.apache.org/jira/browse/NIFI-11627
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Affects Versions: 1.19.1
>Reporter: Chuck Tilly
>Assignee: Daniel Stieglitz
>Priority: Major
>
> For the ValidateJSON processor, add support for flowfile attribute references
> that will allow for a JSON schema located in the Parameter Contexts, to be
> referenced dynamically based on a flowfile attribute. e.g.
> {code:java}
> #{${schema.name}} {code}
>
> The benefits of adding support for attribute references are significant.
> Adding this capability will allow a single processor to be used for all JSON
> schema validation. Unfortunately, the current version of this processor
> requires a dedicated processor for every schema, i.e. 12 schemas requires 12
> ValidateJSON processors. This is very laborious to construct and maintain,
> and resource expensive.
> ValidateJSON processor (https://issues.apache.org/jira/browse/NIFI-7392)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-11841) Nifi in cluster setup doesn't stop within the graceful shutdown period
[
https://issues.apache.org/jira/browse/NIFI-11841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17770585#comment-17770585
]
Michael W Moser commented on NIFI-11841:
I have duplicated this problem and I will be looking more into it.
If I use embedded Zookeeper with just 1 server in the ZK cluster, I do not see
this problem. As soon as I increase the number of ZK servers to 2 or 3, I
begin to see the shutdown not happen gracefully.
> Nifi in cluster setup doesn't stop within the graceful shutdown period
> --
>
> Key: NIFI-11841
> URL: https://issues.apache.org/jira/browse/NIFI-11841
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.22.0
>Reporter: Dheeraj Joshi
>Priority: Major
> Attachments: Logs.zip, bootstrap.conf, nifi-app.log,
> nifi-bootstrap.log, nifi.properties, zookeeper.properties
>
>
> Bootstrap logs show the following logs when one of the nodes in a 3-node
> cluster is stopped
> {code:java}
> 2023-07-20 11:10:49,847 INFO [main] org.apache.nifi.bootstrap.Command
> Launched Apache NiFi with Process ID 18579
> 2023-07-20 11:10:50,475 INFO [NiFi Bootstrap Command Listener]
> org.apache.nifi.bootstrap.RunNiFi Apache NiFi now running and listening for
> Bootstrap requests on port 64739
> 2023-07-20 11:12:15,723 INFO [main] o.a.n.b.NotificationServiceManager
> Successfully loaded the following 0 services: []
> 2023-07-20 11:12:15,725 INFO [main] org.apache.nifi.bootstrap.RunNiFi
> Registered no Notification Services for Notification Type NIFI_STARTED
> 2023-07-20 11:12:15,725 INFO [main] org.apache.nifi.bootstrap.RunNiFi
> Registered no Notification Services for Notification Type NIFI_STOPPED
> 2023-07-20 11:12:15,725 INFO [main] org.apache.nifi.bootstrap.RunNiFi
> Registered no Notification Services for Notification Type NIFI_DIED
> 2023-07-20 11:12:15,750 INFO [main] org.apache.nifi.bootstrap.Command Apache
> NiFi has accepted the Shutdown Command and is shutting down now
> 2023-07-20 11:12:15,859 INFO [main] org.apache.nifi.bootstrap.Command NiFi
> PID [18579] shutdown in progress...
> 2023-07-20 11:12:17,896 INFO [main] org.apache.nifi.bootstrap.Command NiFi
> PID [18579] shutdown in progress...
> 2023-07-20 11:12:19,919 INFO [main] org.apache.nifi.bootstrap.Command NiFi
> PID [18579] shutdown in progress...{code}
> The shutdown in progress message gets repeated till
> {code:java}
> graceful.shutdown.seconds{code}
> is elapsed.
> Nifi app logs print socket exception during the shutdown but eventually shuts
> down the application server server
> {code:java}
> 2023-07-20 11:12:15,750 INFO [pool-2-thread-2]
> org.apache.nifi.BootstrapListener Received SHUTDOWN request from Bootstrap
> 2023-07-20 11:12:15,750 INFO [pool-2-thread-2] org.apache.nifi.NiFi
> Application Server shutdown started
> 2023-07-20 11:12:15,755 INFO [pool-2-thread-2]
> o.eclipse.jetty.server.AbstractConnector Stopped
> ServerConnector@2407a36c{HTTP/1.1, (http/1.1)}
> {127.0.0.1:8080}
> 2023-07-20 11:12:15,756 INFO [pool-2-thread-2]
> org.eclipse.jetty.server.session node0 Stopped scavenging
> 2023-07-20 11:12:15,769 INFO [pool-2-thread-2]
> o.e.jetty.server.handler.ContextHandler Stopped
> o.e.j.w.WebAppContext@7cfb8e98{nifi-error,/,null,STOPPED}
> {./work/nar/extensions/nifi-server-nar-1.22.0.nar-unpacked/NAR-INF/bundled-dependencies/nifi-web-error-1.22.0.war}
> 2023-07-20 11:12:15,782 INFO [pool-2-thread-2]
> o.e.jetty.server.handler.ContextHandler Stopped
> o.e.j.w.WebAppContext@2aa5bd48{nifi-docs,/nifi-docs,null,STOPPED}
> {./work/nar/extensions/nifi-server-nar-1.22.0.nar-unpacked/NAR-INF/bundled-dependencies/nifi-web-docs-1.22.0.war}
> 2023-07-20 11:12:15,798 INFO [pool-2-thread-2]
> o.e.jetty.server.handler.ContextHandler Stopped
> o.e.j.w.WebAppContext@133aacbe{nifi-content-viewer,/nifi-content-viewer,null,STOPPED}
> {./work/nar/extensions/nifi-server-nar-1.22.0.nar-unpacked/NAR-INF/bundled-dependencies/nifi-web-content-viewer-1.22.0.war}
> 2023-07-20 11:12:15,803 INFO [pool-2-thread-2]
> o.a.n.w.c.ApplicationStartupContextListener Flow Service shutdown started
> 2023-07-20 11:12:15,803 INFO [pool-2-thread-2]
> o.a.n.c.c.node.NodeClusterCoordinator Status of 127.0.0.1:8080 changed from
> NodeConnectionStatus[nodeId=127.0.0.1:8080, state=CONNECTED, updateId=50] to
> NodeConnectionStatus[nodeId=127.0.0.1:8080, state=DISCONNECTED, Disconnect
> Code=Node was Shutdown, Disconnect Reason=Node was Shutdown, updateId=50]
> 2023-07-20 11:12:15,809 INFO [pool-2-thread-2]
> o.a.n.c.c.node.NodeClusterCoordinator Successfully notified other nodes that
> I am shutting down
> 2023-07-20 11:12:15,811 INFO [Curator-Framework-0]
> o.a.c.f.imps.CuratorFrameworkImpl backgroundOperationsLoop exiting
> 2023-07-20
[jira] [Comment Edited] (NIFI-11841) Nifi in cluster setup doesn't stop within the graceful shutdown period
[
https://issues.apache.org/jira/browse/NIFI-11841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17770585#comment-17770585
]
Michael W Moser edited comment on NIFI-11841 at 9/29/23 8:07 PM:
-
I have duplicated this problem and I will be looking more into it.
If I use embedded Zookeeper with just 1 server in the ZK cluster, I do not see
this problem. As soon as I increase the number of ZK servers to 2 or 3, I
begin to see the shutdown not happen gracefully.
I'm using 1.24.0-SNAPSHOT version of NiFi from the support/1.x branch.
was (Author: mosermw):
I have duplicated this problem and I will be looking more into it.
If I use embedded Zookeeper with just 1 server in the ZK cluster, I do not see
this problem. As soon as I increase the number of ZK servers to 2 or 3, I
begin to see the shutdown not happen gracefully.
> Nifi in cluster setup doesn't stop within the graceful shutdown period
> --
>
> Key: NIFI-11841
> URL: https://issues.apache.org/jira/browse/NIFI-11841
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.22.0
>Reporter: Dheeraj Joshi
>Priority: Major
> Attachments: Logs.zip, bootstrap.conf, nifi-app.log,
> nifi-bootstrap.log, nifi.properties, zookeeper.properties
>
>
> Bootstrap logs show the following logs when one of the nodes in a 3-node
> cluster is stopped
> {code:java}
> 2023-07-20 11:10:49,847 INFO [main] org.apache.nifi.bootstrap.Command
> Launched Apache NiFi with Process ID 18579
> 2023-07-20 11:10:50,475 INFO [NiFi Bootstrap Command Listener]
> org.apache.nifi.bootstrap.RunNiFi Apache NiFi now running and listening for
> Bootstrap requests on port 64739
> 2023-07-20 11:12:15,723 INFO [main] o.a.n.b.NotificationServiceManager
> Successfully loaded the following 0 services: []
> 2023-07-20 11:12:15,725 INFO [main] org.apache.nifi.bootstrap.RunNiFi
> Registered no Notification Services for Notification Type NIFI_STARTED
> 2023-07-20 11:12:15,725 INFO [main] org.apache.nifi.bootstrap.RunNiFi
> Registered no Notification Services for Notification Type NIFI_STOPPED
> 2023-07-20 11:12:15,725 INFO [main] org.apache.nifi.bootstrap.RunNiFi
> Registered no Notification Services for Notification Type NIFI_DIED
> 2023-07-20 11:12:15,750 INFO [main] org.apache.nifi.bootstrap.Command Apache
> NiFi has accepted the Shutdown Command and is shutting down now
> 2023-07-20 11:12:15,859 INFO [main] org.apache.nifi.bootstrap.Command NiFi
> PID [18579] shutdown in progress...
> 2023-07-20 11:12:17,896 INFO [main] org.apache.nifi.bootstrap.Command NiFi
> PID [18579] shutdown in progress...
> 2023-07-20 11:12:19,919 INFO [main] org.apache.nifi.bootstrap.Command NiFi
> PID [18579] shutdown in progress...{code}
> The shutdown in progress message gets repeated till
> {code:java}
> graceful.shutdown.seconds{code}
> is elapsed.
> Nifi app logs print socket exception during the shutdown but eventually shuts
> down the application server server
> {code:java}
> 2023-07-20 11:12:15,750 INFO [pool-2-thread-2]
> org.apache.nifi.BootstrapListener Received SHUTDOWN request from Bootstrap
> 2023-07-20 11:12:15,750 INFO [pool-2-thread-2] org.apache.nifi.NiFi
> Application Server shutdown started
> 2023-07-20 11:12:15,755 INFO [pool-2-thread-2]
> o.eclipse.jetty.server.AbstractConnector Stopped
> ServerConnector@2407a36c{HTTP/1.1, (http/1.1)}
> {127.0.0.1:8080}
> 2023-07-20 11:12:15,756 INFO [pool-2-thread-2]
> org.eclipse.jetty.server.session node0 Stopped scavenging
> 2023-07-20 11:12:15,769 INFO [pool-2-thread-2]
> o.e.jetty.server.handler.ContextHandler Stopped
> o.e.j.w.WebAppContext@7cfb8e98{nifi-error,/,null,STOPPED}
> {./work/nar/extensions/nifi-server-nar-1.22.0.nar-unpacked/NAR-INF/bundled-dependencies/nifi-web-error-1.22.0.war}
> 2023-07-20 11:12:15,782 INFO [pool-2-thread-2]
> o.e.jetty.server.handler.ContextHandler Stopped
> o.e.j.w.WebAppContext@2aa5bd48{nifi-docs,/nifi-docs,null,STOPPED}
> {./work/nar/extensions/nifi-server-nar-1.22.0.nar-unpacked/NAR-INF/bundled-dependencies/nifi-web-docs-1.22.0.war}
> 2023-07-20 11:12:15,798 INFO [pool-2-thread-2]
> o.e.jetty.server.handler.ContextHandler Stopped
> o.e.j.w.WebAppContext@133aacbe{nifi-content-viewer,/nifi-content-viewer,null,STOPPED}
> {./work/nar/extensions/nifi-server-nar-1.22.0.nar-unpacked/NAR-INF/bundled-dependencies/nifi-web-content-viewer-1.22.0.war}
> 2023-07-20 11:12:15,803 INFO [pool-2-thread-2]
> o.a.n.w.c.ApplicationStartupContextListener Flow Service shutdown started
> 2023-07-20 11:12:15,803 INFO [pool-2-thread-2]
> o.a.n.c.c.node.NodeClusterCoordinator Status of 127.0.0.1:8080 changed from
> NodeConnectionStatus[nodeId=127.0.0.1:8080, state=CONNECTED, updateId=50] to
> NodeConnectionStatus
[GitHub] [nifi] exceptionfactory commented on a diff in pull request #7781: NIFI-11463 Allow IdentifyMimeType to use custom MIME types in addition to defaults
exceptionfactory commented on code in PR #7781:
URL: https://github.com/apache/nifi/pull/7781#discussion_r1341754971
##
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/IdentifyMimeType.java:
##
@@ -99,6 +101,9 @@
}
)
public class IdentifyMimeType extends AbstractProcessor {
+static final AllowableValue REPLACE = new AllowableValue("replace",
"replace", "Use config MIME Types only.");
+static final AllowableValue ADD = new AllowableValue("add", "add",
Review Comment:
Recommend adjusting the casing of the values:
```suggestion
static final AllowableValue REPLACE = new AllowableValue("Replace",
"Replace", "Use config MIME Types only.");
static final AllowableValue ADD = new AllowableValue("Add", "Add",
```
##
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/IdentifyMimeType.java:
##
@@ -206,34 +236,50 @@ public void onTrigger(final ProcessContext context, final
ProcessSession session
final ComponentLog logger = getLogger();
final AtomicReference mimeTypeRef = new
AtomicReference<>(null);
+final AtomicReference extensionRef = new
AtomicReference<>(null);
final String filename =
flowFile.getAttribute(CoreAttributes.FILENAME.key());
session.read(flowFile, new InputStreamCallback() {
@Override
public void process(final InputStream stream) throws IOException {
try (final InputStream in = new BufferedInputStream(stream);
final TikaInputStream tikaStream =
TikaInputStream.get(in)) {
-Metadata metadata = new Metadata();
-if (filename != null &&
context.getProperty(USE_FILENAME_IN_DETECTION).asBoolean()) {
-metadata.add(TikaCoreProperties.RESOURCE_NAME_KEY,
filename);
+if (customDetector != null) {
+detectMimeType(customDetector, customMimeTypes,
tikaStream);
+}
+
+String mimeType = mimeTypeRef.get();
+
+if (defaultDetector != null && (mimeType == null ||
mimeType.equals(MediaType.OCTET_STREAM.toString()) ||
mimeType.equals(MediaType.TEXT_PLAIN.toString( {
+detectMimeType(defaultDetector, defaultMimeTypes,
tikaStream);
}
-// Get mime type
-MediaType mediatype = detector.detect(tikaStream,
metadata);
-mimeTypeRef.set(mediatype.toString());
}
}
+
+private void detectMimeType(Detector detector, MimeTypes
mimeTypes, TikaInputStream tikaStream) throws IOException {
+Metadata metadata = new Metadata();
+if (filename != null &&
context.getProperty(USE_FILENAME_IN_DETECTION).asBoolean()) {
+metadata.add(TikaCoreProperties.RESOURCE_NAME_KEY,
filename);
+}
+
+MediaType mediatype = detector.detect(tikaStream, metadata);
+String extension = "";
+
+try {
+MimeType mimetype =
mimeTypes.forName(mediatype.toString());
+extension = mimetype.getExtension();
+} catch (MimeTypeException ex) {
+logger.warn("MIME type extension lookup failed: {}", new
Object[]{ex});
Review Comment:
The `Object[]` wrapper is deprecated and should not be used.
```suggestion
logger.warn("MIME type extension lookup failed", e);
```
##
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/IdentifyMimeType.java:
##
@@ -161,31 +180,42 @@ protected void init(final ProcessorInitializationContext
context) {
public void setup(final ProcessContext context) {
String configBody = context.getProperty(MIME_CONFIG_BODY).getValue();
String configFile =
context.getProperty(MIME_CONFIG_FILE).evaluateAttributeExpressions().getValue();
+String configStrategy =
context.getProperty(CONFIG_STRATEGY).getValue();
-if (configBody == null && configFile == null){
-this.detector = config.getDetector();
-this.mimeTypes = config.getMimeRepository();
-} else if (configBody != null) {
-try {
-this.detector = MimeTypesFactory.create(new
ByteArrayInputStream(configBody.getBytes()));
-this.mimeTypes = (MimeTypes)this.detector;
-} catch (Exception e) {
-context.yield();
-throw new ProcessException("Failed to load config body", e);
-}
-
+if (configBody == null && configFile == null) {
+setDefaultMimeTypes();
+} else if (configStrategy.equals(ADD.getValue
[GitHub] [nifi] bbende opened a new pull request, #7816: NIFI-12149 Create nifi-redis-utils and minor improvements to util methods
bbende opened a new pull request, #7816: URL: https://github.com/apache/nifi/pull/7816 This is the 1.x version of #7812 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] turcsanyip opened a new pull request, #7817: NIFI-12151 Fixed StandardPrivateKeyService fails due to missing Bounc…
turcsanyip opened a new pull request, #7817: URL: https://github.com/apache/nifi/pull/7817 …yCastleProvider # Summary [NIFI-12151](https://issues.apache.org/jira/browse/NIFI-12151) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-12151) StandardPrivateKeyService fails due to missing BouncyCastleProvider
[
https://issues.apache.org/jira/browse/NIFI-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Turcsanyi updated NIFI-12151:
---
Status: Patch Available (was: Open)
> StandardPrivateKeyService fails due to missing BouncyCastleProvider
> ---
>
> Key: NIFI-12151
> URL: https://issues.apache.org/jira/browse/NIFI-12151
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Peter Turcsanyi
>Assignee: Peter Turcsanyi
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {{StandardPrivateKeyService}} depends on {{BouncyCastleProvider}} but it does
> not register it or pass it to BC methods calls. The service was relying on
> other components registering BC. Like {{EncryptContent}} which was removed
> recently.
> {{StandardPrivateKeyService}} should handle {{BouncyCastleProvider}} on its
> own. It can be done via passing {{BouncyCastleProvider}} instance to the key
> decryptor classes.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12151) StandardPrivateKeyService fails due to missing BouncyCastleProvider
[
https://issues.apache.org/jira/browse/NIFI-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Turcsanyi updated NIFI-12151:
---
Fix Version/s: 1.latest
2.latest
> StandardPrivateKeyService fails due to missing BouncyCastleProvider
> ---
>
> Key: NIFI-12151
> URL: https://issues.apache.org/jira/browse/NIFI-12151
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Peter Turcsanyi
>Assignee: Peter Turcsanyi
>Priority: Major
> Fix For: 1.latest, 2.latest
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {{StandardPrivateKeyService}} depends on {{BouncyCastleProvider}} but it does
> not register it or pass it to BC methods calls. The service was relying on
> other components registering BC. Like {{EncryptContent}} which was removed
> recently.
> {{StandardPrivateKeyService}} should handle {{BouncyCastleProvider}} on its
> own. It can be done via passing {{BouncyCastleProvider}} instance to the key
> decryptor classes.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (NIFI-12152) Refactor Security.addProvider References to Bouncy Castle
David Handermann created NIFI-12152: --- Summary: Refactor Security.addProvider References to Bouncy Castle Key: NIFI-12152 URL: https://issues.apache.org/jira/browse/NIFI-12152 Project: Apache NiFi Issue Type: Improvement Components: Extensions, MiNiFi Reporter: David Handermann Assignee: David Handermann Fix For: 2.0.0 Several tests and server components include Security.addProvider() statements that register the BouncyCastleProvider in static initialization methods. This change applies to the entire Java Virtual Machine. Most cryptographic operations support passing a Provider as an argument, and this approach should be used when the Bouncy Castle Provider is required for certain algorithms not included in the standard JDK, such as the Bouncy Castle FIPS Key Store BCFKS. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] exceptionfactory opened a new pull request, #7818: NIFI-12152 Refactored addProvider() Bouncy Castle references
exceptionfactory opened a new pull request, #7818: URL: https://github.com/apache/nifi/pull/7818 # Summary [NIFI-12152](https://issues.apache.org/jira/browse/NIFI-12152) Refactors references to `Security.addProvider()` with global registration of the Bouncy Castle Provider. Changes include removing unnecessary registration from multiple test classes and adjusting references to use an instance of `BouncyCastleProvider` instead of the `BC` provider name string that requires global registration. # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [X] Build completed using `mvn clean install -P contrib-check` - [X] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-11627) Add Dynamic Schema References to ValidateJSON Processor
[
https://issues.apache.org/jira/browse/NIFI-11627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17770593#comment-17770593
]
Chuck Tilly commented on NIFI-11627:
[~exceptionfactory] A specific use-case would be every single time you want
to validate your data, which is always. Hence the need for this processor.
But storing the schema in the attributes is not the solution, you are right
about that. Rather, it would be better if the flow file contained a +reference+
to its schema. The schema itself is stored as a value in the Parameter
Contexts, and the ValidateJSON Processor uses this schema to perform the
validation. This is exactly the same paradigm that the AvroSchemaRegistry
uses, except the schemas are stored in a Controller Service instead of a
Parameter Context (the principle is the same though). So assuming you have a
flowfile with an attribute named "schema.name" (e.g. schema.name = blue-cars),
then the syntax for referencing a schema stored as a Parameter Context value
would be:
JSON Schema = #\{${schema.name}}
Within the Parameter Contexts there would be a value "blue-cars", and it would
contain the JSON schema for blue-cars.
The benefits are:
1) Only a single ValidateJSON processor is needed to perform all the
validations. This allows for clean and simple flows that are easy to manage,
and consume fewer resources.
2) This approach is consistent with how schema validation is done in Nifi
using schema registries. It is a well established pattern in NiFi.
> Add Dynamic Schema References to ValidateJSON Processor
> ---
>
> Key: NIFI-11627
> URL: https://issues.apache.org/jira/browse/NIFI-11627
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Affects Versions: 1.19.1
>Reporter: Chuck Tilly
>Assignee: Daniel Stieglitz
>Priority: Major
>
> For the ValidateJSON processor, add support for flowfile attribute references
> that will allow for a JSON schema located in the Parameter Contexts, to be
> referenced dynamically based on a flowfile attribute. e.g.
> {code:java}
> #{${schema.name}} {code}
>
> The benefits of adding support for attribute references are significant.
> Adding this capability will allow a single processor to be used for all JSON
> schema validation. Unfortunately, the current version of this processor
> requires a dedicated processor for every schema, i.e. 12 schemas requires 12
> ValidateJSON processors. This is very laborious to construct and maintain,
> and resource expensive.
> ValidateJSON processor (https://issues.apache.org/jira/browse/NIFI-7392)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi] joewitt opened a new pull request, #7819: NIFI-12147 Cleaned poms, updated Dockerfile, fixed Jersey version to …
joewitt opened a new pull request, #7819: URL: https://github.com/apache/nifi/pull/7819 …get appropriate bundled ASM # Summary [NIFI-0](https://issues.apache.org/jira/browse/NIFI-0) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-11627) Add Dynamic Schema References to ValidateJSON Processor
[
https://issues.apache.org/jira/browse/NIFI-11627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17770614#comment-17770614
]
Mark Payne commented on NIFI-11627:
---
It does make sense to allow for a reference to be stored in a FlowFile
attribute and then reference it in much the same way that you would when
looking up a schema in a SchemaRegistry.
But Parameter Contexts are not the right approach. Making use of a Controller
Service would work well. We could have a Controller Service that allows
user-added properties where the values are JSON Schemas, and then allow
ValidateJson to be configured with a Controller Service and take in the name of
the schema, which would allow for Expression Language to be used. So I believe
that would give you exactly what you're looking for, [~nwchuckster], no?
The reason that Parameter Contexts won't work here are two-fold. Firstly, you
cannot use Expression Language within a Parameter Context because parameters'
values are resolved before the processor ever even has access to the property
value. So, if you were to enter #\{${schema.name}} what would happen is that
NiFi would resolve that to a parameter named ${schema.name} and the processor
would be invalid, before it ever had any chance to even evaluate Expression
Language.
Secondly, we never allow creating a String value and then evaluating it to get
a parameter due to security concerns. There are security policies that guard
who is allow to reference parameters, etc. and allowing dynamic creation of
parameter names would violate the security constraints.
> Add Dynamic Schema References to ValidateJSON Processor
> ---
>
> Key: NIFI-11627
> URL: https://issues.apache.org/jira/browse/NIFI-11627
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Affects Versions: 1.19.1
>Reporter: Chuck Tilly
>Assignee: Daniel Stieglitz
>Priority: Major
>
> For the ValidateJSON processor, add support for flowfile attribute references
> that will allow for a JSON schema located in the Parameter Contexts, to be
> referenced dynamically based on a flowfile attribute. e.g.
> {code:java}
> #{${schema.name}} {code}
>
> The benefits of adding support for attribute references are significant.
> Adding this capability will allow a single processor to be used for all JSON
> schema validation. Unfortunately, the current version of this processor
> requires a dedicated processor for every schema, i.e. 12 schemas requires 12
> ValidateJSON processors. This is very laborious to construct and maintain,
> and resource expensive.
> ValidateJSON processor (https://issues.apache.org/jira/browse/NIFI-7392)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
