[GitHub] [nifi-minifi-cpp] lordgamez commented on a change in pull request #1253: MINIFICPP-1736 Add MiNiFi agent C2 operation capabilities to agent manifest
lordgamez commented on a change in pull request #1253:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1253#discussion_r806563819
##
File path: libminifi/include/properties/Configuration.h
##
@@ -99,6 +139,30 @@ class Configuration : public Properties {
static constexpr const char *minifi_disk_space_watchdog_interval =
"minifi.disk.space.watchdog.interval";
static constexpr const char *minifi_disk_space_watchdog_stop_threshold =
"minifi.disk.space.watchdog.stop.threshold";
static constexpr const char *minifi_disk_space_watchdog_restart_threshold =
"minifi.disk.space.watchdog.restart.threshold";
+
+ // JNI options
+ static constexpr const char *nifi_framework_dir = "nifi.framework.dir";
+ static constexpr const char *nifi_jvm_options = "nifi.jvm.options";
+ static constexpr const char *nifi_nar_directory = "nifi.nar.directory";
+ static constexpr const char *nifi_nar_deploy_directory =
"nifi.nar.deploy.directory";
+
+ // Log options
+ static constexpr const char *nifi_log_spdlog_pattern =
"nifi.log.spdlog.pattern";
+ static constexpr const char *nifi_log_spdlog_shorten_names =
"nifi.log.spdlog.shorten_names";
+ static constexpr const char *nifi_log_appender_rolling =
"nifi.log.appender.rolling";
+ static constexpr const char *nifi_log_appender_rolling_directory =
"nifi.log.appender.rolling.directory";
+ static constexpr const char *nifi_log_appender_rolling_file_name =
"nifi.log.appender.rolling.file_name";
+ static constexpr const char *nifi_log_appender_rolling_max_files =
"nifi.log.appender.rolling.max_files";
+ static constexpr const char *nifi_log_appender_rolling_max_file_size =
"nifi.log.appender.rolling.max_file_size";
+ static constexpr const char *nifi_log_appender_stdout =
"nifi.log.appender.stdout";
+ static constexpr const char *nifi_log_appender_stderr =
"nifi.log.appender.stderr";
+ static constexpr const char *nifi_log_appender_null =
"nifi.log.appender.null";
+ static constexpr const char *nifi_log_appender_syslog =
"nifi.log.appender.syslog";
+ static constexpr const char *nifi_log_logger_root = "nifi.log.logger.root";
+ static constexpr const char *nifi_log_compression_cached_log_max_size =
"nifi.log.compression.cached.log.max.size";
+ static constexpr const char *nifi_log_compression_compressed_log_max_size =
"nifi.log.compression.compressed.log.max.size";
+
+ MINIFIAPI static const std::vector
CONFIGURATION_PROPERTIES;
Review comment:
The reason the vector was introduced to be able to iterate through all
the available properties to be added to the agent manifest.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] lawye1973 opened a new pull request #5767: Correct FetchS3Object.java for provenance data URL
lawye1973 opened a new pull request #5767:
URL: https://github.com/apache/nifi/pull/5767
Current provenance data URL is incorrect as it's hardcoded.
Follow PutS3Object.java method, using AmazonS3Client (instead of AmazonS3)
to use getResourceUrl(bucket,key) to build the correct URL for provenance data
Thank you for submitting a contribution to Apache NiFi.
Please provide a short description of the PR here:
Description of PR
_Enables X functionality; fixes bug NIFI-._
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [ ] Does your PR title start with **NIFI-** where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [ ] Has your PR been rebased against the latest commit within the target
branch (typically `main`)?
- [ ] Is your initial contribution a single, squashed commit? _Additional
commits in response to PR reviewer feedback should be made on this branch and
pushed to allow change tracking. Do not `squash` or use `--force` when pushing
to allow for clean monitoring of changes._
### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via `mvn
-Pcontrib-check clean install` at the root `nifi` folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] Have you verified that the full build is successful on JDK 8?
- [ ] Have you verified that the full build is successful on JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main
`LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main
`NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI for
build issues and submit an update to your PR as soon as possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a change in pull request #1253: MINIFICPP-1736 Add MiNiFi agent C2 operation capabilities to agent manifest
lordgamez commented on a change in pull request #1253:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1253#discussion_r806580053
##
File path: libminifi/src/c2/C2Agent.cpp
##
@@ -334,44 +334,7 @@ struct C2DebugBundleError : public C2TransferError {
void C2Agent::handle_c2_server_response(const C2ContentResponse &resp) {
switch (resp.op.value()) {
case Operation::CLEAR:
- // we've been told to clear something
- if (resp.name == "connection") {
-for (const auto& connection : resp.operation_arguments) {
- logger_->log_debug("Clearing connection %s",
connection.second.to_string());
- update_sink_->clearConnection(connection.second.to_string());
-}
-C2Payload response(Operation::ACKNOWLEDGE, resp.ident, true);
-enqueue_c2_response(std::move(response));
- } else if (resp.name == "repositories") {
-update_sink_->drainRepositories();
-C2Payload response(Operation::ACKNOWLEDGE, resp.ident, true);
-enqueue_c2_response(std::move(response));
- } else if (resp.name == "corecomponentstate") {
-// TODO(bakaid): untested
-std::vector> components =
update_sink_->getComponents(resp.name);
-auto state_manager_provider =
core::ProcessContext::getStateManagerProvider(logger_, controller_,
configuration_);
-if (state_manager_provider != nullptr) {
- for (auto &component : components) {
-logger_->log_debug("Clearing state for component %s",
component->getComponentName());
-auto state_manager =
state_manager_provider->getCoreComponentStateManager(component->getComponentUUID());
-if (state_manager != nullptr) {
- component->stop();
- state_manager->clear();
- state_manager->persist();
- component->start();
-} else {
- logger_->log_warn("Failed to get StateManager for component %s",
component->getComponentUUID().to_string());
-}
- }
-} else {
- logger_->log_error("Failed to get StateManagerProvider");
-}
-C2Payload response(Operation::ACKNOWLEDGE, resp.ident, true);
-enqueue_c2_response(std::move(response));
- } else {
-logger_->log_debug("Clearing unknown %s", resp.name);
- }
-
+ handle_clear(resp);
Review comment:
The definition of the function changed in both PRs in different ways, so
it will definitely conflict either way. I would keep it this way for now and
would resolve the conflict when the other PR is merged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a change in pull request #1253: MINIFICPP-1736 Add MiNiFi agent C2 operation capabilities to agent manifest
lordgamez commented on a change in pull request #1253:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1253#discussion_r806585558
##
File path: encrypt-config/tests/ConfigFileEncryptorTests.cpp
##
@@ -76,13 +77,13 @@ TEST_CASE("ConfigFileEncryptor can encrypt the sensitive
properties", "[encrypt-
SECTION("default properties") {
ConfigFile test_file{std::ifstream{"resources/minifi.properties"}};
-std::string original_password =
test_file.getValue("nifi.rest.api.password").value();
+std::string original_password =
test_file.getValue(org::apache::nifi::minifi::Configuration::nifi_rest_api_password).value();
Review comment:
Updated in 107193764914b719074c45b1ce4b978c3d1292f4
##
File path: encrypt-config/ConfigFile.cpp
##
@@ -23,11 +23,10 @@
#include
#include "utils/StringUtils.h"
+#include "properties/Configuration.h"
namespace {
-constexpr std::array
DEFAULT_SENSITIVE_PROPERTIES{"nifi.security.client.pass.phrase",
-
"nifi.rest.api.password"};
-constexpr const char* ADDITIONAL_SENSITIVE_PROPS_PROPERTY_NAME =
"nifi.sensitive.props.additional.keys";
+
Review comment:
Updated in 107193764914b719074c45b1ce4b978c3d1292f4
##
File path: extensions/http-curl/protocols/RESTSender.cpp
##
@@ -43,9 +44,9 @@ void
RESTSender::initialize(core::controller::ControllerServiceProvider* control
// base URL when one is not specified.
if (nullptr != configure) {
std::string update_str, ssl_context_service_str;
-configure->get("nifi.c2.rest.url", "c2.rest.url", rest_uri_);
-configure->get("nifi.c2.rest.url.ack", "c2.rest.url.ack", ack_uri_);
-if (configure->get("nifi.c2.rest.ssl.context.service",
"c2.rest.ssl.context.service", ssl_context_service_str)) {
+configure->get(minifi::Configuration::nifi_c2_rest_url, "c2.rest.url",
rest_uri_);
+configure->get(minifi::Configuration::nifi_c2_rest_url_ack,
"c2.rest.url.ack", ack_uri_);
+if
(configure->get(minifi::Configuration::nifi_c2_rest_ssl_context_service,
"c2.rest.ssl.context.service", ssl_context_service_str)) {
Review comment:
Updated in 107193764914b719074c45b1ce4b978c3d1292f4
##
File path: extensions/http-curl/protocols/RESTSender.cpp
##
@@ -74,8 +75,8 @@ C2Payload RESTSender::consumePayload(const C2Payload
&payload, Direction directi
void RESTSender::update(const std::shared_ptr &configure) {
std::string url;
- configure->get("nifi.c2.rest.url", "c2.rest.url", url);
- configure->get("nifi.c2.rest.url.ack", "c2.rest.url.ack", url);
+ configure->get(minifi::Configuration::nifi_c2_rest_url, "c2.rest.url", url);
+ configure->get(minifi::Configuration::nifi_c2_rest_url_ack,
"c2.rest.url.ack", url);
Review comment:
Updated in 107193764914b719074c45b1ce4b978c3d1292f4
##
File path: libminifi/include/core/Property.h
##
@@ -467,6 +468,16 @@ class ConstrainedProperty : public
std::enable_shared_from_this>& validator_ =
StandardValidators::get().VALID_VALIDATOR)
+: name(name_),
+ validator(validator_) {
+ }
+
+ std::string_view name;
+ gsl::not_null> validator;
Review comment:
Updated in 107193764914b719074c45b1ce4b978c3d1292f4
##
File path: libminifi/src/Configure.cpp
##
@@ -68,15 +69,15 @@ bool Configure::isEncrypted(const std::string& key) const {
std::optional Configure::getAgentClass() const {
std::string agent_class;
- if (get("nifi.c2.agent.class", "c2.agent.class", agent_class) &&
!agent_class.empty()) {
+ if (get(minifi::Configuration::nifi_c2_agent_class, "c2.agent.class",
agent_class) && !agent_class.empty()) {
Review comment:
Updated in 107193764914b719074c45b1ce4b978c3d1292f4
##
File path: libminifi/src/Configure.cpp
##
@@ -68,15 +69,15 @@ bool Configure::isEncrypted(const std::string& key) const {
std::optional Configure::getAgentClass() const {
std::string agent_class;
- if (get("nifi.c2.agent.class", "c2.agent.class", agent_class) &&
!agent_class.empty()) {
+ if (get(minifi::Configuration::nifi_c2_agent_class, "c2.agent.class",
agent_class) && !agent_class.empty()) {
return agent_class;
}
return {};
}
std::string Configure::getAgentIdentifier() const {
std::string agent_id;
- if (!get("nifi.c2.agent.identifier", "c2.agent.identifier", agent_id) ||
agent_id.empty()) {
+ if (!get(minifi::Configuration::nifi_c2_agent_identifier,
"c2.agent.identifier", agent_id) || agent_id.empty()) {
Review comment:
Updated in 107193764914b719074c45b1ce4b978c3d1292f4
##
File path: libminifi/src/c2/C2Agent.cpp
##
@@ -138,7 +138,7 @@ void C2Agent::configure(const std::shared_ptr
&configure, bool reconf
std::string clazz, heartbeat_period, device;
if (!reconfigure) {
-if (!configure->get("nifi.c2.agent.protocol.class",
"c2.agent.protocol.class", clazz)) {
+if (!configure->get(minifi::
[jira] [Created] (MINIFICPP-1758) Log received C2 operations
Arpad Boda created MINIFICPP-1758: - Summary: Log received C2 operations Key: MINIFICPP-1758 URL: https://issues.apache.org/jira/browse/MINIFICPP-1758 Project: Apache NiFi MiNiFi C++ Issue Type: Improvement Reporter: Arpad Boda Assignee: Arpad Boda Logging received C2 operations would be helpful to debug and improve C2 related features. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [nifi-minifi-cpp] fgerlits commented on a change in pull request #1259: MINIFICPP-1748 - Modify log properties through the c2 protocol
fgerlits commented on a change in pull request #1259:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1259#discussion_r806621990
##
File path: libminifi/include/properties/Properties.h
##
@@ -29,16 +29,23 @@
#include "core/logging/Logger.h"
#include "utils/ChecksumCalculator.h"
+#include "utils/StringUtils.h"
namespace org {
namespace apache {
namespace nifi {
namespace minifi {
+enum class PropertyChangeLifetime {
+ TRANSIENT, // the changed value will not be committed to disk
+ PERSISTENT // the changed value will be written to the source file
+};
+
class Properties {
struct PropertyValue {
-std::string value;
-bool changed;
+std::string persisted_value;
+std::string active_value;
+bool changed{false}; // persisted_value should be written to disk if
requested
Review comment:
I would rename this to something like `need_to_persist_new_value`, and
then the comment is no longer needed
##
File path: libminifi/src/core/logging/LoggerConfiguration.cpp
##
@@ -359,6 +323,57 @@ void LoggerConfiguration::initializeCompression(const
std::lock_guard
LoggerConfiguration::getRotatingFileSink(const std::string& appender_key, const
std::shared_ptr& properties) {
+ static std::map> rotating_file_sinks;
+ static std::mutex sink_map_mtx;
+
+ std::string file_name;
+ if (!properties->getString(appender_key + ".file_name", file_name)) {
+file_name = "minifi-app.log";
+ }
+ std::string directory;
+ if (!properties->getString(appender_key + ".directory", directory)) {
+// The below part assumes logger_properties->getHome() is existing
+// Cause minifiHome must be set at MiNiFiMain.cpp?
+directory = properties->getHome() +
utils::file::FileUtils::get_separator() + "logs";
+ }
+
+ file_name = directory + utils::file::FileUtils::get_separator() + file_name;
+ if (utils::file::FileUtils::create_dir(directory) == -1) {
+std::cerr << directory << " cannot be created\n";
+exit(1);
+ }
+
+ int max_files = 3;
+ std::string max_files_str = "";
+ if (properties->getString(appender_key + ".max_files", max_files_str)) {
+try {
+ max_files = std::stoi(max_files_str);
+} catch (const std::invalid_argument &) {
+} catch (const std::out_of_range &) {
+}
+ }
+
+ int max_file_size = 5_MiB;
+ std::string max_file_size_str = "";
+ if (properties->getString(appender_key + ".max_file_size",
max_file_size_str)) {
+try {
+ max_file_size = std::stoi(max_file_size_str);
+} catch (const std::invalid_argument &) {
+} catch (const std::out_of_range &) {
+}
+ }
+
+ std::lock_guard guard(sink_map_mtx);
+ auto it = rotating_file_sinks.find(file_name);
+ if (it != rotating_file_sinks.end()) {
+return it->second;
+ }
Review comment:
What is the reason for this new caching? It looks a bit weird that we
do it for the rolling appender but not for the other appender types. Is it
possible that some update will not get applied because of a cached old value?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] asfgit closed pull request #5766: NIFI-9685 Upgrade JNA to 5.10.0
asfgit closed pull request #5766: URL: https://github.com/apache/nifi/pull/5766 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-9685) Upgrade JNA to 5.10.0
[
https://issues.apache.org/jira/browse/NIFI-9685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-9685:
-
Fix Version/s: 1.16.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Upgrade JNA to 5.10.0
> -
>
> Key: NIFI-9685
> URL: https://issues.apache.org/jira/browse/NIFI-9685
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework, Extensions
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Minor
> Fix For: 1.16.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The {{nifi-bootstrap-utils}} module uses JNA to resolve the Process
> Identifier when running on Windows, and the
> {{nifi-windows-event-log-processors}} module uses JNA for reading Windows
> Event Logs. The {{nifi-toolkit-cli}} also relies on JLine and JNA for command
> parsing. These references should be upgraded to the latest version.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (NIFI-9685) Upgrade JNA to 5.10.0
[
https://issues.apache.org/jira/browse/NIFI-9685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17492487#comment-17492487
]
ASF subversion and git services commented on NIFI-9685:
---
Commit b09a778d9016e09f23c3aa065f740bb7156e07ec in nifi's branch
refs/heads/main from David Handermann
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=b09a778 ]
NIFI-9685 Upgraded JNA to 5.10.0
- Upgraded nifi-bootstrap-utils JNA from 4.4.0 to 5.10.0
- Upgraded nifi-windows-event-log-processors JNA from 4.5.2 to 5.10.0
- Upgraded nifi-toolkit-cli JLine from 3.5.2 to 3.21.0
Signed-off-by: Pierre Villard
This closes #5766.
> Upgrade JNA to 5.10.0
> -
>
> Key: NIFI-9685
> URL: https://issues.apache.org/jira/browse/NIFI-9685
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework, Extensions
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Minor
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The {{nifi-bootstrap-utils}} module uses JNA to resolve the Process
> Identifier when running on Windows, and the
> {{nifi-windows-event-log-processors}} module uses JNA for reading Windows
> Event Logs. The {{nifi-toolkit-cli}} also relies on JLine and JNA for command
> parsing. These references should be upgraded to the latest version.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #1259: MINIFICPP-1748 - Modify log properties through the c2 protocol
adamdebreceni commented on a change in pull request #1259:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1259#discussion_r806660908
##
File path: libminifi/include/properties/Properties.h
##
@@ -29,16 +29,23 @@
#include "core/logging/Logger.h"
#include "utils/ChecksumCalculator.h"
+#include "utils/StringUtils.h"
namespace org {
namespace apache {
namespace nifi {
namespace minifi {
+enum class PropertyChangeLifetime {
+ TRANSIENT, // the changed value will not be committed to disk
+ PERSISTENT // the changed value will be written to the source file
+};
+
class Properties {
struct PropertyValue {
-std::string value;
-bool changed;
+std::string persisted_value;
+std::string active_value;
+bool changed{false}; // persisted_value should be written to disk if
requested
Review comment:
renamed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #1259: MINIFICPP-1748 - Modify log properties through the c2 protocol
adamdebreceni commented on a change in pull request #1259:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1259#discussion_r806664139
##
File path: libminifi/src/core/logging/LoggerConfiguration.cpp
##
@@ -359,6 +323,57 @@ void LoggerConfiguration::initializeCompression(const
std::lock_guard
LoggerConfiguration::getRotatingFileSink(const std::string& appender_key, const
std::shared_ptr& properties) {
+ static std::map> rotating_file_sinks;
+ static std::mutex sink_map_mtx;
+
+ std::string file_name;
+ if (!properties->getString(appender_key + ".file_name", file_name)) {
+file_name = "minifi-app.log";
+ }
+ std::string directory;
+ if (!properties->getString(appender_key + ".directory", directory)) {
+// The below part assumes logger_properties->getHome() is existing
+// Cause minifiHome must be set at MiNiFiMain.cpp?
+directory = properties->getHome() +
utils::file::FileUtils::get_separator() + "logs";
+ }
+
+ file_name = directory + utils::file::FileUtils::get_separator() + file_name;
+ if (utils::file::FileUtils::create_dir(directory) == -1) {
+std::cerr << directory << " cannot be created\n";
+exit(1);
+ }
+
+ int max_files = 3;
+ std::string max_files_str = "";
+ if (properties->getString(appender_key + ".max_files", max_files_str)) {
+try {
+ max_files = std::stoi(max_files_str);
+} catch (const std::invalid_argument &) {
+} catch (const std::out_of_range &) {
+}
+ }
+
+ int max_file_size = 5_MiB;
+ std::string max_file_size_str = "";
+ if (properties->getString(appender_key + ".max_file_size",
max_file_size_str)) {
+try {
+ max_file_size = std::stoi(max_file_size_str);
+} catch (const std::invalid_argument &) {
+} catch (const std::out_of_range &) {
+}
+ }
+
+ std::lock_guard guard(sink_map_mtx);
+ auto it = rotating_file_sinks.find(file_name);
+ if (it != rotating_file_sinks.end()) {
+return it->second;
+ }
Review comment:
added comment: in the spdlog implementation I found no guarantee that
multiple sinks to the same directory won't interfere with each other (it might
be guaranteed at the OS level though), also found this
https://github.com/gabime/spdlog/discussions/2151
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #1259: MINIFICPP-1748 - Modify log properties through the c2 protocol
adamdebreceni commented on a change in pull request #1259:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1259#discussion_r806665620
##
File path: libminifi/src/core/logging/LoggerConfiguration.cpp
##
@@ -359,6 +323,57 @@ void LoggerConfiguration::initializeCompression(const
std::lock_guard
LoggerConfiguration::getRotatingFileSink(const std::string& appender_key, const
std::shared_ptr& properties) {
+ static std::map> rotating_file_sinks;
+ static std::mutex sink_map_mtx;
+
+ std::string file_name;
+ if (!properties->getString(appender_key + ".file_name", file_name)) {
+file_name = "minifi-app.log";
+ }
+ std::string directory;
+ if (!properties->getString(appender_key + ".directory", directory)) {
+// The below part assumes logger_properties->getHome() is existing
+// Cause minifiHome must be set at MiNiFiMain.cpp?
+directory = properties->getHome() +
utils::file::FileUtils::get_separator() + "logs";
+ }
+
+ file_name = directory + utils::file::FileUtils::get_separator() + file_name;
+ if (utils::file::FileUtils::create_dir(directory) == -1) {
+std::cerr << directory << " cannot be created\n";
+exit(1);
+ }
+
+ int max_files = 3;
+ std::string max_files_str = "";
+ if (properties->getString(appender_key + ".max_files", max_files_str)) {
+try {
+ max_files = std::stoi(max_files_str);
+} catch (const std::invalid_argument &) {
+} catch (const std::out_of_range &) {
+}
+ }
+
+ int max_file_size = 5_MiB;
+ std::string max_file_size_str = "";
+ if (properties->getString(appender_key + ".max_file_size",
max_file_size_str)) {
+try {
+ max_file_size = std::stoi(max_file_size_str);
+} catch (const std::invalid_argument &) {
+} catch (const std::out_of_range &) {
+}
+ }
+
+ std::lock_guard guard(sink_map_mtx);
+ auto it = rotating_file_sinks.find(file_name);
+ if (it != rotating_file_sinks.end()) {
+return it->second;
+ }
Review comment:
this also might be relevant: https://github.com/gabime/spdlog/issues/243
> If you want multiple loggers to share the same file, they must share the
same sink. Otherwise bad things will happen as you have noticed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #1259: MINIFICPP-1748 - Modify log properties through the c2 protocol
adamdebreceni commented on a change in pull request #1259:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1259#discussion_r80678
##
File path: libminifi/src/core/logging/LoggerConfiguration.cpp
##
@@ -359,6 +323,57 @@ void LoggerConfiguration::initializeCompression(const
std::lock_guard
LoggerConfiguration::getRotatingFileSink(const std::string& appender_key, const
std::shared_ptr& properties) {
+ static std::map> rotating_file_sinks;
+ static std::mutex sink_map_mtx;
+
+ std::string file_name;
+ if (!properties->getString(appender_key + ".file_name", file_name)) {
+file_name = "minifi-app.log";
+ }
+ std::string directory;
+ if (!properties->getString(appender_key + ".directory", directory)) {
+// The below part assumes logger_properties->getHome() is existing
+// Cause minifiHome must be set at MiNiFiMain.cpp?
+directory = properties->getHome() +
utils::file::FileUtils::get_separator() + "logs";
+ }
+
+ file_name = directory + utils::file::FileUtils::get_separator() + file_name;
+ if (utils::file::FileUtils::create_dir(directory) == -1) {
+std::cerr << directory << " cannot be created\n";
+exit(1);
+ }
+
+ int max_files = 3;
+ std::string max_files_str = "";
+ if (properties->getString(appender_key + ".max_files", max_files_str)) {
+try {
+ max_files = std::stoi(max_files_str);
+} catch (const std::invalid_argument &) {
+} catch (const std::out_of_range &) {
+}
+ }
+
+ int max_file_size = 5_MiB;
+ std::string max_file_size_str = "";
+ if (properties->getString(appender_key + ".max_file_size",
max_file_size_str)) {
+try {
+ max_file_size = std::stoi(max_file_size_str);
+} catch (const std::invalid_argument &) {
+} catch (const std::out_of_range &) {
+}
+ }
+
+ std::lock_guard guard(sink_map_mtx);
+ auto it = rotating_file_sinks.find(file_name);
+ if (it != rotating_file_sinks.end()) {
+return it->second;
+ }
Review comment:
and this indeed means that some updates will not take effect until
restart, e.g. changing the rotating file count/size
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] fgerlits commented on a change in pull request #1259: MINIFICPP-1748 - Modify log properties through the c2 protocol
fgerlits commented on a change in pull request #1259:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1259#discussion_r806679388
##
File path: libminifi/src/core/logging/LoggerConfiguration.cpp
##
@@ -359,6 +323,57 @@ void LoggerConfiguration::initializeCompression(const
std::lock_guard
LoggerConfiguration::getRotatingFileSink(const std::string& appender_key, const
std::shared_ptr& properties) {
+ static std::map> rotating_file_sinks;
+ static std::mutex sink_map_mtx;
+
+ std::string file_name;
+ if (!properties->getString(appender_key + ".file_name", file_name)) {
+file_name = "minifi-app.log";
+ }
+ std::string directory;
+ if (!properties->getString(appender_key + ".directory", directory)) {
+// The below part assumes logger_properties->getHome() is existing
+// Cause minifiHome must be set at MiNiFiMain.cpp?
+directory = properties->getHome() +
utils::file::FileUtils::get_separator() + "logs";
+ }
+
+ file_name = directory + utils::file::FileUtils::get_separator() + file_name;
+ if (utils::file::FileUtils::create_dir(directory) == -1) {
+std::cerr << directory << " cannot be created\n";
+exit(1);
+ }
+
+ int max_files = 3;
+ std::string max_files_str = "";
+ if (properties->getString(appender_key + ".max_files", max_files_str)) {
+try {
+ max_files = std::stoi(max_files_str);
+} catch (const std::invalid_argument &) {
+} catch (const std::out_of_range &) {
+}
+ }
+
+ int max_file_size = 5_MiB;
+ std::string max_file_size_str = "";
+ if (properties->getString(appender_key + ".max_file_size",
max_file_size_str)) {
+try {
+ max_file_size = std::stoi(max_file_size_str);
+} catch (const std::invalid_argument &) {
+} catch (const std::out_of_range &) {
+}
+ }
+
+ std::lock_guard guard(sink_map_mtx);
+ auto it = rotating_file_sinks.find(file_name);
+ if (it != rotating_file_sinks.end()) {
+return it->second;
+ }
Review comment:
Thanks, makes sense. But I don't think the new comment is clear;
something like
```
// According to spdlog docs, if two loggers write to the same file, they
must use the same sink object.
// Note that some logging configuration changes will not take effect until
MiNiFi is restarted.
```
would be better.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #1259: MINIFICPP-1748 - Modify log properties through the c2 protocol
adamdebreceni commented on a change in pull request #1259:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1259#discussion_r806684711
##
File path: libminifi/src/core/logging/LoggerConfiguration.cpp
##
@@ -359,6 +323,57 @@ void LoggerConfiguration::initializeCompression(const
std::lock_guard
LoggerConfiguration::getRotatingFileSink(const std::string& appender_key, const
std::shared_ptr& properties) {
+ static std::map> rotating_file_sinks;
+ static std::mutex sink_map_mtx;
+
+ std::string file_name;
+ if (!properties->getString(appender_key + ".file_name", file_name)) {
+file_name = "minifi-app.log";
+ }
+ std::string directory;
+ if (!properties->getString(appender_key + ".directory", directory)) {
+// The below part assumes logger_properties->getHome() is existing
+// Cause minifiHome must be set at MiNiFiMain.cpp?
+directory = properties->getHome() +
utils::file::FileUtils::get_separator() + "logs";
+ }
+
+ file_name = directory + utils::file::FileUtils::get_separator() + file_name;
+ if (utils::file::FileUtils::create_dir(directory) == -1) {
+std::cerr << directory << " cannot be created\n";
+exit(1);
+ }
+
+ int max_files = 3;
+ std::string max_files_str = "";
+ if (properties->getString(appender_key + ".max_files", max_files_str)) {
+try {
+ max_files = std::stoi(max_files_str);
+} catch (const std::invalid_argument &) {
+} catch (const std::out_of_range &) {
+}
+ }
+
+ int max_file_size = 5_MiB;
+ std::string max_file_size_str = "";
+ if (properties->getString(appender_key + ".max_file_size",
max_file_size_str)) {
+try {
+ max_file_size = std::stoi(max_file_size_str);
+} catch (const std::invalid_argument &) {
+} catch (const std::out_of_range &) {
+}
+ }
+
+ std::lock_guard guard(sink_map_mtx);
+ auto it = rotating_file_sinks.find(file_name);
+ if (it != rotating_file_sinks.end()) {
+return it->second;
+ }
Review comment:
changed comment
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (NIFI-9572) Failed to index Provenance Events and (Too many Files)
[
https://issues.apache.org/jira/browse/NIFI-9572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17492538#comment-17492538
]
mayki commented on NIFI-9572:
-
Hello
[~joewitt] here the information, this time, NIFI failed to start
{code:java}
# grep java /appl/nifi/nifi-current/bin/nifi-env.sh
# The java implementation to use.
export JAVA_HOME=$(readlink -f /appl/pkg/jdk-current/bin/java | sed
"s:/bin/java::")
# ls -rtl /appl/pkg |grep jdk-current
lrwxrwxrwx 1 root root 13 Feb 15 10:47 jdk-current -> jdk-1.8.0.261
# grep run.as bootstrap.conf
run.as=nifi {code}
I send the nifi-app.log about restart failed[^nifi-app.log]
> Failed to index Provenance Events and (Too many Files)
> --
>
> Key: NIFI-9572
> URL: https://issues.apache.org/jira/browse/NIFI-9572
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core UI
>Affects Versions: 1.15.2
>Reporter: mayki
>Priority: Major
> Attachments: nifi-app.log
>
>
> Hello
> I have upgraded NIFI 1.15.2 since 2022/01/05
> No issue until this night 2022/01/13
> * nifi version 1.15.2
> * jdk-1.8.0_311
> And the limit is high
> {code:java}
> Last login: Fri Jan 14 09:57:06 CET 2022 on pts/2
> -bash-4.2@nifi$ ulimit -a
> core file size (blocks, -c) 0
> data seg size (kbytes, -d) unlimited
> scheduling priority (-e) 0
> file size (blocks, -f) rg
> pending signals (-i) 63278
> max locked memory (kbytes, -l) 64
> max memory size (kbytes, -m) unlimited
> open files (-n) 5
> pipe size (512 bytes, -p) 8
> POSIX message queues (bytes, -q) 819200
> real-time priority (-r) 0
> stack size (kbytes, -s) 8192
> cpu time (seconds, -t) unlimited
> max user processes (-u) 1
> virtual memory (kbytes, -v) unlimited
> file locks (-x) unlimited
> {code}
>
> We got a lot error about provenance_repository, it fill our filesystem logs ..
>
> {code:java}
> 2022-01-14 10:19:00,963 ERROR [Index Provenance Events-2]
> o.a.n.p.index.lucene.EventIndexTask Failed to index Provenance Events
> org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
> at
> org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:877)
> at
> org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:891)
> at
> org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1468)
> at
> org.apache.lucene.index.IndexWriter.addDocuments(IndexWriter.java:1444)
> at
> org.apache.nifi.provenance.lucene.LuceneEventIndexWriter.index(LuceneEventIndexWriter.java:70)
> at
> org.apache.nifi.provenance.index.lucene.EventIndexTask.index(EventIndexTask.java:202)
> at
> org.apache.nifi.provenance.index.lucene.EventIndexTask.run(EventIndexTask.java:113)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.nio.file.FileSystemException:
> /data/nifi/provenance_repository/lucene-8-index-1642145908399/_4_Lucene80_0.dvd:
> Too many open files
> {code}
>
>
> We expect upgrade all nifi instances to 1.15.2 to avoid log4j vulnerability.
> But it is impossible to do that if we got this error.
>
> Thanks for you help.
>
> Regards
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (NIFI-9572) Failed to index Provenance Events and (Too many Files)
[
https://issues.apache.org/jira/browse/NIFI-9572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
mayki updated NIFI-9572:
Attachment: nifi-app.log
> Failed to index Provenance Events and (Too many Files)
> --
>
> Key: NIFI-9572
> URL: https://issues.apache.org/jira/browse/NIFI-9572
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core UI
>Affects Versions: 1.15.2
>Reporter: mayki
>Priority: Major
> Attachments: nifi-app.log
>
>
> Hello
> I have upgraded NIFI 1.15.2 since 2022/01/05
> No issue until this night 2022/01/13
> * nifi version 1.15.2
> * jdk-1.8.0_311
> And the limit is high
> {code:java}
> Last login: Fri Jan 14 09:57:06 CET 2022 on pts/2
> -bash-4.2@nifi$ ulimit -a
> core file size (blocks, -c) 0
> data seg size (kbytes, -d) unlimited
> scheduling priority (-e) 0
> file size (blocks, -f) rg
> pending signals (-i) 63278
> max locked memory (kbytes, -l) 64
> max memory size (kbytes, -m) unlimited
> open files (-n) 5
> pipe size (512 bytes, -p) 8
> POSIX message queues (bytes, -q) 819200
> real-time priority (-r) 0
> stack size (kbytes, -s) 8192
> cpu time (seconds, -t) unlimited
> max user processes (-u) 1
> virtual memory (kbytes, -v) unlimited
> file locks (-x) unlimited
> {code}
>
> We got a lot error about provenance_repository, it fill our filesystem logs ..
>
> {code:java}
> 2022-01-14 10:19:00,963 ERROR [Index Provenance Events-2]
> o.a.n.p.index.lucene.EventIndexTask Failed to index Provenance Events
> org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
> at
> org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:877)
> at
> org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:891)
> at
> org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1468)
> at
> org.apache.lucene.index.IndexWriter.addDocuments(IndexWriter.java:1444)
> at
> org.apache.nifi.provenance.lucene.LuceneEventIndexWriter.index(LuceneEventIndexWriter.java:70)
> at
> org.apache.nifi.provenance.index.lucene.EventIndexTask.index(EventIndexTask.java:202)
> at
> org.apache.nifi.provenance.index.lucene.EventIndexTask.run(EventIndexTask.java:113)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.nio.file.FileSystemException:
> /data/nifi/provenance_repository/lucene-8-index-1642145908399/_4_Lucene80_0.dvd:
> Too many open files
> {code}
>
>
> We expect upgrade all nifi instances to 1.15.2 to avoid log4j vulnerability.
> But it is impossible to do that if we got this error.
>
> Thanks for you help.
>
> Regards
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (NIFI-9574) Failed to decrypt data from Peer
[
https://issues.apache.org/jira/browse/NIFI-9574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
davide updated NIFI-9574:
-
Attachment: version.png
activeThread.png
cores.png
heapUse.png
flow.png
> Failed to decrypt data from Peer
> -
>
> Key: NIFI-9574
> URL: https://issues.apache.org/jira/browse/NIFI-9574
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Mahieddine Cherif
>Priority: Major
> Attachments: Screenshot 2022-01-14 at 19.21.11.png, Screenshot
> 2022-01-14 at 19.21.16.png, Screenshot 2022-01-14 at 19.21.36.png, Screenshot
> 2022-01-14 at 19.21.46.png, activeThread.png, cores.png, flow.png,
> heapUse.png, version.png
>
>
> After a migration to 1.15.2 it seems like we have almost systematically this
> error on our cluster all the time
> {code:java}
> Failed to communicate with Peer
> nifi-0.nifi-headless.apache-nifi.svc.cluster.local:8443 when load balancing
> data for Connection with ID b76e7297-e8a0-3b2b-ba30-d338db411301 due to
> java.io.IOException: Failed to decrypt data from Peer
> nifi-0.nifi-headless.apache-nifi.svc.cluster.local:8443 because Peer
> unexpectedly closed connection
> {code}
> Files get stuck in the queue mentioned, it's not always the same, these are
> simple round robin queues with or without compression.
> When we recreate the cluster it goes for like some time and then it occurs
> again and again
> Is there a particular reason ?
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (NIFI-9574) Failed to decrypt data from Peer
[
https://issues.apache.org/jira/browse/NIFI-9574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17492552#comment-17492552
]
davide commented on NIFI-9574:
--
Hi [~markap14], I have the same problem like Mahieddine.
I have a 3 nodes cluster(as you can see on attached images) with a connection
with a Load Balance Strategy set to Partition by attribute, sometimes some
flowfiles get stuck in this connection (see image1). Furthermore we have this
log error in bulletin board
{{10:56:28 UTC ERROR -xxx-xx-nifi0.:8443 }}
{{Failed to receive FlowFiles for Load Balancing due to
org.apache.nifi.controller.queue.clustered.server.TransactionAbortedException:
Attempted to receive FlowFiles from Peer -xxx-xx-nifi2. for Connection
with ID 0c20343f-f63d-1ef1-8da0-6d3cb9fc670d but no Connection exists with that
ID}}
{{10:56:28 UTC ERROR -xxx-xx-nifi0.:8443}}
{{{}Failed to communicate with Peer -xxx-xx-nifi2.:8443 when load
balancing data for Connection with ID c12f3c02-9f05-1232--f8d925a9 due
to java.io.IOException: Failed to decrypt data from Peer
-xxx-xx-nifi2.:8443 because Peer unexpectedly closed
connection{}}}{{{}{}}}
The weird thing is that only some flowfiles are stucked other flowfiles goes
forward in the flows.
How can i fix it?
Any suggestion?
Thank you
!flow.png!!heapUse.png!!activeThread.png!!version.png!!cores.png!
> Failed to decrypt data from Peer
> -
>
> Key: NIFI-9574
> URL: https://issues.apache.org/jira/browse/NIFI-9574
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Mahieddine Cherif
>Priority: Major
> Attachments: Screenshot 2022-01-14 at 19.21.11.png, Screenshot
> 2022-01-14 at 19.21.16.png, Screenshot 2022-01-14 at 19.21.36.png, Screenshot
> 2022-01-14 at 19.21.46.png, activeThread.png, cores.png, flow.png,
> heapUse.png, version.png
>
>
> After a migration to 1.15.2 it seems like we have almost systematically this
> error on our cluster all the time
> {code:java}
> Failed to communicate with Peer
> nifi-0.nifi-headless.apache-nifi.svc.cluster.local:8443 when load balancing
> data for Connection with ID b76e7297-e8a0-3b2b-ba30-d338db411301 due to
> java.io.IOException: Failed to decrypt data from Peer
> nifi-0.nifi-headless.apache-nifi.svc.cluster.local:8443 because Peer
> unexpectedly closed connection
> {code}
> Files get stuck in the queue mentioned, it's not always the same, these are
> simple round robin queues with or without compression.
> When we recreate the cluster it goes for like some time and then it occurs
> again and again
> Is there a particular reason ?
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [nifi] nuttybrewer opened a new pull request #5768: NIFI-8641: AWS WebIdentity for Nifi
nuttybrewer opened a new pull request #5768:
URL: https://github.com/apache/nifi/pull/5768
Thank you for submitting a contribution to Apache NiFi.
Please provide a short description of the PR here:
Description of PR
Enables AWS WebToken Identity for K8S Containers
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [ X] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [ X] Does your PR title start with **NIFI-** where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [X ] Has your PR been rebased against the latest commit within the target
branch (typically `main`)?
- [ X] Is your initial contribution a single, squashed commit? _Additional
commits in response to PR reviewer feedback should be made on this branch and
pushed to allow change tracking. Do not `squash` or use `--force` when pushing
to allow for clean monitoring of changes._
### For code changes:
- [X ] Have you ensured that the full suite of tests is executed via `mvn
-Pcontrib-check clean install` at the root `nifi` folder?
- [ X] Have you written or updated unit tests to verify your changes?
- [ X] Have you verified that the full build is successful on JDK 8?
- [ X] Have you verified that the full build is successful on JDK 11?
- [ X] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main
`LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main
`NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI for
build issues and submit an update to your PR as soon as possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] nrnalamalpu commented on pull request #5291: NIFI-9036 - Add AWS MSK IAM support to Kafka processors
nrnalamalpu commented on pull request #5291: URL: https://github.com/apache/nifi/pull/5291#issuecomment-1040277206 Can anyone please look into this PR and see if we can close this out? thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] bejancsaba commented on a change in pull request #5755: NIFI-9538: Add C2 heartbeat capability to minifi-c2-service
bejancsaba commented on a change in pull request #5755:
URL: https://github.com/apache/nifi/pull/5755#discussion_r806855762
##
File path:
minifi/minifi-c2/minifi-c2-service/src/main/java/org/apache/nifi/minifi/c2/service/SimpleC2ProtocolService.java
##
@@ -0,0 +1,120 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.service;
+
+import org.apache.nifi.c2.protocol.api.C2Heartbeat;
+import org.apache.nifi.c2.protocol.api.C2HeartbeatResponse;
+import org.apache.nifi.c2.protocol.api.C2Operation;
+import org.apache.nifi.c2.protocol.api.C2OperationAck;
+import org.apache.nifi.c2.protocol.api.C2OperationState;
+import org.apache.nifi.c2.protocol.api.OperandType;
+import org.apache.nifi.c2.protocol.api.OperationState;
+import org.apache.nifi.c2.protocol.api.OperationType;
+import org.apache.nifi.minifi.c2.util.LogMarkerUtil;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.slf4j.Marker;
+import org.springframework.stereotype.Service;
+
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.UUID;
+
+@Service
+public class SimpleC2ProtocolService implements C2ProtocolService {
+
+private static final Logger logger =
LoggerFactory.getLogger(SimpleC2ProtocolService.class);
+
+private static final Set issuedOperationIds = new HashSet<>();
+
+public SimpleC2ProtocolService() {
+}
+
+@Override
+public void processOperationAck(final C2OperationAck operationAck, final
C2ProtocolContext context) {
+// This service assumes there is a single Operation UPDATE to pass
over the updated flow
+Marker marker = LogMarkerUtil.getMarker(operationAck);
+
+logger.debug(marker, "Received operation acknowledgement: {}; {}",
operationAck, context);
+// Remove the operator ID from the list of issued operations and log
the state
+final String operationId = operationAck.getOperationId();
+try {
+OperationState opState = OperationState.DONE;
+String details = null;
+
+/* Partial applications are rare and only happen when an operation
consists of updating multiple config
+ * items and some succeed ( we don't yet have the concept of
rollback in agents ).
+ * Fully Applied yields an operation success.
+ * Operation Not Understood and Not Applied give little details
but also will result in Operation Failure.
+ * We should explore if providing textual details. */
+final C2OperationState c2OperationState =
operationAck.getOperationState();
+if (null != c2OperationState) {
+details = c2OperationState.getDetails();
+if (c2OperationState.getState() !=
C2OperationState.OperationState.FULLY_APPLIED) {
+opState = OperationState.FAILED;
+}
+}
+
+if (!issuedOperationIds.remove(operationId)) {
+logger.warn(marker, "Operation with ID " + operationId + " has
either already been acknowledged or is unknown to this server");
+} else if (null != c2OperationState) {
+final C2OperationState.OperationState operationState =
c2OperationState.getState();
+logger.debug("Operation with ID " + operationId + "
acknowledged with a state of " + operationState.name() + "(" + opState.name() +
"), details = "
++ (details == null ? "" : details));
+}
+
+// Optionally, an acknowledgement can include some of the info
normally passed in a heartbeat.
+// If this info is present, process it as a heartbeat, so we
update our latest known state of the agent.
+if (operationAck.getAgentInfo() != null
+|| operationAck.getDeviceInfo() != null
+|| operationAck.getFlowInfo() != null) {
+final C2Heartbeat heartbeatInfo = toHeartbeat(operationAck);
+logger.trace(marker, "Operation acknowledgement contains
additional info. Processing as heartbeat: {}", heartbeatInfo);
+processHeartbeat(heartbeatInf
[GitHub] [nifi] mcgilman commented on pull request #5765: NIFI-9543: Add bring-to-front functionality to labels
mcgilman commented on pull request #5765: URL: https://github.com/apache/nifi/pull/5765#issuecomment-1040309958 Reviewing... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (MINIFICPP-1758) Log received C2 operations
[ https://issues.apache.org/jira/browse/MINIFICPP-1758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Debreceni reassigned MINIFICPP-1758: - Assignee: Adam Debreceni (was: Arpad Boda) > Log received C2 operations > -- > > Key: MINIFICPP-1758 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1758 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Arpad Boda >Assignee: Adam Debreceni >Priority: Major > > Logging received C2 operations would be helpful to debug and improve C2 > related features. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (NIFI-9572) Failed to index Provenance Events and (Too many Files)
[
https://issues.apache.org/jira/browse/NIFI-9572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17492641#comment-17492641
]
Joe Witt commented on NIFI-9572:
Hello [~mayki]
In requesting the 'java -version' output you instead appear to have sent the
output of other bash commands. Further the link implies you're on JDK 1.8
update 261. However, the beginning of the JIRA you suggest you're on update
311.
In requesting the lsof output periodically as the number of open files grow you
are instead showing that now nifi fails to start with an open file handle limit
hit during the startup process. I again suspect you do not have the proper
file limit set for the user you're actually running nifi as (which the above
info implies might be the user 'nifi'). I dont know that the ulimit results
you've shown are for the user nifi or whether they're for the root user or some
other user.
It is very difficult to be helpful when the various details don't line up
because we're not making progress. I appreciate there is a chance that there
is some bug in NIFi causing this but at this stage I dont suspect that - I
suspect environment/config setup not matching what you're intending.
We really need to see full commands/full output at this point to help progress.
I think for me to be helpful via a JIRA here I still need to see:
1. 'whoami'
2. 'java -version' including command entry and results
3. 'ulimit -a' including command entry and results
4. 'lsof -p ' run every 30 minutes while nifi is running
Thanks
Joe
> Failed to index Provenance Events and (Too many Files)
> --
>
> Key: NIFI-9572
> URL: https://issues.apache.org/jira/browse/NIFI-9572
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core UI
>Affects Versions: 1.15.2
>Reporter: mayki
>Priority: Major
> Attachments: nifi-app.log
>
>
> Hello
> I have upgraded NIFI 1.15.2 since 2022/01/05
> No issue until this night 2022/01/13
> * nifi version 1.15.2
> * jdk-1.8.0_311
> And the limit is high
> {code:java}
> Last login: Fri Jan 14 09:57:06 CET 2022 on pts/2
> -bash-4.2@nifi$ ulimit -a
> core file size (blocks, -c) 0
> data seg size (kbytes, -d) unlimited
> scheduling priority (-e) 0
> file size (blocks, -f) rg
> pending signals (-i) 63278
> max locked memory (kbytes, -l) 64
> max memory size (kbytes, -m) unlimited
> open files (-n) 5
> pipe size (512 bytes, -p) 8
> POSIX message queues (bytes, -q) 819200
> real-time priority (-r) 0
> stack size (kbytes, -s) 8192
> cpu time (seconds, -t) unlimited
> max user processes (-u) 1
> virtual memory (kbytes, -v) unlimited
> file locks (-x) unlimited
> {code}
>
> We got a lot error about provenance_repository, it fill our filesystem logs ..
>
> {code:java}
> 2022-01-14 10:19:00,963 ERROR [Index Provenance Events-2]
> o.a.n.p.index.lucene.EventIndexTask Failed to index Provenance Events
> org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
> at
> org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:877)
> at
> org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:891)
> at
> org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1468)
> at
> org.apache.lucene.index.IndexWriter.addDocuments(IndexWriter.java:1444)
> at
> org.apache.nifi.provenance.lucene.LuceneEventIndexWriter.index(LuceneEventIndexWriter.java:70)
> at
> org.apache.nifi.provenance.index.lucene.EventIndexTask.index(EventIndexTask.java:202)
> at
> org.apache.nifi.provenance.index.lucene.EventIndexTask.run(EventIndexTask.java:113)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.nio.file.FileSystemException:
> /data/nifi/provenance_repository/lucene-8-index-1642145908399/_4_Lucene80_0.dvd:
> Too many open files
> {code}
>
>
> We expect upgrade all nifi instances to 1.15.2 to avoid log4j vulnerability.
> But it is impossible to do that if we got this error.
>
> Thanks for you help.
>
> Regards
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [nifi] markap14 commented on a change in pull request #5593: NIFI-9475 Provide Framework-Level Retries for NiFi Relationships
markap14 commented on a change in pull request #5593:
URL: https://github.com/apache/nifi/pull/5593#discussion_r806896677
##
File path:
nifi-system-tests/nifi-stateless-system-test-suite/src/test/java/org/apache/nifi/stateless/VersionedFlowBuilder.java
##
@@ -144,6 +144,10 @@ public VersionedProcessor createProcessor(final Bundle
bundle, final String type
processor.setType(type);
processor.setYieldDuration("1 sec");
processor.setSchedulingStrategy("TIMER_DRIVEN");
+processor.setRetryCount(0);
+processor.setBackoffMechanism("PENALIZE_FLOWFILE");
Review comment:
Nevermind, I don't think the dependency is in scope here, and it's not
worth bringing in the dependency just for that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Created] (NIFI-9686) Unstable Build due to SetSNMPIntegrationTest
David Handermann created NIFI-9686:
--
Summary: Unstable Build due to SetSNMPIntegrationTest
Key: NIFI-9686
URL: https://issues.apache.org/jira/browse/NIFI-9686
Project: Apache NiFi
Issue Type: Bug
Components: Extensions, Tools and Build
Reporter: David Handermann
The following error output shows intermittent failures related to the
{{SetSNMPIntegrationTest}} class. The {{BindException}} indicates another
process is using the same port, which could mean that the test is not checking
for an available port, or a previous test did not close a socket correctly.
{noformat}
Error: Tests run: 3, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.099 s
<<< FAILURE! - in org.apache.nifi.snmp.processors.SetSNMPIntegrationTest
Error: SetSNMPIntegrationTest.testSnmpSet Time elapsed: 0.003 s <<< FAILURE!
org.opentest4j.MultipleFailuresError:
Multiple Failures (2 failures)
java.lang.RuntimeException: java.net.BindException: Address already in
use (Bind failed)
java.lang.NullPointerException:
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [nifi-minifi-cpp] fgerlits closed pull request #1259: MINIFICPP-1748 - Modify log properties through the c2 protocol
fgerlits closed pull request #1259: URL: https://github.com/apache/nifi-minifi-cpp/pull/1259 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] fgerlits closed pull request #1255: MINIFICPP-1750 Fix C2 clear operation with corecomponentstate operand
fgerlits closed pull request #1255: URL: https://github.com/apache/nifi-minifi-cpp/pull/1255 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #1252: MINIFICPP-1686 - Processor destructors are not called
szaszm commented on a change in pull request #1252:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1252#discussion_r798799427
##
File path: extensions/aws/processors/ListS3.h
##
@@ -33,6 +33,7 @@ namespace org {
namespace apache {
namespace nifi {
namespace minifi {
+
Review comment:
Please remove this unnecessary new line
##
File path: extensions/http-curl/tests/C2NullConfiguration.cpp
##
@@ -61,10 +61,10 @@ class VerifyC2Server : public HTTPIntegrationBase {
}
void queryRootProcessGroup(std::shared_ptr pg) override {
-std::shared_ptr proc = pg->findProcessorByName("invoke");
+auto* const proc = pg->findProcessorByName("invoke");
assert(proc != nullptr);
-std::shared_ptr inv =
std::dynamic_pointer_cast(proc);
+const auto inv = dynamic_cast(proc);
Review comment:
Use `auto* const` here as well. Consider adding const to the pointer as
well, since it's only read from in these functions.
##
File path: extensions/coap/tests/CoapC2VerifyHeartbeat.cpp
##
@@ -92,10 +92,10 @@ class VerifyCoAPServer : public CoapIntegrationBase {
}
void queryRootProcessGroup(std::shared_ptr pg) override {
-std::shared_ptr proc = pg->findProcessorByName("invoke");
+auto proc = pg->findProcessorByName("invoke");
Review comment:
Use `auto* const` here as well
##
File path: CMakeLists.txt
##
@@ -611,6 +612,11 @@ if (ENABLE_KUBERNETES)
createExtension(KUBERNETES-EXTENSIONS "KUBERNETES EXTENSIONS" "This enables
Kubernetes support" "extensions/kubernetes")
endif()
+## Add the test-processors extension
+if (ENABLE_ALL OR NOT SKIP_TESTS OR ENABLE_TEST_PROCESSORS)
+ createExtension(TEST-PROCESSORS-EXTENSION "TEST-PROCESSORS EXTENSION"
"This enables processors purely for testing purposes"
"extensions/test-processors")
+endif()
Review comment:
I would run them when `NOT SKIP_TESTS`, regardless of the other options,
since it's basically testing.
##
File path: extensions/coap/tests/CoapC2VerifyHeartbeat.cpp
##
@@ -92,10 +92,10 @@ class VerifyCoAPServer : public CoapIntegrationBase {
}
void queryRootProcessGroup(std::shared_ptr pg) override {
-std::shared_ptr proc = pg->findProcessorByName("invoke");
+auto proc = pg->findProcessorByName("invoke");
assert(proc != nullptr);
-std::shared_ptr inv =
std::dynamic_pointer_cast(proc);
+auto inv = dynamic_cast(proc);
Review comment:
Use `auto* const` here as well. Consider adding const to the pointer as
well, since it's only read from in these functions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (MINIFICPP-1750) Investigate C2 clear operation with corecomponentstate operand
[
https://issues.apache.org/jira/browse/MINIFICPP-1750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gábor Gyimesi updated MINIFICPP-1750:
-
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Investigate C2 clear operation with corecomponentstate operand
> ---
>
> Key: MINIFICPP-1750
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1750
> Project: Apache NiFi MiNiFi C++
> Issue Type: Bug
>Reporter: Gábor Gyimesi
>Assignee: Gábor Gyimesi
>Priority: Minor
> Time Spent: 40m
> Remaining Estimate: 0h
>
> This operation is probably not working, because the component argument is
> always the same "corecomponentstate" which must be a mistake. Also this
> operation is untested.
> C2Agent.cpp:349
>
> {code:java}
> } else if (resp.name == "corecomponentstate") {
> // TODO(bakaid): untested
> std::vector> components =
> update_sink_->getComponents(resp.name); {code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [nifi-minifi-cpp] szaszm commented on pull request #1261: MINIFICPP-1752 Remove the catch2 dependency from our the test infrast…
szaszm commented on pull request #1261:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1261#issuecomment-1040382220
During a pre-merge check, I got this compilation error:
```
/home/szaszm/nifi-minifi-cpp-3/extensions/expression-language/tests/ProcessContextExprTests.cpp:54:10:
error: expected constructor, destructor, or type conversion before ‘(’ token
54 | TEST_CASE("ProcessContextExpr can update existing processor
properties", "[setProperty][getProperty]") {
| ^
make[2]: ***
[extensions/expression-language/tests/CMakeFiles/ProcessContextExprTests.dir/build.make:76:
extensions/expression-language/tests/CMakeFiles/ProcessContextExprTests.dir/ProcessContextExprTests.cpp.o]
Error 1
make[1]: *** [CMakeFiles/Makefile2:9470:
extensions/expression-language/tests/CMakeFiles/ProcessContextExprTests.dir/all]
Error 2
make[1]: *** Waiting for unfinished jobs
```
Could you fix it? We just need another Catch.h include in
ProcessContextExprTests.cpp, but I don't want to change files in a merge commit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Assigned] (NIFI-9686) Unstable Build due to SetSNMPIntegrationTest
[
https://issues.apache.org/jira/browse/NIFI-9686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lehel Boér reassigned NIFI-9686:
Assignee: Lehel Boér
> Unstable Build due to SetSNMPIntegrationTest
>
>
> Key: NIFI-9686
> URL: https://issues.apache.org/jira/browse/NIFI-9686
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions, Tools and Build
>Reporter: David Handermann
>Assignee: Lehel Boér
>Priority: Critical
>
> The following error output shows intermittent failures related to the
> {{SetSNMPIntegrationTest}} class. The {{BindException}} indicates another
> process is using the same port, which could mean that the test is not
> checking for an available port, or a previous test did not close a socket
> correctly.
> {noformat}
> Error: Tests run: 3, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.099
> s <<< FAILURE! - in org.apache.nifi.snmp.processors.SetSNMPIntegrationTest
> Error: SetSNMPIntegrationTest.testSnmpSet Time elapsed: 0.003 s <<<
> FAILURE!
> org.opentest4j.MultipleFailuresError:
> Multiple Failures (2 failures)
> java.lang.RuntimeException: java.net.BindException: Address already in
> use (Bind failed)
> java.lang.NullPointerException:
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [nifi] Lehel44 opened a new pull request #5769: NIFI-9686: Renamed SNMP integration tests correctly
Lehel44 opened a new pull request #5769:
URL: https://github.com/apache/nifi/pull/5769
Thank you for submitting a contribution to Apache NiFi.
Please provide a short description of the PR here:
Description of PR
https://issues.apache.org/jira/browse/NIFI-9686
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [ ] Does your PR title start with **NIFI-** where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [ ] Has your PR been rebased against the latest commit within the target
branch (typically `main`)?
- [ ] Is your initial contribution a single, squashed commit? _Additional
commits in response to PR reviewer feedback should be made on this branch and
pushed to allow change tracking. Do not `squash` or use `--force` when pushing
to allow for clean monitoring of changes._
### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via `mvn
-Pcontrib-check clean install` at the root `nifi` folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] Have you verified that the full build is successful on JDK 8?
- [ ] Have you verified that the full build is successful on JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main
`LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main
`NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI for
build issues and submit an update to your PR as soon as possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (NIFI-9686) Unstable Build due to SetSNMPIntegrationTest
[
https://issues.apache.org/jira/browse/NIFI-9686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lehel Boér updated NIFI-9686:
-
Status: Patch Available (was: In Progress)
> Unstable Build due to SetSNMPIntegrationTest
>
>
> Key: NIFI-9686
> URL: https://issues.apache.org/jira/browse/NIFI-9686
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions, Tools and Build
>Reporter: David Handermann
>Assignee: Lehel Boér
>Priority: Critical
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The following error output shows intermittent failures related to the
> {{SetSNMPIntegrationTest}} class. The {{BindException}} indicates another
> process is using the same port, which could mean that the test is not
> checking for an available port, or a previous test did not close a socket
> correctly.
> {noformat}
> Error: Tests run: 3, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.099
> s <<< FAILURE! - in org.apache.nifi.snmp.processors.SetSNMPIntegrationTest
> Error: SetSNMPIntegrationTest.testSnmpSet Time elapsed: 0.003 s <<<
> FAILURE!
> org.opentest4j.MultipleFailuresError:
> Multiple Failures (2 failures)
> java.lang.RuntimeException: java.net.BindException: Address already in
> use (Bind failed)
> java.lang.NullPointerException:
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [nifi-minifi-cpp] adam-markovics commented on a change in pull request #1252: MINIFICPP-1686 - Processor destructors are not called
adam-markovics commented on a change in pull request #1252:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1252#discussion_r806957651
##
File path: extensions/aws/processors/ListS3.h
##
@@ -33,6 +33,7 @@ namespace org {
namespace apache {
namespace nifi {
namespace minifi {
+
Review comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] adam-markovics commented on a change in pull request #1252: MINIFICPP-1686 - Processor destructors are not called
adam-markovics commented on a change in pull request #1252:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1252#discussion_r806966150
##
File path: CMakeLists.txt
##
@@ -611,6 +612,11 @@ if (ENABLE_KUBERNETES)
createExtension(KUBERNETES-EXTENSIONS "KUBERNETES EXTENSIONS" "This enables
Kubernetes support" "extensions/kubernetes")
endif()
+## Add the test-processors extension
+if (ENABLE_ALL OR NOT SKIP_TESTS OR ENABLE_TEST_PROCESSORS)
+ createExtension(TEST-PROCESSORS-EXTENSION "TEST-PROCESSORS EXTENSION"
"This enables processors purely for testing purposes"
"extensions/test-processors")
+endif()
Review comment:
So you just want the condition to be `if (NOT SKIP_TESTS)` ?
`ENABLE_TEST_PROCESSORS` was introduced to avoid building all unit tests in
docker integration tests, where `SKIP_TESTS` is true by default. But these
processors are still needed for integration tests.
##
File path: extensions/coap/tests/CoapC2VerifyHeartbeat.cpp
##
@@ -92,10 +92,10 @@ class VerifyCoAPServer : public CoapIntegrationBase {
}
void queryRootProcessGroup(std::shared_ptr pg) override {
-std::shared_ptr proc = pg->findProcessorByName("invoke");
+auto proc = pg->findProcessorByName("invoke");
Review comment:
Done.
##
File path: extensions/coap/tests/CoapC2VerifyHeartbeat.cpp
##
@@ -92,10 +92,10 @@ class VerifyCoAPServer : public CoapIntegrationBase {
}
void queryRootProcessGroup(std::shared_ptr pg) override {
-std::shared_ptr proc = pg->findProcessorByName("invoke");
+auto proc = pg->findProcessorByName("invoke");
assert(proc != nullptr);
-std::shared_ptr inv =
std::dynamic_pointer_cast(proc);
+auto inv = dynamic_cast(proc);
Review comment:
Done.
##
File path: extensions/http-curl/tests/C2NullConfiguration.cpp
##
@@ -61,10 +61,10 @@ class VerifyC2Server : public HTTPIntegrationBase {
}
void queryRootProcessGroup(std::shared_ptr pg) override {
-std::shared_ptr proc = pg->findProcessorByName("invoke");
+auto* const proc = pg->findProcessorByName("invoke");
assert(proc != nullptr);
-std::shared_ptr inv =
std::dynamic_pointer_cast(proc);
+const auto inv = dynamic_cast(proc);
Review comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #1252: MINIFICPP-1686 - Processor destructors are not called
szaszm commented on a change in pull request #1252: URL: https://github.com/apache/nifi-minifi-cpp/pull/1252#discussion_r806978749 ## File path: CMakeLists.txt ## @@ -611,6 +612,11 @@ if (ENABLE_KUBERNETES) createExtension(KUBERNETES-EXTENSIONS "KUBERNETES EXTENSIONS" "This enables Kubernetes support" "extensions/kubernetes") endif() +## Add the test-processors extension +if (ENABLE_ALL OR NOT SKIP_TESTS OR ENABLE_TEST_PROCESSORS) + createExtension(TEST-PROCESSORS-EXTENSION "TEST-PROCESSORS EXTENSION" "This enables processors purely for testing purposes" "extensions/test-processors") +endif() Review comment: Ok, this makes sense, in this case could you please add a comment about this relationship? And consider removing ENABLE_ALL, since these are not end-user feature processors. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] markobean commented on pull request #5743: NIFI-8899 - Add NiFi Registry version information to the Registry under an "about" button.
markobean commented on pull request #5743: URL: https://github.com/apache/nifi/pull/5743#issuecomment-1040434810 I installed Registry to see how the UI reports the version. This is a good start. I'd like to recommend the UI dialog mimic the "About" dialog for Apache NiFi for a consistent look across related applications. For example, "About NiFi Registry" placed on a colored background, inclusion of NiFi Registry logo, and a short description of Registry. Perhaps the description can be that of the Registry website: "Registry—a subproject of Apache NiFi—is a complementary application that provides a central location for storage and management of shared resources across one or more instances of NiFi and/or MiNiFi." -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm closed pull request #1261: MINIFICPP-1752 Remove the catch2 dependency from our the test infrast…
szaszm closed pull request #1261: URL: https://github.com/apache/nifi-minifi-cpp/pull/1261 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (MINIFICPP-1752) Remove the catch2 dependency from our the test infrastructure
[ https://issues.apache.org/jira/browse/MINIFICPP-1752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz resolved MINIFICPP-1752. - Resolution: Fixed > Remove the catch2 dependency from our the test infrastructure > - > > Key: MINIFICPP-1752 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1752 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Martin Zink >Assignee: Martin Zink >Priority: Major > Time Spent: 1.5h > Remaining Estimate: 0h > > There is a direct dependency in TestBase.h and SingleInputTestController.h to > Catch2 which prevents any other testframework to use them. > We might want to use another framework if it simplifies the testing. > e.g.: MINIFICPP-1735 needs gtest because the GCS mocks are implemented in > gmock. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [nifi] thenatog commented on pull request #5701: NIFI-9607: Honor Update Keys when Quoting Identifiers in PutDatabaseRecord
thenatog commented on pull request #5701: URL: https://github.com/apache/nifi/pull/5701#issuecomment-1040463251 Will review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-9687) Add additional documentation for setting nifi.cluster.node.protocol.max.threads property
Andrew M. Lim created NIFI-9687: --- Summary: Add additional documentation for setting nifi.cluster.node.protocol.max.threads property Key: NIFI-9687 URL: https://issues.apache.org/jira/browse/NIFI-9687 Project: Apache NiFi Issue Type: Improvement Components: Documentation & Website Reporter: Andrew M. Lim Current documentation in Admin Guide does not help the user set a value for this property. Will add: When a request is made to one node, it must be forwarded to the coordinator. The coordinator then replicates it to all nodes. There could be up to n+2 threads for a given request, where n = number of nodes in your cluster. As an example, if 4 requests are made, a 5 node cluster will use 4 * 7 = 28 threads. to these existing sections: https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#basic-cluster-setup https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#cluster_node_properties -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (NIFI-9687) Add additional documentation for setting nifi.cluster.node.protocol.max.threads property
[ https://issues.apache.org/jira/browse/NIFI-9687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew M. Lim reassigned NIFI-9687: --- Assignee: Andrew M. Lim > Add additional documentation for setting > nifi.cluster.node.protocol.max.threads property > > > Key: NIFI-9687 > URL: https://issues.apache.org/jira/browse/NIFI-9687 > Project: Apache NiFi > Issue Type: Improvement > Components: Documentation & Website >Reporter: Andrew M. Lim >Assignee: Andrew M. Lim >Priority: Major > > Current documentation in Admin Guide does not help the user set a value for > this property. Will add: > When a request is made to one node, it must be forwarded to the coordinator. > The coordinator then replicates it to all nodes. There could be up to n+2 > threads for a given request, where n = number of nodes in your cluster. As an > example, if 4 requests are made, a 5 node cluster will use 4 * 7 = 28 threads. > to these existing sections: > https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#basic-cluster-setup > https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#cluster_node_properties -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (NIFI-6740) No way to configure bootstrap or nifi command/control ports
[
https://issues.apache.org/jira/browse/NIFI-6740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Burgess updated NIFI-6740:
---
Status: Patch Available (was: Open)
> No way to configure bootstrap or nifi command/control ports
> ---
>
> Key: NIFI-6740
> URL: https://issues.apache.org/jira/browse/NIFI-6740
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Steve Lawrence
>Priority: Major
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> Currently, when Bootstrap starts it binds to a random ephemeral port on
> localhost in {{NifiListener.java}}:
> {code:java}
> serverSocket = new ServerSocket();
> serverSocket.bind(new InetSocketAddress("localhost", 0));
> {code}
> And then it passes this port to NiFi via the
> {{-Dnifi.bootstrap.listener.port}} argument.
> Subsequently, NiFi will also bind to a random ephemeral port in
> {{BootstrapListener.java}}, and then pass that port to Boostrap via the port
> that was passed in.
> These two ports provide a method of communication for command/control between
> the Bootstrap and Nifi processes.
> Unfortunately, because these ports are ephemeral and change on every start
> up, it makes it difficult to have predfined security policies (e.g.
> iptables/firewall) in place for these ports.
> To resolve this, one solution is to add new configuration options to
> bootstrap.conf/nifi.properties, which would allow administrators to define
> values for these ports. This ensures that everytime NiFi starts, the
> command/control ports will remain constant allowing for pre-defined security
> policies to be created. For backwards compatbility, the options can default
> to zero to maintain the ephemeral port behavior.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (NIFI-6740) No way to configure bootstrap or nifi command/control ports
[
https://issues.apache.org/jira/browse/NIFI-6740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17492727#comment-17492727
]
ASF subversion and git services commented on NIFI-6740:
---
Commit 4815df29db8b32b6992622b77ff1c7f7e9001989 in nifi's branch
refs/heads/main from Steve Lawrence
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=4815df2 ]
NIFI-6740: Add configuration options to specify NiFi/Bootstrap communication
ports
The NiFi and NiFi Bootstrap processes both bind to random ephemeral
ports to allow for inter-process communication (e.g. shutdown, port,
ping, etc.). However, the randomness of these ephemeral ports can pose
challenges for some security policies and firewall rules.
This adds two configuration options, nifi.bootstrap.listen.port and
nifi.listener.bootstrap.port, that allow an administrator to define
which ports the two processes should bind to for this communication,
making it easier to define security policies. The options default to
zero to maintain the current ephemeral port behavior.
NIFI-6740: Add configuration options to specify NiFi/Bootstrap communication
ports
Signed-off-by: Matthew Burgess
This closes #5746
> No way to configure bootstrap or nifi command/control ports
> ---
>
> Key: NIFI-6740
> URL: https://issues.apache.org/jira/browse/NIFI-6740
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Steve Lawrence
>Priority: Major
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> Currently, when Bootstrap starts it binds to a random ephemeral port on
> localhost in {{NifiListener.java}}:
> {code:java}
> serverSocket = new ServerSocket();
> serverSocket.bind(new InetSocketAddress("localhost", 0));
> {code}
> And then it passes this port to NiFi via the
> {{-Dnifi.bootstrap.listener.port}} argument.
> Subsequently, NiFi will also bind to a random ephemeral port in
> {{BootstrapListener.java}}, and then pass that port to Boostrap via the port
> that was passed in.
> These two ports provide a method of communication for command/control between
> the Bootstrap and Nifi processes.
> Unfortunately, because these ports are ephemeral and change on every start
> up, it makes it difficult to have predfined security policies (e.g.
> iptables/firewall) in place for these ports.
> To resolve this, one solution is to add new configuration options to
> bootstrap.conf/nifi.properties, which would allow administrators to define
> values for these ports. This ensures that everytime NiFi starts, the
> command/control ports will remain constant allowing for pre-defined security
> policies to be created. For backwards compatbility, the options can default
> to zero to maintain the ephemeral port behavior.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [nifi] mattyb149 closed pull request #5746: NIFI-6740: Add configuration options to specify NiFi/Bootstrap communication ports
mattyb149 closed pull request #5746: URL: https://github.com/apache/nifi/pull/5746 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] mattyb149 commented on pull request #5746: NIFI-6740: Add configuration options to specify NiFi/Bootstrap communication ports
mattyb149 commented on pull request #5746: URL: https://github.com/apache/nifi/pull/5746#issuecomment-1040508165 +1 LGTM, verified the expected behavior. Thanks for the improvement! Merged to main -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-6740) No way to configure bootstrap or nifi command/control ports
[
https://issues.apache.org/jira/browse/NIFI-6740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Burgess updated NIFI-6740:
---
Fix Version/s: 1.16.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
> No way to configure bootstrap or nifi command/control ports
> ---
>
> Key: NIFI-6740
> URL: https://issues.apache.org/jira/browse/NIFI-6740
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Steve Lawrence
>Priority: Major
> Fix For: 1.16.0
>
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> Currently, when Bootstrap starts it binds to a random ephemeral port on
> localhost in {{NifiListener.java}}:
> {code:java}
> serverSocket = new ServerSocket();
> serverSocket.bind(new InetSocketAddress("localhost", 0));
> {code}
> And then it passes this port to NiFi via the
> {{-Dnifi.bootstrap.listener.port}} argument.
> Subsequently, NiFi will also bind to a random ephemeral port in
> {{BootstrapListener.java}}, and then pass that port to Boostrap via the port
> that was passed in.
> These two ports provide a method of communication for command/control between
> the Bootstrap and Nifi processes.
> Unfortunately, because these ports are ephemeral and change on every start
> up, it makes it difficult to have predfined security policies (e.g.
> iptables/firewall) in place for these ports.
> To resolve this, one solution is to add new configuration options to
> bootstrap.conf/nifi.properties, which would allow administrators to define
> values for these ports. This ensures that everytime NiFi starts, the
> command/control ports will remain constant allowing for pre-defined security
> policies to be created. For backwards compatbility, the options can default
> to zero to maintain the ephemeral port behavior.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Created] (NIFI-9688) Log Messages on Process Shutdown are Discarded due to Logging Deinitialize
Paul Grey created NIFI-9688: --- Summary: Log Messages on Process Shutdown are Discarded due to Logging Deinitialize Key: NIFI-9688 URL: https://issues.apache.org/jira/browse/NIFI-9688 Project: Apache NiFi Issue Type: Improvement Reporter: Paul Grey NiFi uses the SLF4J logging API [https://www.slf4j.org/] and the Logback implementation [https://logback.qos.ch/] to capture useful information about the running state of the application. While this subsystem seems to work well during the normal lifetime of the process, log messages associated with process shutdown activity seem to be lost; they are not written to the configured output files. As logging is one way to ensure that the all NiFi process components stop cleanly, it would be an improvement find ways to address this issue. NiFi seems to use portions of the Spring framework to control component lifecycle, including the configuration found in this web descriptor: - https://github.com/apache/nifi/blob/main/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/webapp/WEB-INF/web.xml The Logback implementation library [https://logback.qos.ch/] seems to be integrated with this framework; it is initialized and active during the lifetime of the embedded NiFi components, but at shutdown, this context is destroyed. Here are a couple of posts describing a similar problem, and suggesting a possible remediation: - https://stackoverflow.com/questions/33844659/no-log-output-on-contextdestroyed-using-servletcontextlistener-slf4j - https://stackoverflow.com/questions/37557754/logging-from-servlet-context-destroyed-event -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (NIFI-9688) Log Messages on Process Shutdown are Discarded due to Logging Deinitialize
[ https://issues.apache.org/jira/browse/NIFI-9688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Grey updated NIFI-9688: Description: NiFi uses the SLF4J logging API [https://www.slf4j.org/] and the Logback implementation [https://logback.qos.ch/] to capture useful information about the running state of the application. While this subsystem seems to work well during the normal lifetime of the process, log messages associated with process shutdown activity seem to be lost; they are not written to the configured output files. As logging is one way to ensure that the all NiFi process components stop cleanly, it would be an improvement find ways to address this issue. NiFi seems to use portions of the Spring framework to control component lifecycle, including the configuration found in this web descriptor: - [https://github.com/apache/nifi/blob/main/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/webapp/WEB-INF/web.xml] Logback seems to be integrated with this framework; it is initialized and active during the lifetime of the embedded NiFi components, but at shutdown, this context is destroyed. Here are a couple of posts describing a similar problem, and suggesting a possible remediation: - https://stackoverflow.com/questions/33844659/no-log-output-on-contextdestroyed-using-servletcontextlistener-slf4j - https://stackoverflow.com/questions/37557754/logging-from-servlet-context-destroyed-event was: NiFi uses the SLF4J logging API [https://www.slf4j.org/] and the Logback implementation [https://logback.qos.ch/] to capture useful information about the running state of the application. While this subsystem seems to work well during the normal lifetime of the process, log messages associated with process shutdown activity seem to be lost; they are not written to the configured output files. As logging is one way to ensure that the all NiFi process components stop cleanly, it would be an improvement find ways to address this issue. NiFi seems to use portions of the Spring framework to control component lifecycle, including the configuration found in this web descriptor: - https://github.com/apache/nifi/blob/main/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/webapp/WEB-INF/web.xml The Logback implementation library [https://logback.qos.ch/] seems to be integrated with this framework; it is initialized and active during the lifetime of the embedded NiFi components, but at shutdown, this context is destroyed. Here are a couple of posts describing a similar problem, and suggesting a possible remediation: - https://stackoverflow.com/questions/33844659/no-log-output-on-contextdestroyed-using-servletcontextlistener-slf4j - https://stackoverflow.com/questions/37557754/logging-from-servlet-context-destroyed-event > Log Messages on Process Shutdown are Discarded due to Logging Deinitialize > -- > > Key: NIFI-9688 > URL: https://issues.apache.org/jira/browse/NIFI-9688 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Paul Grey >Priority: Major > > NiFi uses the SLF4J logging API [https://www.slf4j.org/] and the Logback > implementation [https://logback.qos.ch/] to capture useful information about > the running state of the application. While this subsystem seems to work > well during the normal lifetime of the process, log messages associated with > process shutdown activity seem to be lost; they are not written to the > configured output files. As logging is one way to ensure that the all NiFi > process components stop cleanly, it would be an improvement find ways to > address this issue. > NiFi seems to use portions of the Spring framework to control component > lifecycle, including the configuration found in this web descriptor: > - > [https://github.com/apache/nifi/blob/main/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/webapp/WEB-INF/web.xml] > Logback seems to be integrated with this framework; it is initialized and > active during the lifetime of the embedded NiFi components, but at shutdown, > this context is destroyed. > Here are a couple of posts describing a similar problem, and suggesting a > possible remediation: > - > https://stackoverflow.com/questions/33844659/no-log-output-on-contextdestroyed-using-servletcontextlistener-slf4j > - > https://stackoverflow.com/questions/37557754/logging-from-servlet-context-destroyed-event -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [nifi] asfgit closed pull request #5769: NIFI-9686: Renamed SNMP integration tests correctly
asfgit closed pull request #5769: URL: https://github.com/apache/nifi/pull/5769 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-9686) Unstable Build due to SetSNMPIntegrationTest
[
https://issues.apache.org/jira/browse/NIFI-9686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann updated NIFI-9686:
---
Fix Version/s: 1.16.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Unstable Build due to SetSNMPIntegrationTest
>
>
> Key: NIFI-9686
> URL: https://issues.apache.org/jira/browse/NIFI-9686
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions, Tools and Build
>Reporter: David Handermann
>Assignee: Lehel Boér
>Priority: Critical
> Fix For: 1.16.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The following error output shows intermittent failures related to the
> {{SetSNMPIntegrationTest}} class. The {{BindException}} indicates another
> process is using the same port, which could mean that the test is not
> checking for an available port, or a previous test did not close a socket
> correctly.
> {noformat}
> Error: Tests run: 3, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.099
> s <<< FAILURE! - in org.apache.nifi.snmp.processors.SetSNMPIntegrationTest
> Error: SetSNMPIntegrationTest.testSnmpSet Time elapsed: 0.003 s <<<
> FAILURE!
> org.opentest4j.MultipleFailuresError:
> Multiple Failures (2 failures)
> java.lang.RuntimeException: java.net.BindException: Address already in
> use (Bind failed)
> java.lang.NullPointerException:
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (NIFI-9686) Unstable Build due to SetSNMPIntegrationTest
[
https://issues.apache.org/jira/browse/NIFI-9686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17492735#comment-17492735
]
ASF subversion and git services commented on NIFI-9686:
---
Commit 51202d7c2d95fdb4f5d80621efe8e2d8b269bd94 in nifi's branch
refs/heads/main from Lehel
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=51202d7 ]
NIFI-9686: Renamed SNMP integration tests correctly
This closes #5769
Signed-off-by: David Handermann
> Unstable Build due to SetSNMPIntegrationTest
>
>
> Key: NIFI-9686
> URL: https://issues.apache.org/jira/browse/NIFI-9686
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions, Tools and Build
>Reporter: David Handermann
>Assignee: Lehel Boér
>Priority: Critical
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The following error output shows intermittent failures related to the
> {{SetSNMPIntegrationTest}} class. The {{BindException}} indicates another
> process is using the same port, which could mean that the test is not
> checking for an available port, or a previous test did not close a socket
> correctly.
> {noformat}
> Error: Tests run: 3, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.099
> s <<< FAILURE! - in org.apache.nifi.snmp.processors.SetSNMPIntegrationTest
> Error: SetSNMPIntegrationTest.testSnmpSet Time elapsed: 0.003 s <<<
> FAILURE!
> org.opentest4j.MultipleFailuresError:
> Multiple Failures (2 failures)
> java.lang.RuntimeException: java.net.BindException: Address already in
> use (Bind failed)
> java.lang.NullPointerException:
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [nifi] andrewmlim opened a new pull request #5770: NIFI-9687 Add additional documentation for nifi.cluster.node.protocol…
andrewmlim opened a new pull request #5770: URL: https://github.com/apache/nifi/pull/5770 ….max.threads property to Admin Guide -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-9680) Add "quick copy to clipboard" feature to component dialogs
[ https://issues.apache.org/jira/browse/NIFI-9680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17492768#comment-17492768 ] Kevin Doran commented on NIFI-9680: --- What if we added a component-level button/menu to collect this configuration as text, and provide a choice (menu list or pop-up dialog box) for just the component or the parent PG? > Add "quick copy to clipboard" feature to component dialogs > -- > > Key: NIFI-9680 > URL: https://issues.apache.org/jira/browse/NIFI-9680 > Project: Apache NiFi > Issue Type: New Feature > Components: Core UI >Reporter: Matt Burgess >Priority: Major > > The idea for this Jira is to be able to copy the property information from a > component dialog as text to the clipboard, to have something like what you > would if you took a screenshot. Dealing with text is much easier. Sensitive > property values would be omitted (possibly being output as "Sensitive value > set") and references to other components could be by name and possibly type > and ID. Information from all tabs would be included to save the trouble of > multiple screenshots/copies for a single component. > For more detailed information, the flow definition for the parent Process > Group can be exported to JSON as exists today. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [nifi] markap14 merged pull request #5749: NIFI-9233 - Improve reliability of system integration tests
markap14 merged pull request #5749: URL: https://github.com/apache/nifi/pull/5749 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-9233) Improve reliability of system integration tests
[
https://issues.apache.org/jira/browse/NIFI-9233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17492770#comment-17492770
]
ASF subversion and git services commented on NIFI-9233:
---
Commit 87cfd43f6f872e14e904d63fe74645d41aaa8a02 in nifi's branch
refs/heads/main from greyp9
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=87cfd43 ]
NIFI-9233 - Improve reliability of system integration tests (#5749)
* NIFI-9233 - Improve reliability of system integration tests
> Improve reliability of system integration tests
> ---
>
> Key: NIFI-9233
> URL: https://issues.apache.org/jira/browse/NIFI-9233
> Project: Apache NiFi
> Issue Type: Task
> Components: Tools and Build
>Reporter: David Handermann
>Assignee: Paul Grey
>Priority: Major
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> The {{nifi-system-tests}} module contains multiple integration tests for
> verifying behavior using minimal instances of NiFi. The GitHub
> [system-tests|https://github.com/apache/nifi/actions/workflows/system-tests.yml]
> workflow fails on some occasions due to apparent timing issues when making
> requests or evaluating expected results. The polling approach and instance
> configuration properties should be evaluated for potential improvements to
> make the tests more reliable in environments with limited resources.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #1152: MINIFICPP-1593 ProcFsMonitor to monitor /proc pseduo filesystem
szaszm commented on a change in pull request #1152:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1152#discussion_r807159061
##
File path: extensions/procfs/DiskStat.h
##
@@ -0,0 +1,152 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#pragma once
+
+#include
+#include
+#include
+#include
+#include "rapidjson/document.h"
+
+namespace org::apache::nifi::minifi::procfs {
+
+class DiskStatData {
+ DiskStatData() = default;
+
+ public:
+ struct MonotonicIncreasingMembers {
+uint64_t reads_completed_;
+uint64_t reads_merged_;
+uint64_t sectors_read_;
+uint64_t milliseconds_spent_reading_;
+uint64_t writes_completed_;
+uint64_t writes_merges_;
+uint64_t sectors_written_;
+uint64_t milliseconds_spent_writing_;
+uint64_t milliseconds_spent_io_;
+uint64_t weighted_milliseconds_spent_io_;
+
+bool operator>=(const MonotonicIncreasingMembers& rhs) const {
+ return reads_completed_ >= rhs.reads_completed_
+ && reads_merged_ >= rhs.reads_merged_
+ && sectors_read_ >= rhs.sectors_read_
+ && milliseconds_spent_reading_ >= rhs.milliseconds_spent_reading_
+ && writes_completed_ >= rhs.writes_completed_
+ && writes_merges_ >= rhs.writes_merges_
+ && sectors_written_ >= rhs.sectors_written_
+ && milliseconds_spent_writing_ >= rhs.milliseconds_spent_writing_
+ && milliseconds_spent_io_ >= rhs.milliseconds_spent_io_
+ && weighted_milliseconds_spent_io_ >=
rhs.weighted_milliseconds_spent_io_;
+}
+ };
+ static std::optional parseDiskStatLine(std::istringstream&
iss) {
+DiskStatData disk_stat_data;
+iss >> disk_stat_data.major_device_number_ >>
disk_stat_data.minor_device_number_ >> disk_stat_data.disk_name >>
disk_stat_data.monotonic_increasing_members_.reads_completed_
+>> disk_stat_data.monotonic_increasing_members_.reads_merged_ >>
disk_stat_data.monotonic_increasing_members_.sectors_read_
+>>
disk_stat_data.monotonic_increasing_members_.milliseconds_spent_reading_ >>
disk_stat_data.monotonic_increasing_members_.writes_completed_
+>> disk_stat_data.monotonic_increasing_members_.writes_merges_ >>
disk_stat_data.monotonic_increasing_members_.sectors_written_
+>>
disk_stat_data.monotonic_increasing_members_.milliseconds_spent_reading_ >>
disk_stat_data.monotonic_increasing_members_.writes_completed_
+>> disk_stat_data.monotonic_increasing_members_.writes_merges_ >>
disk_stat_data.monotonic_increasing_members_.sectors_written_
+>>
disk_stat_data.monotonic_increasing_members_.milliseconds_spent_writing_ >>
disk_stat_data.ios_in_progress_ >> disk_stat_data.ios_in_progress_
+>> disk_stat_data.monotonic_increasing_members_.milliseconds_spent_io_
>> disk_stat_data.monotonic_increasing_members_.weighted_milliseconds_spent_io_;
+
+if (iss.fail())
+ return std::nullopt;
+return disk_stat_data;
+ }
+
+ uint64_t getMajorDeviceNumber() const { return major_device_number_; }
+ uint64_t getMinorDeviceNumber() const { return minor_device_number_; }
+ const std::string& getDiskName() const { return disk_name; }
+ uint64_t getReadsCompleted() const { return
monotonic_increasing_members_.reads_completed_; }
+ uint64_t getReadsMerged() const { return
monotonic_increasing_members_.reads_merged_; }
+ uint64_t getSectorsRead() const { return
monotonic_increasing_members_.sectors_read_; }
+ uint64_t getMillisecondsSpentReading() const { return
monotonic_increasing_members_.milliseconds_spent_reading_; }
+ uint64_t getWritesCompleted() const { return
monotonic_increasing_members_.writes_completed_; }
+ uint64_t getWritesMerged() const { return
monotonic_increasing_members_.writes_merges_; }
+ uint64_t getSectorsWritten() const { return
monotonic_increasing_members_.sectors_written_; }
+ uint64_t getMillisecondsSpentWriting() const { return
monotonic_increasing_members_.milliseconds_spent_writing_; }
+ uint64_t getIosInProgress() const { return ios_in_progress_; }
+ uint64_t getMillisecondsSpentIo() const { return
monotonic_increasing_members_.milliseconds_spent_io_; }
+ uint64_t getWeigh
[GitHub] [nifi] stevedlawrence commented on a change in pull request #5746: NIFI-6740: Add configuration options to specify NiFi/Bootstrap communication ports
stevedlawrence commented on a change in pull request #5746:
URL: https://github.com/apache/nifi/pull/5746#discussion_r806036801
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-resources/src/main/resources/conf/nifi.properties
##
@@ -244,6 +244,9 @@
nifi.security.user.saml.http.client.read.timeout=${nifi.security.user.saml.http.
# nifi.security.group.mapping.value.anygroup=$1
# nifi.security.group.mapping.transform.anygroup=LOWER
+# listener bootstrap properties
+nifi.listener.bootstrap.port=${nifi.listener.bootstrap.port}
Review comment:
Yeah, the names are a bit confusing, I'm open to other suggestions that
might be more clear.
I've added a comment as suggested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #1152: MINIFICPP-1593 ProcFsMonitor to monitor /proc pseduo filesystem
szaszm commented on a change in pull request #1152:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1152#discussion_r732604183
##
File path: extensions/procfs/CPPLINT.cfg
##
@@ -0,0 +1 @@
+filter=-runtime/int
Review comment:
Is this still relevant? I didn't see any `long` in the code and this
checks for `short`, `long`, and `long long` AFAIK.
##
File path: extensions/procfs/CpuStat.cpp
##
@@ -0,0 +1,116 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "CpuStat.h"
+
+namespace org::apache::nifi::minifi::procfs {
+
+std::optional CpuStatData::parseCpuStatLine(std::istringstream&
iss) {
Review comment:
This only depends on the `std::istream` interface, consider relaxing the
requirements by accepting `std::istream&`.
##
File path: extensions/procfs/CpuStat.h
##
@@ -0,0 +1,140 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#pragma once
+
+#include
+#include
+#include
+#include
+#include "rapidjson/document.h"
+
+namespace org::apache::nifi::minifi::procfs {
+
+class CpuStatData {
+ CpuStatData() = default;
+
+ public:
+ CpuStatData(const CpuStatData& src) = default;
+ CpuStatData(CpuStatData&& src) noexcept = default;
+
+ static std::optional parseCpuStatLine(std::istringstream& iss);
+
+ bool operator>=(const CpuStatData &rhs) const;
+ bool operator==(const CpuStatData &rhs) const;
+ CpuStatData operator-(const CpuStatData &rhs) const;
+
+ uint64_t getUser() const { return user_; }
+ uint64_t getNice() const { return nice_; }
+ uint64_t getSystem() const { return system_; }
+ uint64_t getIdle() const { return idle_; }
+ uint64_t getIoWait() const { return io_wait_; }
+ uint64_t getIrq() const { return irq_; }
+ uint64_t getSoftIrq() const { return soft_irq_; }
+ uint64_t getSteal() const { return steal_; }
+ uint64_t getGuest() const { return guest_; }
+ uint64_t getGuestNice() const { return guest_nice_; }
+
+ uint64_t getIdleAll() const { return idle_ + io_wait_; }
+ uint64_t getSystemAll() const { return system_ + irq_ + soft_irq_; }
+ uint64_t getVirtAll() const { return guest_ + guest_nice_; }
+ uint64_t getTotal() const { return user_ + nice_ + getSystemAll() +
getIdleAll() + steal_ + getVirtAll(); }
Review comment:
Cosider using `[[nodiscard]]` and `noexcept` on getters
##
File path: extensions/procfs/CpuStat.cpp
##
@@ -0,0 +1,116 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "CpuStat.h"
+
+namespace org::apache::nifi::minifi::procfs {
+
+std::optional CpuStatData::parseCpuStatLine(std::istringstream&
iss) {
+ CpuStatData data;
+ iss >> data.user_ >> data.nice_ >> data
[GitHub] [nifi] ohnoitsyou commented on a change in pull request #5743: NIFI-8899 - Add NiFi Registry version information to the Registry under an "about" button.
ohnoitsyou commented on a change in pull request #5743:
URL: https://github.com/apache/nifi/pull/5743#discussion_r806146305
##
File path:
nifi-registry/nifi-registry-core/nifi-registry-web-ui/src/main/package.json
##
@@ -66,6 +66,7 @@
"reset-css": "4.0.1",
"roboto-fontface": "0.10.0",
"rxjs": "6.5.2",
+"sass": "^1.49.0",
Review comment:
`node-sass` is deprecated. It's been replaced by `dart-sass` and the
pure javascript implementation is contained in `sass`
I'll nix the `^`
##
File path:
nifi-registry/nifi-registry-core/nifi-registry-web-api/src/main/java/org/apache/nifi/registry/web/api/VersionResource.java
##
@@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.registry.web.api;
+
+import io.swagger.annotations.Api;
+import io.swagger.annotations.ApiOperation;
+import io.swagger.annotations.Authorization;
+import org.apache.nifi.registry.NiFiRegistryApiApplication;
+import org.apache.nifi.registry.RegistryVersion;
+import org.apache.nifi.registry.event.EventService;
+import org.apache.nifi.registry.web.service.ServiceFacade;
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.stereotype.Component;
+
+import javax.ws.rs.GET;
+import javax.ws.rs.Path;
+import javax.ws.rs.Produces;
+import javax.ws.rs.core.MediaType;
+import javax.ws.rs.core.Response;
+
+@Component
+@Path("/version")
+@Api(
+value = "version",
+description = "Retrieves the version information for this NiFi
Registry.",
+authorizations = { @Authorization("Authorization") }
+)
+public class VersionResource extends ApplicationResource {
+
+@Autowired
+public VersionResource(
+final ServiceFacade serviceFacade,
+final EventService eventService) {
+super(serviceFacade, eventService);
+}
+
+@GET
+@Produces(MediaType.APPLICATION_JSON)
+@ApiOperation(
+value = "Get version",
+notes = "Gets the NiFi Registry version.",
+response = RegistryVersion.class
+)
+public Response getVersion() {
+final String implVersion =
NiFiRegistryApiApplication.class.getPackage().getImplementationVersion();
Review comment:
It ultimately returns the `project.version` from the
`nifi-registry-web-api` pom
##
File path:
nifi-registry/nifi-registry-core/nifi-registry-web-api/src/main/java/org/apache/nifi/registry/web/api/VersionResource.java
##
@@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.registry.web.api;
+
+import io.swagger.annotations.Api;
+import io.swagger.annotations.ApiOperation;
+import io.swagger.annotations.Authorization;
+import org.apache.nifi.registry.NiFiRegistryApiApplication;
+import org.apache.nifi.registry.RegistryVersion;
+import org.apache.nifi.registry.event.EventService;
+import org.apache.nifi.registry.web.service.ServiceFacade;
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.stereotype.Component;
+
+import javax.ws.rs.GET;
+import javax.ws.rs.Path;
+import javax.ws.rs.Produces;
+import javax.ws.rs.core.MediaType;
+import javax.ws.rs.core.Response;
+
+@Component
+@Path("/version")
+@Api(
+value = "version",
+description = "Retrieves the version information for this NiFi
Registry.",
+authorizations = { @Autho
[GitHub] [nifi] mcgilman commented on pull request #5247: NIFI-8927 - Add option to start/stop all controllers
mcgilman commented on pull request #5247: URL: https://github.com/apache/nifi/pull/5247#issuecomment-1039496144 @eduardofontes Thanks for the PR! The process of enabling and disabling Controller Services is complex. Currently, the NiFi UI supports enabling and disabling a single Controller Service. When enabling the user has the option to conditionally enable any dependent Controller Services and starting any referencing Processors/Reporting Tasks. When disabling the user does not have this option as the user must stop all referencing Processors/Reporting Tasks and disable any dependent Controller Services. Since there could be any number of affected components, this helps keep the operation as concise as possible, helps guarantee its success, and provides the user the necessary insight into the operation. The API that your invoking is actually part of a larger sequence [1] to update Variables and Parameters that could be referenced by Controller Services and their referencing components across all Nodes in a cluster. The UX for those actions show the user the affected components and provides updates as the cluster performs the sequence of operations. I think this feature would also benefit from a similar UX where the user can see the full scope of their action and gets updates while the action completes. [1] https://github.com/apache/nifi/blob/main/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProcessGroupResource.java#L827 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] timeabarna commented on a change in pull request #5593: NIFI-9475 Provide Framework-Level Retries for NiFi Relationships
timeabarna commented on a change in pull request #5593:
URL: https://github.com/apache/nifi/pull/5593#discussion_r806488746
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core-api/src/main/java/org/apache/nifi/controller/StandardFunnel.java
##
@@ -577,6 +588,58 @@ public String getComponentType() {
return "Funnel";
}
+@Override
+public int getRetryCount() {
+return 0;
+}
+
+@Override
+public void setRetryCount(Integer retryCount) {
+this.retryCount = retryCount;
+}
+
+@Override
+public Set getRetriedRelationships() {
+return Collections.EMPTY_SET;
+}
+
+@Override
+public void setRetriedRelationships(Set retriedRelationships) {
+this.retriedRelationships = (retriedRelationships == null) ?
Collections.emptySet() : new HashSet<>(retriedRelationships);
+}
+
+@Override
+public boolean isRelationshipRetried(Relationship relationship) {
+return false;
+}
+
+@Override
+public BackoffMechanism getBackoffMechanism() {
+return BackoffMechanism.PENALIZE_FLOWFILE;
+}
+
+@Override
+public void setBackoffMechanism(BackoffMechanism backoffMechanism) {
+this.backoffMechanism = (backoffMechanism == null) ?
BackoffMechanism.PENALIZE_FLOWFILE : backoffMechanism;
+}
+
+@Override
+public String getMaxBackoffPeriod() {
+return DEFAULT_MAX_BACKOFF_PERIOD;
+}
+
+@Override
+public void setMaxBackoffPeriod(String maxBackoffPeriod) {
+if (maxBackoffPeriod == null) {
+maxBackoffPeriod = DEFAULT_MAX_BACKOFF_PERIOD;
+}
+final long backoffNanos =
FormatUtils.getTimeDuration(maxBackoffPeriod, TimeUnit.NANOSECONDS);
+if (backoffNanos < 0) {
+throw new IllegalArgumentException("Max Backoff Period must be
positive");
+}
+this.maxBackoffPeriod = maxBackoffPeriod;
Review comment:
Setters are required because of Connectable interface. Implementing them
with doing nothing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] joewitt commented on pull request #5247: NIFI-8927 - Add option to start/stop all controllers
joewitt commented on pull request #5247: URL: https://github.com/apache/nifi/pull/5247#issuecomment-1039406858 We need to have more information on what has been verified here. What happens on a taxed system with many controller services? Does this cause blocking while that is happening? How does this play with the async handling? I'd be hesitant to merge until someone more knowledgeable has looked at this AND we have some information on testing. Obviously this is a useful feature but needs engagement from more folks in the community with more knowledge on these parts. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a change in pull request #1253: MINIFICPP-1736 Add MiNiFi agent C2 operation capabilities to agent manifest
lordgamez commented on a change in pull request #1253:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1253#discussion_r805990249
##
File path: libminifi/src/FlowController.cpp
##
@@ -431,8 +431,9 @@ int16_t FlowController::clearConnection(const std::string
&connection) {
return -1;
}
-std::shared_ptr
FlowController::getAgentManifest() const {
+std::shared_ptr
FlowController::getAgentManifest() {
auto agentInfo =
std::make_shared("agentInfo");
+
agentInfo->setUpdatePolicyController(std::static_pointer_cast(getControllerService(c2::C2Agent::UPDATE_NAME)));
Review comment:
Good point, I was a bit hesitant changing it because it was very low
core level stuff and I thought it would be convoluted, but after trying it, it
seems these controller provider getters could be made const quite easily.
Updated it in 5e37b8b76b97c6e04ee145c80293b5557348ba66
##
File path: extensions/http-curl/tests/C2NullConfiguration.cpp
##
@@ -72,15 +72,15 @@ class VerifyC2Server : public HTTPIntegrationBase {
std::string port, scheme, path;
parse_http_components(url, port, scheme, path);
-configuration->set("c2.enable", "true");
-configuration->set("c2.agent.class", "test");
-configuration->set("c2.agent.protocol.class", "RESTSender");
-configuration->set("c2.rest.url", "");
-configuration->set("c2.rest.url.ack", "");
-configuration->set("c2.agent.heartbeat.reporter.classes", "null");
-configuration->set("c2.rest.listener.port", "null");
-configuration->set("c2.agent.heartbeat.period", "null");
-configuration->set("c2.rest.listener.heartbeat.rooturi", "null");
+
configuration->set(org::apache::nifi::minifi::Configuration::nifi_c2_enable,
"true");
+
configuration->set(org::apache::nifi::minifi::Configuration::nifi_c2_agent_class,
"test");
+
configuration->set(org::apache::nifi::minifi::Configuration::nifi_c2_agent_protocol_class,
"RESTSender");
+
configuration->set(org::apache::nifi::minifi::Configuration::nifi_c2_rest_url,
"");
+
configuration->set(org::apache::nifi::minifi::Configuration::nifi_c2_rest_url_ack,
"");
+
configuration->set(org::apache::nifi::minifi::Configuration::nifi_c2_agent_heartbeat_reporter_classes,
"null");
+
configuration->set(org::apache::nifi::minifi::Configuration::nifi_c2_rest_listener_port,
"null");
+
configuration->set(org::apache::nifi::minifi::Configuration::nifi_c2_agent_heartbeat_period,
"null");
+
configuration->set(org::apache::nifi::minifi::Configuration::nifi_c2_rest_listener_heartbeat_rooturi,
"null");
Review comment:
Do you mean moving the VerifyC2Server class under minifi namespace or
something else?
##
File path: extensions/http-curl/tests/C2NullConfiguration.cpp
##
@@ -72,15 +72,15 @@ class VerifyC2Server : public HTTPIntegrationBase {
std::string port, scheme, path;
parse_http_components(url, port, scheme, path);
-configuration->set("c2.enable", "true");
-configuration->set("c2.agent.class", "test");
-configuration->set("c2.agent.protocol.class", "RESTSender");
-configuration->set("c2.rest.url", "");
-configuration->set("c2.rest.url.ack", "");
-configuration->set("c2.agent.heartbeat.reporter.classes", "null");
-configuration->set("c2.rest.listener.port", "null");
-configuration->set("c2.agent.heartbeat.period", "null");
-configuration->set("c2.rest.listener.heartbeat.rooturi", "null");
+
configuration->set(org::apache::nifi::minifi::Configuration::nifi_c2_enable,
"true");
+
configuration->set(org::apache::nifi::minifi::Configuration::nifi_c2_agent_class,
"test");
+
configuration->set(org::apache::nifi::minifi::Configuration::nifi_c2_agent_protocol_class,
"RESTSender");
+
configuration->set(org::apache::nifi::minifi::Configuration::nifi_c2_rest_url,
"");
+
configuration->set(org::apache::nifi::minifi::Configuration::nifi_c2_rest_url_ack,
"");
+
configuration->set(org::apache::nifi::minifi::Configuration::nifi_c2_agent_heartbeat_reporter_classes,
"null");
+
configuration->set(org::apache::nifi::minifi::Configuration::nifi_c2_rest_listener_port,
"null");
+
configuration->set(org::apache::nifi::minifi::Configuration::nifi_c2_agent_heartbeat_period,
"null");
+
configuration->set(org::apache::nifi::minifi::Configuration::nifi_c2_rest_listener_heartbeat_rooturi,
"null");
Review comment:
I agree, I was just curious as it is not common currently, updated in
530a1d11cce113c61dbbee173aa515f7baf6b716
##
File path: extensions/coap/controllerservice/CoapConnector.cpp
##
@@ -67,7 +67,7 @@ void CoapConnectorService::onEnable() {
core::Property::StringToInt(port_str, port_);
} else {
// this is the case where we aren't being used in the context of a single
controller service.
-if (configuration_->get("nifi.c2.agent.coap.host", host_) &&
configuration_->get("nifi.c2.agent.coap.port", port_str)
[GitHub] [nifi] markap14 commented on a change in pull request #5593: NIFI-9475 Provide Framework-Level Retries for NiFi Relationships
markap14 commented on a change in pull request #5593:
URL: https://github.com/apache/nifi/pull/5593#discussion_r792878063
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core-api/src/main/java/org/apache/nifi/controller/AbstractPort.java
##
@@ -659,4 +670,56 @@ public void setVersionedComponentId(final String
versionedComponentId) {
}
}
}
+
+@Override
+public int getRetryCount() {
+return 0;
+}
+
+@Override
+public void setRetryCount(Integer retryCount) {
+this.retryCount = retryCount;
+}
+
+@Override
+public Set getRetriedRelationships() {
+return Collections.EMPTY_SET;
Review comment:
Should use `Collections.emptySet()`
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core-api/src/main/java/org/apache/nifi/controller/AbstractPort.java
##
@@ -659,4 +670,56 @@ public void setVersionedComponentId(final String
versionedComponentId) {
}
}
}
+
+@Override
+public int getRetryCount() {
+return 0;
+}
+
+@Override
+public void setRetryCount(Integer retryCount) {
+this.retryCount = retryCount;
Review comment:
There's no need to store this if we always return `0`. This can be a
no-op
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-components/src/main/java/org/apache/nifi/controller/repository/StandardProcessSession.java
##
@@ -289,63 +292,112 @@ private void checkpoint(final boolean copyCollections) {
// validate that all records have a transfer relationship for them and
if so determine the destination node and clone as necessary
final Map toAdd = new HashMap<>();
+final Map uuidsToRecords =
records.values()
+.stream()
+.collect(Collectors.toMap(record ->
record.getCurrent().getAttribute(CoreAttributes.UUID.key()),
Function.identity()));
+
for (final StandardRepositoryRecord record : records.values()) {
if (record.isMarkedForDelete()) {
continue;
}
+if (records.get(record.getCurrent().getId()) == null) {
+continue;
+}
+
final Relationship relationship = record.getTransferRelationship();
-final List destinations = new
ArrayList<>(context.getConnections(relationship));
-if (destinations.isEmpty() && relationship == Relationship.SELF) {
-record.setDestination(record.getOriginalQueue());
-} else if (destinations.isEmpty()) {
-record.markForDelete();
+final Connectable connectable = context.getConnectable();
+final FlowFileRecord currentFlowFile = record.getOriginal();
Review comment:
Is the intent here to get the original version of the FlowFile or the
current version? I'm confused by the variable name `currentFlowFile` when we're
using `record.getOriginal()` - we should either rename variable to
`originalFlowFile` if that's what we want or call `record.getCurrent()` if
that's what we want.
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-components/src/main/java/org/apache/nifi/controller/repository/StandardProcessSession.java
##
@@ -301,13 +301,13 @@ private void checkpoint(final boolean copyCollections) {
continue;
}
-final Relationship relationship = record.getTransferRelationship();
-ProcessorNode processorNode = null;
-
-if (context.getConnectable() instanceof ProcessorNode) {
-processorNode = (ProcessorNode) context.getConnectable();
+if (records.get(record.getCurrent().getId()) == null) {
+continue;
}
Review comment:
Why was this check added? The Record came from the `this.records` Map -
why are we checking that it exists in the map?
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core-api/src/main/java/org/apache/nifi/controller/AbstractPort.java
##
@@ -659,4 +670,56 @@ public void setVersionedComponentId(final String
versionedComponentId) {
}
}
}
+
+@Override
+public int getRetryCount() {
+return 0;
+}
+
+@Override
+public void setRetryCount(Integer retryCount) {
+this.retryCount = retryCount;
+}
+
+@Override
+public Set getRetriedRelationships() {
+return Collections.EMPTY_SET;
+}
+
+@Override
+public void setRetriedRelationships(Set retriedRelationships) {
+this.retriedRelationships = (retriedRelationships == null) ?
Collections.emptySet() : new HashSet<>(retriedRelationships);
+}
+
+@Override
+public boolean isRelationshipRetried(Relationship relationship) {
+
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #1253: MINIFICPP-1736 Add MiNiFi agent C2 operation capabilities to agent manifest
szaszm commented on a change in pull request #1253:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1253#discussion_r806045280
##
File path: extensions/http-curl/tests/C2NullConfiguration.cpp
##
@@ -72,15 +72,15 @@ class VerifyC2Server : public HTTPIntegrationBase {
std::string port, scheme, path;
parse_http_components(url, port, scheme, path);
-configuration->set("c2.enable", "true");
-configuration->set("c2.agent.class", "test");
-configuration->set("c2.agent.protocol.class", "RESTSender");
-configuration->set("c2.rest.url", "");
-configuration->set("c2.rest.url.ack", "");
-configuration->set("c2.agent.heartbeat.reporter.classes", "null");
-configuration->set("c2.rest.listener.port", "null");
-configuration->set("c2.agent.heartbeat.period", "null");
-configuration->set("c2.rest.listener.heartbeat.rooturi", "null");
+
configuration->set(org::apache::nifi::minifi::Configuration::nifi_c2_enable,
"true");
+
configuration->set(org::apache::nifi::minifi::Configuration::nifi_c2_agent_class,
"test");
+
configuration->set(org::apache::nifi::minifi::Configuration::nifi_c2_agent_protocol_class,
"RESTSender");
+
configuration->set(org::apache::nifi::minifi::Configuration::nifi_c2_rest_url,
"");
+
configuration->set(org::apache::nifi::minifi::Configuration::nifi_c2_rest_url_ack,
"");
+
configuration->set(org::apache::nifi::minifi::Configuration::nifi_c2_agent_heartbeat_reporter_classes,
"null");
+
configuration->set(org::apache::nifi::minifi::Configuration::nifi_c2_rest_listener_port,
"null");
+
configuration->set(org::apache::nifi::minifi::Configuration::nifi_c2_agent_heartbeat_period,
"null");
+
configuration->set(org::apache::nifi::minifi::Configuration::nifi_c2_rest_listener_heartbeat_rooturi,
"null");
Review comment:
yes, the `VerifyC2Server` class. I think we should do all testing in the
same namespaces that we're working in, not in the global namespace.
##
File path: encrypt-config/ConfigFile.cpp
##
@@ -23,11 +23,12 @@
#include
#include "utils/StringUtils.h"
+#include "properties/Configuration.h"
namespace {
-constexpr std::array
DEFAULT_SENSITIVE_PROPERTIES{"nifi.security.client.pass.phrase",
-
"nifi.rest.api.password"};
-constexpr const char* ADDITIONAL_SENSITIVE_PROPS_PROPERTY_NAME =
"nifi.sensitive.props.additional.keys";
+constexpr std::array
DEFAULT_SENSITIVE_PROPERTIES{org::apache::nifi::minifi::Configuration::nifi_security_client_pass_phrase,
+
org::apache::nifi::minifi::Configuration::nifi_rest_api_password};
+constexpr const char* ADDITIONAL_SENSITIVE_PROPS_PROPERTY_NAME =
org::apache::nifi::minifi::Configuration::nifi_sensitive_props_additional_keys;
Review comment:
the `minifi::` prefix is also redundant, unless there is a conflicting
Configuration class in the encrypt_config namespace
##
File path: encrypt-config/ConfigFile.cpp
##
@@ -23,11 +23,10 @@
#include
#include "utils/StringUtils.h"
+#include "properties/Configuration.h"
namespace {
-constexpr std::array
DEFAULT_SENSITIVE_PROPERTIES{"nifi.security.client.pass.phrase",
-
"nifi.rest.api.password"};
-constexpr const char* ADDITIONAL_SENSITIVE_PROPS_PROPERTY_NAME =
"nifi.sensitive.props.additional.keys";
+
Review comment:
Please remove this empty namespace
##
File path: encrypt-config/tests/ConfigFileEncryptorTests.cpp
##
@@ -76,13 +77,13 @@ TEST_CASE("ConfigFileEncryptor can encrypt the sensitive
properties", "[encrypt-
SECTION("default properties") {
ConfigFile test_file{std::ifstream{"resources/minifi.properties"}};
-std::string original_password =
test_file.getValue("nifi.rest.api.password").value();
+std::string original_password =
test_file.getValue(org::apache::nifi::minifi::Configuration::nifi_rest_api_password).value();
Review comment:
Consider either moving the tests into the minifi namespace or using an
alias. There are already aliases on line 28-30, but I don't think it would be
too difficult to wrap the file.
##
File path: libminifi/include/properties/Configuration.h
##
@@ -99,6 +139,30 @@ class Configuration : public Properties {
static constexpr const char *minifi_disk_space_watchdog_interval =
"minifi.disk.space.watchdog.interval";
static constexpr const char *minifi_disk_space_watchdog_stop_threshold =
"minifi.disk.space.watchdog.stop.threshold";
static constexpr const char *minifi_disk_space_watchdog_restart_threshold =
"minifi.disk.space.watchdog.restart.threshold";
+
+ // JNI options
+ static constexpr const char *nifi_framework_dir = "nifi.framework.dir";
+ static constexpr const char *nifi_jvm_options = "nifi.jvm.options";
+ static constexpr const char
[GitHub] [nifi] bbende commented on a change in pull request #5743: NIFI-8899 - Add NiFi Registry version information to the Registry under an "about" button.
bbende commented on a change in pull request #5743:
URL: https://github.com/apache/nifi/pull/5743#discussion_r806105957
##
File path:
nifi-registry/nifi-registry-core/nifi-registry-web-api/src/main/java/org/apache/nifi/registry/web/api/VersionResource.java
##
@@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.registry.web.api;
+
+import io.swagger.annotations.Api;
+import io.swagger.annotations.ApiOperation;
+import io.swagger.annotations.Authorization;
+import org.apache.nifi.registry.NiFiRegistryApiApplication;
+import org.apache.nifi.registry.RegistryVersion;
+import org.apache.nifi.registry.event.EventService;
+import org.apache.nifi.registry.web.service.ServiceFacade;
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.stereotype.Component;
+
+import javax.ws.rs.GET;
+import javax.ws.rs.Path;
+import javax.ws.rs.Produces;
+import javax.ws.rs.core.MediaType;
+import javax.ws.rs.core.Response;
+
+@Component
+@Path("/version")
+@Api(
+value = "version",
+description = "Retrieves the version information for this NiFi
Registry.",
+authorizations = { @Authorization("Authorization") }
+)
+public class VersionResource extends ApplicationResource {
+
+@Autowired
+public VersionResource(
+final ServiceFacade serviceFacade,
+final EventService eventService) {
+super(serviceFacade, eventService);
+}
+
+@GET
+@Produces(MediaType.APPLICATION_JSON)
+@ApiOperation(
+value = "Get version",
+notes = "Gets the NiFi Registry version.",
+response = RegistryVersion.class
+)
+public Response getVersion() {
+final String implVersion =
NiFiRegistryApiApplication.class.getPackage().getImplementationVersion();
Review comment:
What version is actually being returned here?
We would need the Maven version like 1.16.0-SNAPSHOT. Was expecting that
Maven would have to filter the version into some file that registry core can
read from.
##
File path:
nifi-registry/nifi-registry-core/nifi-registry-web-api/src/main/java/org/apache/nifi/registry/web/api/VersionResource.java
##
@@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.registry.web.api;
+
+import io.swagger.annotations.Api;
+import io.swagger.annotations.ApiOperation;
+import io.swagger.annotations.Authorization;
+import org.apache.nifi.registry.NiFiRegistryApiApplication;
+import org.apache.nifi.registry.RegistryVersion;
+import org.apache.nifi.registry.event.EventService;
+import org.apache.nifi.registry.web.service.ServiceFacade;
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.stereotype.Component;
+
+import javax.ws.rs.GET;
+import javax.ws.rs.Path;
+import javax.ws.rs.Produces;
+import javax.ws.rs.core.MediaType;
+import javax.ws.rs.core.Response;
+
+@Component
+@Path("/version")
+@Api(
+value = "version",
+description = "Retrieves the version information for this NiFi
Registry.",
+authorizations = { @Authorization("Authorization") }
+)
+public class VersionResource extends ApplicationResource {
Review comment:
What do you think about calling this AboutResource? and the object being
returned could be AboutInfo (or something)?
My thought was the using the word "version" could be
[GitHub] [nifi] asfgit closed pull request #5766: NIFI-9685 Upgrade JNA to 5.10.0
asfgit closed pull request #5766: URL: https://github.com/apache/nifi/pull/5766 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] turcsanyip edited a comment on pull request #5692: NIFI-9609 Added nifi-snowflake-bundle with a SnowflakeConnectionPool.
turcsanyip edited a comment on pull request #5692: URL: https://github.com/apache/nifi/pull/5692#issuecomment-1039276217 @tpalfy Please do not forget to add LICENSE / NOTICE files in the nar bundle. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] mtien-apache commented on a change in pull request #5743: NIFI-8899 - Add NiFi Registry version information to the Registry under an "about" button.
mtien-apache commented on a change in pull request #5743: URL: https://github.com/apache/nifi/pull/5743#discussion_r806129029 ## File path: nifi-registry/nifi-registry-core/nifi-registry-web-ui/src/main/package.json ## @@ -66,6 +66,7 @@ "reset-css": "4.0.1", "roboto-fontface": "0.10.0", "rxjs": "6.5.2", +"sass": "^1.49.0", Review comment: What is the reason for removing `node-sass` and adding `sass`? Also, the contributors in this project typically remove `^` or `~` from packages listed in `package.json` so that unintentional version bumps do not occur when running `npm install`. ## File path: nifi-registry/nifi-registry-core/nifi-registry-web-ui/src/main/package.json ## @@ -66,6 +66,7 @@ "reset-css": "4.0.1", "roboto-fontface": "0.10.0", "rxjs": "6.5.2", +"sass": "^1.49.0", Review comment: @ohnoitsyou that makes sense. Looks like now there is a merge conflict. I believe latest upstream has similar changes that switch to the `sass` package. Would you rebase on latest upstream? Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] turcsanyip commented on pull request #5692: NIFI-9609 Added nifi-snowflake-bundle with a SnowflakeConnectionPool.
turcsanyip commented on pull request #5692: URL: https://github.com/apache/nifi/pull/5692#issuecomment-1039276217 @tpalfy Please do not forget to add LICENCE / NOTICE files in the nar bundle. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #1259: MINIFICPP-1748 - Modify log properties through the c2 protocol
adamdebreceni commented on a change in pull request #1259:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1259#discussion_r806660908
##
File path: libminifi/include/properties/Properties.h
##
@@ -29,16 +29,23 @@
#include "core/logging/Logger.h"
#include "utils/ChecksumCalculator.h"
+#include "utils/StringUtils.h"
namespace org {
namespace apache {
namespace nifi {
namespace minifi {
+enum class PropertyChangeLifetime {
+ TRANSIENT, // the changed value will not be committed to disk
+ PERSISTENT // the changed value will be written to the source file
+};
+
class Properties {
struct PropertyValue {
-std::string value;
-bool changed;
+std::string persisted_value;
+std::string active_value;
+bool changed{false}; // persisted_value should be written to disk if
requested
Review comment:
renamed
##
File path: libminifi/src/core/logging/LoggerConfiguration.cpp
##
@@ -359,6 +323,57 @@ void LoggerConfiguration::initializeCompression(const
std::lock_guard
LoggerConfiguration::getRotatingFileSink(const std::string& appender_key, const
std::shared_ptr& properties) {
+ static std::map> rotating_file_sinks;
+ static std::mutex sink_map_mtx;
+
+ std::string file_name;
+ if (!properties->getString(appender_key + ".file_name", file_name)) {
+file_name = "minifi-app.log";
+ }
+ std::string directory;
+ if (!properties->getString(appender_key + ".directory", directory)) {
+// The below part assumes logger_properties->getHome() is existing
+// Cause minifiHome must be set at MiNiFiMain.cpp?
+directory = properties->getHome() +
utils::file::FileUtils::get_separator() + "logs";
+ }
+
+ file_name = directory + utils::file::FileUtils::get_separator() + file_name;
+ if (utils::file::FileUtils::create_dir(directory) == -1) {
+std::cerr << directory << " cannot be created\n";
+exit(1);
+ }
+
+ int max_files = 3;
+ std::string max_files_str = "";
+ if (properties->getString(appender_key + ".max_files", max_files_str)) {
+try {
+ max_files = std::stoi(max_files_str);
+} catch (const std::invalid_argument &) {
+} catch (const std::out_of_range &) {
+}
+ }
+
+ int max_file_size = 5_MiB;
+ std::string max_file_size_str = "";
+ if (properties->getString(appender_key + ".max_file_size",
max_file_size_str)) {
+try {
+ max_file_size = std::stoi(max_file_size_str);
+} catch (const std::invalid_argument &) {
+} catch (const std::out_of_range &) {
+}
+ }
+
+ std::lock_guard guard(sink_map_mtx);
+ auto it = rotating_file_sinks.find(file_name);
+ if (it != rotating_file_sinks.end()) {
+return it->second;
+ }
Review comment:
added comment: in the spdlog implementation I found no guarantee that
multiple sinks to the same directory won't interfere with each other (it might
be guaranteed at the OS level though), also found this
https://github.com/gabime/spdlog/discussions/2151
##
File path: libminifi/src/core/logging/LoggerConfiguration.cpp
##
@@ -359,6 +323,57 @@ void LoggerConfiguration::initializeCompression(const
std::lock_guard
LoggerConfiguration::getRotatingFileSink(const std::string& appender_key, const
std::shared_ptr& properties) {
+ static std::map> rotating_file_sinks;
+ static std::mutex sink_map_mtx;
+
+ std::string file_name;
+ if (!properties->getString(appender_key + ".file_name", file_name)) {
+file_name = "minifi-app.log";
+ }
+ std::string directory;
+ if (!properties->getString(appender_key + ".directory", directory)) {
+// The below part assumes logger_properties->getHome() is existing
+// Cause minifiHome must be set at MiNiFiMain.cpp?
+directory = properties->getHome() +
utils::file::FileUtils::get_separator() + "logs";
+ }
+
+ file_name = directory + utils::file::FileUtils::get_separator() + file_name;
+ if (utils::file::FileUtils::create_dir(directory) == -1) {
+std::cerr << directory << " cannot be created\n";
+exit(1);
+ }
+
+ int max_files = 3;
+ std::string max_files_str = "";
+ if (properties->getString(appender_key + ".max_files", max_files_str)) {
+try {
+ max_files = std::stoi(max_files_str);
+} catch (const std::invalid_argument &) {
+} catch (const std::out_of_range &) {
+}
+ }
+
+ int max_file_size = 5_MiB;
+ std::string max_file_size_str = "";
+ if (properties->getString(appender_key + ".max_file_size",
max_file_size_str)) {
+try {
+ max_file_size = std::stoi(max_file_size_str);
+} catch (const std::invalid_argument &) {
+} catch (const std::out_of_range &) {
+}
+ }
+
+ std::lock_guard guard(sink_map_mtx);
+ auto it = rotating_file_sinks.find(file_name);
+ if (it != rotating_file_sinks.end()) {
+return it->second;
+ }
Review comment:
this also might be relevant: https://github.com/gabime/spdlog/issues/243
> If you want multiple loggers to share the same file, they must share the
same sin
[GitHub] [nifi] markap14 commented on pull request #5324: NIFI-9072: improvements to ValidateXML including validate XML in attr…
markap14 commented on pull request #5324: URL: https://github.com/apache/nifi/pull/5324#issuecomment-1039315005 I think I would be more inclined to accept this PR than to introduce a separate ValidateXMLAttribute processor, as well. If there's concern about adding the character set property, that can be easily addressed by adding an "XML Source" property - either "FlowFile Content" or "FlowFile Attribute" and then make the attribute name dependent, as well as making the "character set" property dependent. That way, we require the Character Set if reading FlowFile content and it's not an option if using an attribute. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] kevdoran closed pull request #5751: [NIFI-9554] Support building and running NiFi on arm64 platforms and consume nifi-fds 0.3.0
kevdoran closed pull request #5751: URL: https://github.com/apache/nifi/pull/5751 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] asfgit closed pull request #5763: NIFI-9681 Upgrade Apache Commons DBCP to 2.9.0
asfgit closed pull request #5763: URL: https://github.com/apache/nifi/pull/5763 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] pvillard31 commented on pull request #5763: NIFI-9681 Upgrade Apache Commons DBCP to 2.9.0
pvillard31 commented on pull request #5763: URL: https://github.com/apache/nifi/pull/5763#issuecomment-1039358787 Merged, thanks @exceptionfactory -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] fgerlits commented on a change in pull request #1259: MINIFICPP-1748 - Modify log properties through the c2 protocol
fgerlits commented on a change in pull request #1259:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1259#discussion_r806621990
##
File path: libminifi/include/properties/Properties.h
##
@@ -29,16 +29,23 @@
#include "core/logging/Logger.h"
#include "utils/ChecksumCalculator.h"
+#include "utils/StringUtils.h"
namespace org {
namespace apache {
namespace nifi {
namespace minifi {
+enum class PropertyChangeLifetime {
+ TRANSIENT, // the changed value will not be committed to disk
+ PERSISTENT // the changed value will be written to the source file
+};
+
class Properties {
struct PropertyValue {
-std::string value;
-bool changed;
+std::string persisted_value;
+std::string active_value;
+bool changed{false}; // persisted_value should be written to disk if
requested
Review comment:
I would rename this to something like `need_to_persist_new_value`, and
then the comment is no longer needed
##
File path: libminifi/src/core/logging/LoggerConfiguration.cpp
##
@@ -359,6 +323,57 @@ void LoggerConfiguration::initializeCompression(const
std::lock_guard
LoggerConfiguration::getRotatingFileSink(const std::string& appender_key, const
std::shared_ptr& properties) {
+ static std::map> rotating_file_sinks;
+ static std::mutex sink_map_mtx;
+
+ std::string file_name;
+ if (!properties->getString(appender_key + ".file_name", file_name)) {
+file_name = "minifi-app.log";
+ }
+ std::string directory;
+ if (!properties->getString(appender_key + ".directory", directory)) {
+// The below part assumes logger_properties->getHome() is existing
+// Cause minifiHome must be set at MiNiFiMain.cpp?
+directory = properties->getHome() +
utils::file::FileUtils::get_separator() + "logs";
+ }
+
+ file_name = directory + utils::file::FileUtils::get_separator() + file_name;
+ if (utils::file::FileUtils::create_dir(directory) == -1) {
+std::cerr << directory << " cannot be created\n";
+exit(1);
+ }
+
+ int max_files = 3;
+ std::string max_files_str = "";
+ if (properties->getString(appender_key + ".max_files", max_files_str)) {
+try {
+ max_files = std::stoi(max_files_str);
+} catch (const std::invalid_argument &) {
+} catch (const std::out_of_range &) {
+}
+ }
+
+ int max_file_size = 5_MiB;
+ std::string max_file_size_str = "";
+ if (properties->getString(appender_key + ".max_file_size",
max_file_size_str)) {
+try {
+ max_file_size = std::stoi(max_file_size_str);
+} catch (const std::invalid_argument &) {
+} catch (const std::out_of_range &) {
+}
+ }
+
+ std::lock_guard guard(sink_map_mtx);
+ auto it = rotating_file_sinks.find(file_name);
+ if (it != rotating_file_sinks.end()) {
+return it->second;
+ }
Review comment:
What is the reason for this new caching? It looks a bit weird that we
do it for the rolling appender but not for the other appender types. Is it
possible that some update will not get applied because of a cached old value?
##
File path: libminifi/src/core/logging/LoggerConfiguration.cpp
##
@@ -359,6 +323,57 @@ void LoggerConfiguration::initializeCompression(const
std::lock_guard
LoggerConfiguration::getRotatingFileSink(const std::string& appender_key, const
std::shared_ptr& properties) {
+ static std::map> rotating_file_sinks;
+ static std::mutex sink_map_mtx;
+
+ std::string file_name;
+ if (!properties->getString(appender_key + ".file_name", file_name)) {
+file_name = "minifi-app.log";
+ }
+ std::string directory;
+ if (!properties->getString(appender_key + ".directory", directory)) {
+// The below part assumes logger_properties->getHome() is existing
+// Cause minifiHome must be set at MiNiFiMain.cpp?
+directory = properties->getHome() +
utils::file::FileUtils::get_separator() + "logs";
+ }
+
+ file_name = directory + utils::file::FileUtils::get_separator() + file_name;
+ if (utils::file::FileUtils::create_dir(directory) == -1) {
+std::cerr << directory << " cannot be created\n";
+exit(1);
+ }
+
+ int max_files = 3;
+ std::string max_files_str = "";
+ if (properties->getString(appender_key + ".max_files", max_files_str)) {
+try {
+ max_files = std::stoi(max_files_str);
+} catch (const std::invalid_argument &) {
+} catch (const std::out_of_range &) {
+}
+ }
+
+ int max_file_size = 5_MiB;
+ std::string max_file_size_str = "";
+ if (properties->getString(appender_key + ".max_file_size",
max_file_size_str)) {
+try {
+ max_file_size = std::stoi(max_file_size_str);
+} catch (const std::invalid_argument &) {
+} catch (const std::out_of_range &) {
+}
+ }
+
+ std::lock_guard guard(sink_map_mtx);
+ auto it = rotating_file_sinks.find(file_name);
+ if (it != rotating_file_sinks.end()) {
+return it->second;
+ }
Review comment:
Thanks, makes sense. But I don't think the new comment is clear;
something like
```
// Acc
[GitHub] [nifi] mcgilman commented on pull request #5765: NIFI-9543: Add bring-to-front functionality to labels
mcgilman commented on pull request #5765: URL: https://github.com/apache/nifi/pull/5765#issuecomment-1040309958 Reviewing... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] markap14 merged pull request #5749: NIFI-9233 - Improve reliability of system integration tests
markap14 merged pull request #5749: URL: https://github.com/apache/nifi/pull/5749 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] fgerlits closed pull request #1255: MINIFICPP-1750 Fix C2 clear operation with corecomponentstate operand
fgerlits closed pull request #1255: URL: https://github.com/apache/nifi-minifi-cpp/pull/1255 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] fgerlits closed pull request #1259: MINIFICPP-1748 - Modify log properties through the c2 protocol
fgerlits closed pull request #1259: URL: https://github.com/apache/nifi-minifi-cpp/pull/1259 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] adam-markovics commented on a change in pull request #1252: MINIFICPP-1686 - Processor destructors are not called
adam-markovics commented on a change in pull request #1252:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1252#discussion_r806957651
##
File path: extensions/aws/processors/ListS3.h
##
@@ -33,6 +33,7 @@ namespace org {
namespace apache {
namespace nifi {
namespace minifi {
+
Review comment:
Done.
##
File path: CMakeLists.txt
##
@@ -611,6 +612,11 @@ if (ENABLE_KUBERNETES)
createExtension(KUBERNETES-EXTENSIONS "KUBERNETES EXTENSIONS" "This enables
Kubernetes support" "extensions/kubernetes")
endif()
+## Add the test-processors extension
+if (ENABLE_ALL OR NOT SKIP_TESTS OR ENABLE_TEST_PROCESSORS)
+ createExtension(TEST-PROCESSORS-EXTENSION "TEST-PROCESSORS EXTENSION"
"This enables processors purely for testing purposes"
"extensions/test-processors")
+endif()
Review comment:
So you just want the condition to be `if (NOT SKIP_TESTS)` ?
`ENABLE_TEST_PROCESSORS` was introduced to avoid building all unit tests in
docker integration tests, where `SKIP_TESTS` is true by default. But these
processors are still needed for integration tests.
##
File path: extensions/coap/tests/CoapC2VerifyHeartbeat.cpp
##
@@ -92,10 +92,10 @@ class VerifyCoAPServer : public CoapIntegrationBase {
}
void queryRootProcessGroup(std::shared_ptr pg) override {
-std::shared_ptr proc = pg->findProcessorByName("invoke");
+auto proc = pg->findProcessorByName("invoke");
Review comment:
Done.
##
File path: extensions/coap/tests/CoapC2VerifyHeartbeat.cpp
##
@@ -92,10 +92,10 @@ class VerifyCoAPServer : public CoapIntegrationBase {
}
void queryRootProcessGroup(std::shared_ptr pg) override {
-std::shared_ptr proc = pg->findProcessorByName("invoke");
+auto proc = pg->findProcessorByName("invoke");
assert(proc != nullptr);
-std::shared_ptr inv =
std::dynamic_pointer_cast(proc);
+auto inv = dynamic_cast(proc);
Review comment:
Done.
##
File path: extensions/http-curl/tests/C2NullConfiguration.cpp
##
@@ -61,10 +61,10 @@ class VerifyC2Server : public HTTPIntegrationBase {
}
void queryRootProcessGroup(std::shared_ptr pg) override {
-std::shared_ptr proc = pg->findProcessorByName("invoke");
+auto* const proc = pg->findProcessorByName("invoke");
assert(proc != nullptr);
-std::shared_ptr inv =
std::dynamic_pointer_cast(proc);
+const auto inv = dynamic_cast(proc);
Review comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] nrnalamalpu commented on pull request #5291: NIFI-9036 - Add AWS MSK IAM support to Kafka processors
nrnalamalpu commented on pull request #5291: URL: https://github.com/apache/nifi/pull/5291#issuecomment-1040277206 Can anyone please look into this PR and see if we can close this out? thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] asfgit closed pull request #5769: NIFI-9686: Renamed SNMP integration tests correctly
asfgit closed pull request #5769: URL: https://github.com/apache/nifi/pull/5769 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] thenatog commented on pull request #5701: NIFI-9607: Honor Update Keys when Quoting Identifiers in PutDatabaseRecord
thenatog commented on pull request #5701: URL: https://github.com/apache/nifi/pull/5701#issuecomment-1040463251 Will review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] bejancsaba commented on a change in pull request #5755: NIFI-9538: Add C2 heartbeat capability to minifi-c2-service
bejancsaba commented on a change in pull request #5755:
URL: https://github.com/apache/nifi/pull/5755#discussion_r806855762
##
File path:
minifi/minifi-c2/minifi-c2-service/src/main/java/org/apache/nifi/minifi/c2/service/SimpleC2ProtocolService.java
##
@@ -0,0 +1,120 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.c2.service;
+
+import org.apache.nifi.c2.protocol.api.C2Heartbeat;
+import org.apache.nifi.c2.protocol.api.C2HeartbeatResponse;
+import org.apache.nifi.c2.protocol.api.C2Operation;
+import org.apache.nifi.c2.protocol.api.C2OperationAck;
+import org.apache.nifi.c2.protocol.api.C2OperationState;
+import org.apache.nifi.c2.protocol.api.OperandType;
+import org.apache.nifi.c2.protocol.api.OperationState;
+import org.apache.nifi.c2.protocol.api.OperationType;
+import org.apache.nifi.minifi.c2.util.LogMarkerUtil;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.slf4j.Marker;
+import org.springframework.stereotype.Service;
+
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.UUID;
+
+@Service
+public class SimpleC2ProtocolService implements C2ProtocolService {
+
+private static final Logger logger =
LoggerFactory.getLogger(SimpleC2ProtocolService.class);
+
+private static final Set issuedOperationIds = new HashSet<>();
+
+public SimpleC2ProtocolService() {
+}
+
+@Override
+public void processOperationAck(final C2OperationAck operationAck, final
C2ProtocolContext context) {
+// This service assumes there is a single Operation UPDATE to pass
over the updated flow
+Marker marker = LogMarkerUtil.getMarker(operationAck);
+
+logger.debug(marker, "Received operation acknowledgement: {}; {}",
operationAck, context);
+// Remove the operator ID from the list of issued operations and log
the state
+final String operationId = operationAck.getOperationId();
+try {
+OperationState opState = OperationState.DONE;
+String details = null;
+
+/* Partial applications are rare and only happen when an operation
consists of updating multiple config
+ * items and some succeed ( we don't yet have the concept of
rollback in agents ).
+ * Fully Applied yields an operation success.
+ * Operation Not Understood and Not Applied give little details
but also will result in Operation Failure.
+ * We should explore if providing textual details. */
+final C2OperationState c2OperationState =
operationAck.getOperationState();
+if (null != c2OperationState) {
+details = c2OperationState.getDetails();
+if (c2OperationState.getState() !=
C2OperationState.OperationState.FULLY_APPLIED) {
+opState = OperationState.FAILED;
+}
+}
+
+if (!issuedOperationIds.remove(operationId)) {
+logger.warn(marker, "Operation with ID " + operationId + " has
either already been acknowledged or is unknown to this server");
+} else if (null != c2OperationState) {
+final C2OperationState.OperationState operationState =
c2OperationState.getState();
+logger.debug("Operation with ID " + operationId + "
acknowledged with a state of " + operationState.name() + "(" + opState.name() +
"), details = "
++ (details == null ? "" : details));
+}
+
+// Optionally, an acknowledgement can include some of the info
normally passed in a heartbeat.
+// If this info is present, process it as a heartbeat, so we
update our latest known state of the agent.
+if (operationAck.getAgentInfo() != null
+|| operationAck.getDeviceInfo() != null
+|| operationAck.getFlowInfo() != null) {
+final C2Heartbeat heartbeatInfo = toHeartbeat(operationAck);
+logger.trace(marker, "Operation acknowledgement contains
additional info. Processing as heartbeat: {}", heartbeatInfo);
+processHeartbeat(heartbeatInf
[GitHub] [nifi] mattyb149 closed pull request #5746: NIFI-6740: Add configuration options to specify NiFi/Bootstrap communication ports
mattyb149 closed pull request #5746: URL: https://github.com/apache/nifi/pull/5746 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] mattyb149 commented on pull request #5746: NIFI-6740: Add configuration options to specify NiFi/Bootstrap communication ports
mattyb149 commented on pull request #5746: URL: https://github.com/apache/nifi/pull/5746#issuecomment-1040508165 +1 LGTM, verified the expected behavior. Thanks for the improvement! Merged to main -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm commented on pull request #1261: MINIFICPP-1752 Remove the catch2 dependency from our the test infrast…
szaszm commented on pull request #1261:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1261#issuecomment-1040382220
During a pre-merge check, I got this compilation error:
```
/home/szaszm/nifi-minifi-cpp-3/extensions/expression-language/tests/ProcessContextExprTests.cpp:54:10:
error: expected constructor, destructor, or type conversion before ‘(’ token
54 | TEST_CASE("ProcessContextExpr can update existing processor
properties", "[setProperty][getProperty]") {
| ^
make[2]: ***
[extensions/expression-language/tests/CMakeFiles/ProcessContextExprTests.dir/build.make:76:
extensions/expression-language/tests/CMakeFiles/ProcessContextExprTests.dir/ProcessContextExprTests.cpp.o]
Error 1
make[1]: *** [CMakeFiles/Makefile2:9470:
extensions/expression-language/tests/CMakeFiles/ProcessContextExprTests.dir/all]
Error 2
make[1]: *** Waiting for unfinished jobs
```
Could you fix it? We just need another Catch.h include in
ProcessContextExprTests.cpp, but I don't want to change files in a merge commit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #1252: MINIFICPP-1686 - Processor destructors are not called
szaszm commented on a change in pull request #1252:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1252#discussion_r798799427
##
File path: extensions/aws/processors/ListS3.h
##
@@ -33,6 +33,7 @@ namespace org {
namespace apache {
namespace nifi {
namespace minifi {
+
Review comment:
Please remove this unnecessary new line
##
File path: extensions/http-curl/tests/C2NullConfiguration.cpp
##
@@ -61,10 +61,10 @@ class VerifyC2Server : public HTTPIntegrationBase {
}
void queryRootProcessGroup(std::shared_ptr pg) override {
-std::shared_ptr proc = pg->findProcessorByName("invoke");
+auto* const proc = pg->findProcessorByName("invoke");
assert(proc != nullptr);
-std::shared_ptr inv =
std::dynamic_pointer_cast(proc);
+const auto inv = dynamic_cast(proc);
Review comment:
Use `auto* const` here as well. Consider adding const to the pointer as
well, since it's only read from in these functions.
##
File path: extensions/coap/tests/CoapC2VerifyHeartbeat.cpp
##
@@ -92,10 +92,10 @@ class VerifyCoAPServer : public CoapIntegrationBase {
}
void queryRootProcessGroup(std::shared_ptr pg) override {
-std::shared_ptr proc = pg->findProcessorByName("invoke");
+auto proc = pg->findProcessorByName("invoke");
Review comment:
Use `auto* const` here as well
##
File path: CMakeLists.txt
##
@@ -611,6 +612,11 @@ if (ENABLE_KUBERNETES)
createExtension(KUBERNETES-EXTENSIONS "KUBERNETES EXTENSIONS" "This enables
Kubernetes support" "extensions/kubernetes")
endif()
+## Add the test-processors extension
+if (ENABLE_ALL OR NOT SKIP_TESTS OR ENABLE_TEST_PROCESSORS)
+ createExtension(TEST-PROCESSORS-EXTENSION "TEST-PROCESSORS EXTENSION"
"This enables processors purely for testing purposes"
"extensions/test-processors")
+endif()
Review comment:
I would run them when `NOT SKIP_TESTS`, regardless of the other options,
since it's basically testing.
##
File path: extensions/coap/tests/CoapC2VerifyHeartbeat.cpp
##
@@ -92,10 +92,10 @@ class VerifyCoAPServer : public CoapIntegrationBase {
}
void queryRootProcessGroup(std::shared_ptr pg) override {
-std::shared_ptr proc = pg->findProcessorByName("invoke");
+auto proc = pg->findProcessorByName("invoke");
assert(proc != nullptr);
-std::shared_ptr inv =
std::dynamic_pointer_cast(proc);
+auto inv = dynamic_cast(proc);
Review comment:
Use `auto* const` here as well. Consider adding const to the pointer as
well, since it's only read from in these functions.
##
File path: CMakeLists.txt
##
@@ -611,6 +612,11 @@ if (ENABLE_KUBERNETES)
createExtension(KUBERNETES-EXTENSIONS "KUBERNETES EXTENSIONS" "This enables
Kubernetes support" "extensions/kubernetes")
endif()
+## Add the test-processors extension
+if (ENABLE_ALL OR NOT SKIP_TESTS OR ENABLE_TEST_PROCESSORS)
+ createExtension(TEST-PROCESSORS-EXTENSION "TEST-PROCESSORS EXTENSION"
"This enables processors purely for testing purposes"
"extensions/test-processors")
+endif()
Review comment:
Ok, this makes sense, in this case could you please add a comment about
this relationship?
And consider removing ENABLE_ALL, since these are not end-user feature
processors.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm closed pull request #1261: MINIFICPP-1752 Remove the catch2 dependency from our the test infrast…
szaszm closed pull request #1261: URL: https://github.com/apache/nifi-minifi-cpp/pull/1261 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] markobean commented on pull request #5743: NIFI-8899 - Add NiFi Registry version information to the Registry under an "about" button.
markobean commented on pull request #5743: URL: https://github.com/apache/nifi/pull/5743#issuecomment-1040434810 I installed Registry to see how the UI reports the version. This is a good start. I'd like to recommend the UI dialog mimic the "About" dialog for Apache NiFi for a consistent look across related applications. For example, "About NiFi Registry" placed on a colored background, inclusion of NiFi Registry logo, and a short description of Registry. Perhaps the description can be that of the Registry website: "Registry—a subproject of Apache NiFi—is a complementary application that provides a central location for storage and management of shared resources across one or more instances of NiFi and/or MiNiFi." -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-9689) Do not schedule processors when all FlowFiles are penalized
Mark Payne created NIFI-9689: Summary: Do not schedule processors when all FlowFiles are penalized Key: NIFI-9689 URL: https://issues.apache.org/jira/browse/NIFI-9689 Project: Apache NiFi Issue Type: Improvement Components: Core Framework Reporter: Mark Payne Assignee: Mark Payne When all FlowFiles in a given queue are penalized, the destination processor is still scheduled to run. As a result, the Processor is triggered, no data is available, and it generally returns. However, this means that the UI shows millions of tasks completed over a short period of time, when no where was actually done. Further, it means that we constantly schedule the processor to run as fast as it can, which wastes compute resources. Instead, we should detect that the head of the queue is penalized (and therefore all FlowFiles in the queue are penalized), and treat it the same as we would if the queue were empty, which allows the "bored yield duration" to take effect, significantly reducing CPU cycles. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [nifi] markap14 opened a new pull request #5771: NIFI-9689: When all FlowFiles in a FlowFile Queue are penalized, do n…
markap14 opened a new pull request #5771:
URL: https://github.com/apache/nifi/pull/5771
…ot schedule the destination to run. Also expose this fact via the
ConnectionStatusSnapshotDTO, as this allows the front-end to render this
information to the user in order to avoid confusion when it appears that the
Processor has data but does nothing
Thank you for submitting a contribution to Apache NiFi.
Please provide a short description of the PR here:
Description of PR
_Enables X functionality; fixes bug NIFI-._
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [ ] Does your PR title start with **NIFI-** where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [ ] Has your PR been rebased against the latest commit within the target
branch (typically `main`)?
- [ ] Is your initial contribution a single, squashed commit? _Additional
commits in response to PR reviewer feedback should be made on this branch and
pushed to allow change tracking. Do not `squash` or use `--force` when pushing
to allow for clean monitoring of changes._
### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via `mvn
-Pcontrib-check clean install` at the root `nifi` folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] Have you verified that the full build is successful on JDK 8?
- [ ] Have you verified that the full build is successful on JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main
`LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main
`NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI for
build issues and submit an update to your PR as soon as possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] markap14 merged pull request #5764: NIFI-9684 FIx: Starting a PG starts RPGs in sibling PGs as well
markap14 merged pull request #5764: URL: https://github.com/apache/nifi/pull/5764 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] markap14 commented on pull request #5764: NIFI-9684 FIx: Starting a PG starts RPGs in sibling PGs as well
markap14 commented on pull request #5764: URL: https://github.com/apache/nifi/pull/5764#issuecomment-1040729587 Thanks for the fix @tpalfy ! I was able to verify that it's working as expected now. +1 will merge to main. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-9684) Starting a PG starts RPGs in sibling PGs as well
[ https://issues.apache.org/jira/browse/NIFI-9684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-9684: - Fix Version/s: 1.16.0 > Starting a PG starts RPGs in sibling PGs as well > > > Key: NIFI-9684 > URL: https://issues.apache.org/jira/browse/NIFI-9684 > Project: Apache NiFi > Issue Type: Bug >Reporter: Tamas Palfy >Assignee: Tamas Palfy >Priority: Major > Fix For: 1.16.0 > > Time Spent: 40m > Remaining Estimate: 0h > > h3. Issue > https://issues.apache.org/jira/browse/NIFI-7788 introduced the change that > whenever a user starts an entire process group, all remote process groups > within get transmission enabled. > This works properly if the user is within the process group and issues a > start by right-clicking on the canvas. > However if the user does does this by right-clicking on the process group > (while being in its parent group), all remote process groups start > transmitting within the parent process group. This issue is also true when > stopping the process group. > h3. Root cause > The 2 cases (right clicking on the canvas vs on the process group) is handled > differently in _nf-actions.js_. When doing the latter the id of the parent of > the selected process group (aka. the current active process group) is sent > via the REST interface. > h3. Solution > Send the id of the selected process group -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (NIFI-9684) Starting a PG starts RPGs in sibling PGs as well
[ https://issues.apache.org/jira/browse/NIFI-9684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne reassigned NIFI-9684: Assignee: Tamas Palfy > Starting a PG starts RPGs in sibling PGs as well > > > Key: NIFI-9684 > URL: https://issues.apache.org/jira/browse/NIFI-9684 > Project: Apache NiFi > Issue Type: Bug >Reporter: Tamas Palfy >Assignee: Tamas Palfy >Priority: Major > Time Spent: 40m > Remaining Estimate: 0h > > h3. Issue > https://issues.apache.org/jira/browse/NIFI-7788 introduced the change that > whenever a user starts an entire process group, all remote process groups > within get transmission enabled. > This works properly if the user is within the process group and issues a > start by right-clicking on the canvas. > However if the user does does this by right-clicking on the process group > (while being in its parent group), all remote process groups start > transmitting within the parent process group. This issue is also true when > stopping the process group. > h3. Root cause > The 2 cases (right clicking on the canvas vs on the process group) is handled > differently in _nf-actions.js_. When doing the latter the id of the parent of > the selected process group (aka. the current active process group) is sent > via the REST interface. > h3. Solution > Send the id of the selected process group -- This message was sent by Atlassian Jira (v8.20.1#820001)
