[jira] [Created] (NIFI-9703) ListSFTP should allow triggering on incoming flowfiles
Armin Aschoff created NIFI-9703: --- Summary: ListSFTP should allow triggering on incoming flowfiles Key: NIFI-9703 URL: https://issues.apache.org/jira/browse/NIFI-9703 Project: Apache NiFi Issue Type: Improvement Components: Core Framework Affects Versions: 1.15.3 Environment: Linux / GCP Reporter: Armin Aschoff ListSFTP does not currently allow incoming connections, and can only be timer or CRON driven. It would be extremely useful if ListSFTP could also be triggered by an incoming flowfile to allow scenarios of only triggering List/Fetch under specific circumstances as opposed to just dumb time triggers. This could also be very useful for changing specific ListSFTP configurations on the fly. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (NIFI-9702) ListSFTP does not set mime.type for corresponding recordwriter
Armin Aschoff created NIFI-9702: --- Summary: ListSFTP does not set mime.type for corresponding recordwriter Key: NIFI-9702 URL: https://issues.apache.org/jira/browse/NIFI-9702 Project: Apache NiFi Issue Type: Bug Components: Core Framework Affects Versions: 1.15.3 Environment: Linux / GCP Reporter: Armin Aschoff When using a recordwriter with ListSFTP, the flowfile does not have mime.type set. For instance, when using JsonRecordSetWriter, one would expect application/json mime.type. This affects display of contents of flowfiles generated by ListSFTP. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (NIFI-9701) ListSFTP creates separate flowfiles when recordwriter is defined and Listing Strategy is "No Tracking"
Armin Aschoff created NIFI-9701: --- Summary: ListSFTP creates separate flowfiles when recordwriter is defined and Listing Strategy is "No Tracking" Key: NIFI-9701 URL: https://issues.apache.org/jira/browse/NIFI-9701 Project: Apache NiFi Issue Type: Bug Components: Core Framework Affects Versions: 1.15.3 Environment: Linux / GCP Reporter: Armin Aschoff SFTP server has multiple files in nominated directory. Configure ListSFTP with a recordwriter and Listing Strategy of "Tracking Entities", ListSFTP creates a single flowfile with content containing records of all found entities. This is as expected. Then, by simply changing the Listing Strategy to "No Tracking", ListSFTP creates one flowfile per found entity. This should not happen - ListSFTP should still produce only one flowfile with content containing records of all found entities. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [nifi] exceptionfactory commented on pull request #5692: NIFI-9609 Added nifi-snowflake-bundle with a SnowflakeConnectionPool.
exceptionfactory commented on pull request #5692: URL: https://github.com/apache/nifi/pull/5692#issuecomment-1042574240 Thanks adding the license and updating the notice information @tpalfy. With those changes, this looks just about complete. Two recent updates for Commons Lang3 (#5773) and Commons DBCP (#5763) standardized the versioning of those dependencies and removed the need to declare those versions explicitly. I can update the branch to double-check the build if there are no other changes necessary. Do you have any additional feedback @turcsanyip? Otherwise I make the dependency adjustments to confirm a successful build and then merge. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] MikeThomsen commented on pull request #5732: NIFI-9647 Added ExtractDocumentText which is a processor based on Tim…
MikeThomsen commented on pull request #5732: URL: https://github.com/apache/nifi/pull/5732#issuecomment-1042491753 @exceptionfactory all changes should be in now. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (MINIFICPP-1759) Use libc++ in the ubuntu clang GH Actions job
[ https://issues.apache.org/jira/browse/MINIFICPP-1759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz updated MINIFICPP-1759: Assignee: Marton Szasz Status: Patch Available (was: Open) https://github.com/apache/nifi-minifi-cpp/pull/1264 > Use libc++ in the ubuntu clang GH Actions job > - > > Key: MINIFICPP-1759 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1759 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Marton Szasz >Assignee: Marton Szasz >Priority: Minor > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #1264: MINIFICPP-1759 Fix linux/libc++ issues, use libc++ on ubuntu
szaszm commented on a change in pull request #1264:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1264#discussion_r808569132
##
File path: extensions/expression-language/Expression.cpp
##
@@ -636,7 +636,7 @@ Value expr_toDate(const std::vector ) {
auto utc = date::locate_zone("UTC");
auto utct = date::make_zoned(utc, t);
auto zt = date::make_zoned(zone, utct.get_local_time());
- return
Value(std::chrono::duration_cast(zt.get_sys_time().time_since_epoch()).count());
+ return
Value(int64_t{std::chrono::duration_cast(zt.get_sys_time().time_since_epoch()).count()});
Review comment:
With libc++, the count return value is `long long`, which is not
convertible to `int64_t` (alias to `long`). `Value` doesn't have a `long long`
constructor, so I had to add a non-narrowing cast to these places.
##
File path: extensions/bustache/ApplyTemplate.cpp
##
@@ -19,81 +19,76 @@
*/
#include "ApplyTemplate.h"
-#include
+#include
#include
+#include
#include
-#include
#include
-
-#include
-
-#include
+#include
#include "core/Resource.h"
+#include "bustache/model.hpp"
+
+namespace org::apache::nifi::minifi::processors {
+
+const core::Property ApplyTemplate::Template("Template", "Path to the input
mustache template file", "");
+const core::Relationship ApplyTemplate::Success("success", "success
operational on the flow record");
+
+namespace {
+class WriteCallback : public OutputStreamCallback {
+ public:
+ WriteCallback(std::filesystem::path templateFile, const core::FlowFile&
flow_file)
+ :template_file_{std::move(templateFile)},
flow_file_{std::move(flow_file)}
+ {}
+ int64_t process(const std::shared_ptr& stream) override {
+logger_->log_info("ApplyTemplate reading template file from %s",
template_file_);
+// TODO(szaszm): we might want to return to memory-mapped input files when
the next todo is done. Until then, the agents stores the whole result in memory
anyway, so no point in not doing the same
+// with the template file itself
+const auto template_file_contents = [this] {
+ std::ifstream ifs{template_file_};
+ return std::string{std::istreambuf_iterator{ifs},
std::istreambuf_iterator{}};
+}();
+
+bustache::format format(template_file_contents);
+bustache::object data;
+
+for (const auto : flow_file_.getAttributes()) {
+ data[attr.first] = attr.second;
+}
+
+// TODO(calebj) write ostream reciever for format() to prevent excessive
copying
+std::string ostring = to_string(format(data));
+stream->write(gsl::make_span(ostring).as_span());
+return gsl::narrow(ostring.length());
+ }
-namespace org {
-namespace apache {
-namespace nifi {
-namespace minifi {
-namespace processors {
-
-core::Property ApplyTemplate::Template("Template", "Path to the input mustache
template file", "");
-core::Relationship ApplyTemplate::Success("success", "success operational on
the flow record");
+ private:
+ std::shared_ptr logger_ =
core::logging::LoggerFactory::getLogger();
+ std::filesystem::path template_file_;
+ const core::FlowFile& flow_file_;
+};
+} // namespace
void ApplyTemplate::initialize() {
- //! Set the supported properties
- std::set properties;
- properties.insert(Template);
- setSupportedProperties(properties);
- //! Set the supported relationships
- std::set relationships;
- relationships.insert(Success);
- setSupportedRelationships(relationships);
+ setSupportedProperties({Template});
+ setSupportedRelationships({Success});
}
void ApplyTemplate::onTrigger(const std::shared_ptr
,
const std::shared_ptr
) {
auto flow_file = session->get();
-
if (!flow_file) {
return;
}
std::string template_file;
context->getProperty(Template, template_file, flow_file);
- WriteCallback cb(template_file, flow_file);
+ WriteCallback cb(template_file, *flow_file);
session->write(flow_file, );
session->transfer(flow_file, Success);
}
-ApplyTemplate::WriteCallback::WriteCallback(const std::string , const
std::shared_ptr _file) {
- template_file_ = path;
- flow_file_ = flow_file;
-}
-
-int64_t ApplyTemplate::WriteCallback::process(const
std::shared_ptr& stream) {
- logger_->log_info("ApplyTemplate reading template file from %s",
template_file_);
- boost::iostreams::mapped_file_source file(template_file_);
Review comment:
This line was crashing (segfault) when compiled with libc++, so I
removed it. I also saw a bunch of refactoring opportunities, so ApplyTemplate
has quite a few changes. As it turns out, this line was the only reason the
bustache extension required boost, so I removed that requirement and checked a
few boost-related pieces of code.
##
File path: libminifi/include/utils/OptionalUtils.h
##
@@ -43,22 +43,19 @@ struct is_optional, void> : std::true_type
{};
namespace detail {
// map implementation
template
-auto operator|(const std::optional&
[GitHub] [nifi-minifi-cpp] szaszm opened a new pull request #1264: MINIFICPP-1759 Fix linux/libc++ issues, use libc++ on ubuntu
szaszm opened a new pull request #1264: URL: https://github.com/apache/nifi-minifi-cpp/pull/1264 - Unfixed issue: OpenCV refuses to compile with libc++/C++20 - Also, removed Boost from the requirements, but didn't remove every trace of boost from the codebase, as that would be too much out of scope for my taste --- Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with MINIFICPP- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically main)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [x] If applicable, have you updated the LICENSE file? - [x] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [x] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check GitHub Actions CI results for build issues and submit an update to your PR as soon as possible. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (MINIFICPP-1756) Fix ARM compilation issues
[ https://issues.apache.org/jira/browse/MINIFICPP-1756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17493549#comment-17493549 ] Marton Szasz commented on MINIFICPP-1756: - https://github.com/szaszm/nifi-minifi-cpp/commit/4a0cc372b5a74626d1d024e6f3e872cd2dc0c809 > Fix ARM compilation issues > -- > > Key: MINIFICPP-1756 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1756 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Affects Versions: 0.11.0 > Environment: raspberry pi 3b+ 64 bit (emulated w/ qemu) >Reporter: Marton Szasz >Priority: Minor > Time Spent: 0.5h > Remaining Estimate: 0h > > - ossp-uuid doesn't recognize the aarch64-linux-gnu target, pass > --build=aarch64-linux-gnu --target=aarch64-linux-gnu to ./configure > - ossp-uuid thinks aarch64 is an invalid platform, needs patch or upgrade > - yaml-cpp lib path issue: passthrough CMAKE_INSTALL_LIBDIR > - spdlog lib path issue: make[2]: *** No rule to make target > 'thirdparty/spdlog-install/lib/aarch64-linux-gnu/libspdlog.a', needed by > 'bin/libcore-minifi.so'. Stop. (found under lib/libspdlog.a without the > target triplet) > - TimeUtilTests.cpp: no "January" on libstdc++10 (this should be a problem on > x86, too) > - StringUtilsTests.cpp:390: narrowing conversions result in -Wnarrowing error > despite no -Werror. Either because of initializer list (shouldn't be an issue > IMO since this is not direct-list-initialization) or compiler bug. > - curl lib path issue: make[2]: *** No rule to make target > 'thirdparty/curl-install/lib/aarch64-linux-gnu/libcurl.a', needed by > 'bin/libminifi-http-curl.so'. Stop. > - civetweb lib path > - rocksdb lib path > - probably all other ExternalProject libs. GNUInstallDirs > CMAKE_INSTALL_LIBDIR references seem to include the architecture triplet > after lib/, but the builds don't -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (NIFI-8492) IPLookupService not working since 1.8.0?
[
https://issues.apache.org/jira/browse/NIFI-8492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Burgess updated NIFI-8492:
---
Affects Version/s: (was: 1.11.4)
(was: 1.13.2)
Status: Patch Available (was: In Progress)
> IPLookupService not working since 1.8.0?
>
>
> Key: NIFI-8492
> URL: https://issues.apache.org/jira/browse/NIFI-8492
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Reporter: Benjamin Charron
>Assignee: Matt Burgess
>Priority: Major
> Labels: IPLookupService
> Time Spent: 10m
> Remaining Estimate: 0h
>
> IPLookupService, like its cousin GeoEnrichIP, uses maxmind/DatabaseReader.java
> However, because they are in different NARs, they each have their own
> "DatabaseReader.java
> ". GeoEnrichIP's version had a bug fixed back in 1.9.0
> (https://issues.apache.org/jira/browse/NIFI-5814), but IPLookupService did
> not.
> As far as I can tell, IPLookupService has also been broken since 1.8.0. We
> get the following error with the IPLookupService in 1.11.4:
> {code}
> Caused by: java.lang.UnsupportedOperationException: null
> at java.util.Collections$UnmodifiableMap.put(Collections.java:1457)
> at
> com.fasterxml.jackson.databind.node.ObjectNode.set(ObjectNode.java:370)
> at
> org.apache.nifi.lookup.maxmind.DatabaseReader.get(DatabaseReader.java:158)
> at
> org.apache.nifi.lookup.maxmind.DatabaseReader.city(DatabaseReader.java:194)
> at
> org.apache.nifi.lookup.maxmind.IPLookupService.doLookup(IPLookupService.java:262)
> {code}
> Should the fixes in GeoEnrichIP's DatabaseReader also be copied to
> IPLookupService's DatabaseReader? Or is there a way for them to use the same
> one without copying the file?
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [nifi] mattyb149 opened a new pull request #5777: NIFI-8492: Addressed issue in DatabaseReader class of IPLookupService…
mattyb149 opened a new pull request #5777:
URL: https://github.com/apache/nifi/pull/5777
… that was attempting to set values on the JSON returned by MaxMind. Instead
of modifying the object directly, we should use an Injectable in the Reader so
that the value read will have the appropriate values but we don't need to
modify those objects returned by MaxMind. Similar solution of NIFI-5814
Thank you for submitting a contribution to Apache NiFi.
Please provide a short description of the PR here:
Description of PR
_Enables X functionality; fixes bug NIFI-._
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [x] Does your PR title start with **NIFI-** where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [x] Has your PR been rebased against the latest commit within the target
branch (typically `main`)?
- [x] Is your initial contribution a single, squashed commit? _Additional
commits in response to PR reviewer feedback should be made on this branch and
pushed to allow change tracking. Do not `squash` or use `--force` when pushing
to allow for clean monitoring of changes._
### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via `mvn
-Pcontrib-check clean install` at the root `nifi` folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] Have you verified that the full build is successful on JDK 8?
- [x] Have you verified that the full build is successful on JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main
`LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main
`NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI for
build issues and submit an update to your PR as soon as possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Created] (MINIFICPP-1759) Use libc++ in the ubuntu clang GH Actions job
Marton Szasz created MINIFICPP-1759: --- Summary: Use libc++ in the ubuntu clang GH Actions job Key: MINIFICPP-1759 URL: https://issues.apache.org/jira/browse/MINIFICPP-1759 Project: Apache NiFi MiNiFi C++ Issue Type: Improvement Reporter: Marton Szasz -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (NIFI-9607) PutDatabaseRecord doesn't honor Update Keys when Quote Identifiers is true
[ https://issues.apache.org/jira/browse/NIFI-9607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17493524#comment-17493524 ] ASF subversion and git services commented on NIFI-9607: --- Commit 03165ad817bae0c981f82e92c5b916324da5a5bf in nifi's branch refs/heads/main from Matt Burgess [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=03165ad ] NIFI-9607: Honor Update Keys when Quoting Identifiers in PutDatabaseRecord NIFI-9607: Fixed wrong column name in WHERE clause for generateUpdate Signed-off-by: Nathan Gough This closes #5701. > PutDatabaseRecord doesn't honor Update Keys when Quote Identifiers is true > -- > > Key: NIFI-9607 > URL: https://issues.apache.org/jira/browse/NIFI-9607 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Major > Fix For: 1.16.0 > > Time Spent: 1h 20m > Remaining Estimate: 0h > > When using the Update Keys property of PutDatabaseTable, if Quote Identifiers > is set then invalid SQL (i.e empty WHERE clause) is generated so a failure > occurs. This combination should work as expected. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (NIFI-9607) PutDatabaseRecord doesn't honor Update Keys when Quote Identifiers is true
[ https://issues.apache.org/jira/browse/NIFI-9607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nathan Gough updated NIFI-9607: --- Fix Version/s: 1.16.0 Resolution: Fixed Status: Resolved (was: Patch Available) > PutDatabaseRecord doesn't honor Update Keys when Quote Identifiers is true > -- > > Key: NIFI-9607 > URL: https://issues.apache.org/jira/browse/NIFI-9607 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Major > Fix For: 1.16.0 > > Time Spent: 1h 20m > Remaining Estimate: 0h > > When using the Update Keys property of PutDatabaseTable, if Quote Identifiers > is set then invalid SQL (i.e empty WHERE clause) is generated so a failure > occurs. This combination should work as expected. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [nifi] thenatog closed pull request #5701: NIFI-9607: Honor Update Keys when Quoting Identifiers in PutDatabaseRecord
thenatog closed pull request #5701: URL: https://github.com/apache/nifi/pull/5701 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] thenatog commented on pull request #5701: NIFI-9607: Honor Update Keys when Quoting Identifiers in PutDatabaseRecord
thenatog commented on pull request #5701: URL: https://github.com/apache/nifi/pull/5701#issuecomment-1042310893 +1 will merge. If we can get more information/a specific case where the issue mentioned by mi-volodin we can create another Jira to resolve it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (NIFI-8492) IPLookupService not working since 1.8.0?
[
https://issues.apache.org/jira/browse/NIFI-8492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Burgess reassigned NIFI-8492:
--
Assignee: Matt Burgess
> IPLookupService not working since 1.8.0?
>
>
> Key: NIFI-8492
> URL: https://issues.apache.org/jira/browse/NIFI-8492
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.11.4, 1.13.2
>Reporter: Benjamin Charron
>Assignee: Matt Burgess
>Priority: Major
> Labels: IPLookupService
>
> IPLookupService, like its cousin GeoEnrichIP, uses maxmind/DatabaseReader.java
> However, because they are in different NARs, they each have their own
> "DatabaseReader.java
> ". GeoEnrichIP's version had a bug fixed back in 1.9.0
> (https://issues.apache.org/jira/browse/NIFI-5814), but IPLookupService did
> not.
> As far as I can tell, IPLookupService has also been broken since 1.8.0. We
> get the following error with the IPLookupService in 1.11.4:
> {code}
> Caused by: java.lang.UnsupportedOperationException: null
> at java.util.Collections$UnmodifiableMap.put(Collections.java:1457)
> at
> com.fasterxml.jackson.databind.node.ObjectNode.set(ObjectNode.java:370)
> at
> org.apache.nifi.lookup.maxmind.DatabaseReader.get(DatabaseReader.java:158)
> at
> org.apache.nifi.lookup.maxmind.DatabaseReader.city(DatabaseReader.java:194)
> at
> org.apache.nifi.lookup.maxmind.IPLookupService.doLookup(IPLookupService.java:262)
> {code}
> Should the fixes in GeoEnrichIP's DatabaseReader also be copied to
> IPLookupService's DatabaseReader? Or is there a way for them to use the same
> one without copying the file?
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (NIFI-9698) When creating avro schema from a RecordSchema, if array field has empty array as default, it fails
[ https://issues.apache.org/jira/browse/NIFI-9698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Sampson updated NIFI-9698: Resolution: Fixed Status: Resolved (was: Patch Available) > When creating avro schema from a RecordSchema, if array field has empty array > as default, it fails > -- > > Key: NIFI-9698 > URL: https://issues.apache.org/jira/browse/NIFI-9698 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 1.16.0 > > Attachments: flow.json > > Time Spent: 0.5h > Remaining Estimate: 0h > > Reported via Apache Slack, the attached flow results in failing on the > ScriptedTransformRecord processor when attempting to create the schema for > the Avro Writer -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (NIFI-9698) When creating avro schema from a RecordSchema, if array field has empty array as default, it fails
[ https://issues.apache.org/jira/browse/NIFI-9698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17493496#comment-17493496 ] ASF subversion and git services commented on NIFI-9698: --- Commit 4d8c79d7f326da46755aef7ea9c1d74295ab7f2f in nifi's branch refs/heads/main from Mark Payne [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=4d8c79d ] NIFI-9698: When creating an Avro schema, ensure that any default value is converted from what is returned by RecordField.getDefaultValue() to what Avro requires. Signed-off-by: Chris Sampson This closes #5776 > When creating avro schema from a RecordSchema, if array field has empty array > as default, it fails > -- > > Key: NIFI-9698 > URL: https://issues.apache.org/jira/browse/NIFI-9698 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 1.16.0 > > Attachments: flow.json > > Time Spent: 20m > Remaining Estimate: 0h > > Reported via Apache Slack, the attached flow results in failing on the > ScriptedTransformRecord processor when attempting to create the schema for > the Avro Writer -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (NIFI-9689) Do not schedule processors when all FlowFiles are penalized
[ https://issues.apache.org/jira/browse/NIFI-9689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17493495#comment-17493495 ] ASF subversion and git services commented on NIFI-9689: --- Commit 2aa6bd1e135cd4959cef5a483c454dc9a1591439 in nifi's branch refs/heads/main from Mark Payne [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=2aa6bd1 ] NIFI-9689: When all FlowFiles in a FlowFile Queue are penalized, do not schedule the destination to run. Also expose this fact via the ConnectionStatusSnapshotDTO, as this allows the front-end to render this information to the user in order to avoid confusion when it appears that the Processor has data but does nothing Signed-off-by: Matthew Burgess This closes #5771 > Do not schedule processors when all FlowFiles are penalized > --- > > Key: NIFI-9689 > URL: https://issues.apache.org/jira/browse/NIFI-9689 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 1.16.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > When all FlowFiles in a given queue are penalized, the destination processor > is still scheduled to run. As a result, the Processor is triggered, no data > is available, and it generally returns. However, this means that the UI shows > millions of tasks completed over a short period of time, when no work was > actually done. > Further, it means that we constantly schedule the processor to run as fast as > it can, which wastes compute resources. Instead, we should detect that the > head of the queue is penalized (and therefore all FlowFiles in the queue are > penalized), and treat it the same as we would if the queue were empty, which > allows the "bored yield duration" to take effect, significantly reducing CPU > cycles. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (NIFI-9689) Do not schedule processors when all FlowFiles are penalized
[ https://issues.apache.org/jira/browse/NIFI-9689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-9689: --- Fix Version/s: 1.16.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Do not schedule processors when all FlowFiles are penalized > --- > > Key: NIFI-9689 > URL: https://issues.apache.org/jira/browse/NIFI-9689 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 1.16.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > When all FlowFiles in a given queue are penalized, the destination processor > is still scheduled to run. As a result, the Processor is triggered, no data > is available, and it generally returns. However, this means that the UI shows > millions of tasks completed over a short period of time, when no work was > actually done. > Further, it means that we constantly schedule the processor to run as fast as > it can, which wastes compute resources. Instead, we should detect that the > head of the queue is penalized (and therefore all FlowFiles in the queue are > penalized), and treat it the same as we would if the queue were empty, which > allows the "bored yield duration" to take effect, significantly reducing CPU > cycles. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [nifi] asfgit closed pull request #5776: NIFI-9698: When creating an Avro schema, ensure that any default valu…
asfgit closed pull request #5776: URL: https://github.com/apache/nifi/pull/5776 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #1263: MINIFICPP-1756 fix libstdc++10 issue
szaszm commented on a change in pull request #1263:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1263#discussion_r808432656
##
File path: libminifi/test/unit/TimeUtilTests.cpp
##
@@ -98,8 +98,9 @@ TEST_CASE("Test time conversion", "[testtimeconversion]") {
TEST_CASE("Test system_clock epoch", "[systemclockepoch]") {
using namespace std::chrono;
+ using namespace date;
time_point epoch;
- time_point unix_epoch_plus_3e9_sec = sys_days(January / 24 /
2065) + 5h + 20min;
+ time_point unix_epoch_plus_3e9_sec =
date::sys_days(date::January / 24 / 2065) + 5h + 20min;
Review comment:
The issue: libstdc++10 doesn't implement month objects in std::chrono.
Version 11 has them, which is why it's passing in GH actions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #1263: MINIFICPP-1756 fix libstdc++10 issue
szaszm commented on a change in pull request #1263:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1263#discussion_r808432656
##
File path: libminifi/test/unit/TimeUtilTests.cpp
##
@@ -98,8 +98,9 @@ TEST_CASE("Test time conversion", "[testtimeconversion]") {
TEST_CASE("Test system_clock epoch", "[systemclockepoch]") {
using namespace std::chrono;
+ using namespace date;
time_point epoch;
- time_point unix_epoch_plus_3e9_sec = sys_days(January / 24 /
2065) + 5h + 20min;
+ time_point unix_epoch_plus_3e9_sec =
date::sys_days(date::January / 24 / 2065) + 5h + 20min;
Review comment:
The issue: libstdc++10 doesn't implement month objects in std::chrono
##
File path: libminifi/include/utils/TimeUtil.h
##
@@ -31,6 +31,8 @@
#include
#include
+#include "date/date.h"
Review comment:
Using transitive include because that's the way chrono itself was
included
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm opened a new pull request #1263: MINIFICPP-1756 fix libstdc++10 issue
szaszm opened a new pull request #1263: URL: https://github.com/apache/nifi-minifi-cpp/pull/1263 Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with MINIFICPP- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically main)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [x] If applicable, have you updated the LICENSE file? - [x] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [x] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check GitHub Actions CI results for build issues and submit an update to your PR as soon as possible. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-9698) When creating avro schema from a RecordSchema, if array field has empty array as default, it fails
[ https://issues.apache.org/jira/browse/NIFI-9698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17493489#comment-17493489 ] ASF subversion and git services commented on NIFI-9698: --- Commit 76ffae424401c878a89ea0319ce593c4ba4a684a in nifi's branch refs/heads/main from Mark Payne [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=76ffae4 ] NIFI-9698: When creating an Avro schema, ensure that any default value is converted from what is returned by RecordField.getDefaultValue() to what Avro requires. This closes #5776 > When creating avro schema from a RecordSchema, if array field has empty array > as default, it fails > -- > > Key: NIFI-9698 > URL: https://issues.apache.org/jira/browse/NIFI-9698 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 1.16.0 > > Attachments: flow.json > > Time Spent: 20m > Remaining Estimate: 0h > > Reported via Apache Slack, the attached flow results in failing on the > ScriptedTransformRecord processor when attempting to create the schema for > the Avro Writer -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [nifi] mattyb149 commented on pull request #5771: NIFI-9689: When all FlowFiles in a FlowFile Queue are penalized, do n…
mattyb149 commented on pull request #5771: URL: https://github.com/apache/nifi/pull/5771#issuecomment-1042236034 +1 LGTM, thanks for the improvement! Merging to main -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] mattyb149 closed pull request #5771: NIFI-9689: When all FlowFiles in a FlowFile Queue are penalized, do n…
mattyb149 closed pull request #5771: URL: https://github.com/apache/nifi/pull/5771 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] ChrisSamo632 commented on pull request #5776: NIFI-9698: When creating an Avro schema, ensure that any default valu…
ChrisSamo632 commented on pull request #5776: URL: https://github.com/apache/nifi/pull/5776#issuecomment-1042227220 > Thanks @markap14, LGTM > > Will wait for the final check to finish (restarted dur to a timeout, which I doubt was to do with these changes but worth making sure it all passes), then will look to merge if all clear Actually, looks like that Check has been timing out for most PRs recently, so appears to be an unrelated issue and not one to block merging this change -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-9689) Do not schedule processors when all FlowFiles are penalized
[ https://issues.apache.org/jira/browse/NIFI-9689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-9689: --- Status: Patch Available (was: Open) > Do not schedule processors when all FlowFiles are penalized > --- > > Key: NIFI-9689 > URL: https://issues.apache.org/jira/browse/NIFI-9689 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > When all FlowFiles in a given queue are penalized, the destination processor > is still scheduled to run. As a result, the Processor is triggered, no data > is available, and it generally returns. However, this means that the UI shows > millions of tasks completed over a short period of time, when no work was > actually done. > Further, it means that we constantly schedule the processor to run as fast as > it can, which wastes compute resources. Instead, we should detect that the > head of the queue is penalized (and therefore all FlowFiles in the queue are > penalized), and treat it the same as we would if the queue were empty, which > allows the "bored yield duration" to take effect, significantly reducing CPU > cycles. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (NIFI-9689) Do not schedule processors when all FlowFiles are penalized
[ https://issues.apache.org/jira/browse/NIFI-9689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-9689: --- Description: When all FlowFiles in a given queue are penalized, the destination processor is still scheduled to run. As a result, the Processor is triggered, no data is available, and it generally returns. However, this means that the UI shows millions of tasks completed over a short period of time, when no work was actually done. Further, it means that we constantly schedule the processor to run as fast as it can, which wastes compute resources. Instead, we should detect that the head of the queue is penalized (and therefore all FlowFiles in the queue are penalized), and treat it the same as we would if the queue were empty, which allows the "bored yield duration" to take effect, significantly reducing CPU cycles. was: When all FlowFiles in a given queue are penalized, the destination processor is still scheduled to run. As a result, the Processor is triggered, no data is available, and it generally returns. However, this means that the UI shows millions of tasks completed over a short period of time, when no where was actually done. Further, it means that we constantly schedule the processor to run as fast as it can, which wastes compute resources. Instead, we should detect that the head of the queue is penalized (and therefore all FlowFiles in the queue are penalized), and treat it the same as we would if the queue were empty, which allows the "bored yield duration" to take effect, significantly reducing CPU cycles. > Do not schedule processors when all FlowFiles are penalized > --- > > Key: NIFI-9689 > URL: https://issues.apache.org/jira/browse/NIFI-9689 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > When all FlowFiles in a given queue are penalized, the destination processor > is still scheduled to run. As a result, the Processor is triggered, no data > is available, and it generally returns. However, this means that the UI shows > millions of tasks completed over a short period of time, when no work was > actually done. > Further, it means that we constantly schedule the processor to run as fast as > it can, which wastes compute resources. Instead, we should detect that the > head of the queue is penalized (and therefore all FlowFiles in the queue are > penalized), and treat it the same as we would if the queue were empty, which > allows the "bored yield duration" to take effect, significantly reducing CPU > cycles. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (NIFI-8533) Replace jython-shaded with jython-slim because jython-shaded project is obsolete
[ https://issues.apache.org/jira/browse/NIFI-8533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-8533: --- Fix Version/s: (was: 1.16.0) > Replace jython-shaded with jython-slim because jython-shaded project is > obsolete > > > Key: NIFI-8533 > URL: https://issues.apache.org/jira/browse/NIFI-8533 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: lindongdong >Assignee: Matt Burgess >Priority: Major > Attachments: image-2021-05-12-09-30-36-729.png > > Time Spent: 10m > Remaining Estimate: 0h > > As jython-shaded is deparecated, so I suggest to replace jython-shaded with > jython-slim > [https://github.com/scijava/jython-shaded] > !image-2021-05-12-09-30-36-729.png|width=589,height=311! -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [nifi] mattyb149 commented on pull request #5701: NIFI-9607: Honor Update Keys when Quoting Identifiers in PutDatabaseRecord
mattyb149 commented on pull request #5701: URL: https://github.com/apache/nifi/pull/5701#issuecomment-1042106843 @mi-volodin I didn't see the same behavior you describe, but did find a bug. I debugged the unit test and `normalizeKeyColumnNamesAndCheckForValues` doesn't quote the column names, just normalizes it by uppercasing it and removing underscores and such. Also the normalized column name it matches against isn't quoted, it should only be quoted when generating the SQL. However it was using `normalizedColumnName` as the column name in the `WHERE` clause when it should be using `desc.getColumnName()` like the `SET` clause does. That can cause issues, I will issue another commit to update the unit test to illustrate this, along with the fix to get the test to pass :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on a change in pull request #5732: NIFI-9647 Added ExtractDocumentText which is a processor based on Tim…
exceptionfactory commented on a change in pull request #5732:
URL: https://github.com/apache/nifi/pull/5732#discussion_r808401004
##
File path:
nifi-nar-bundles/nifi-media-bundle/nifi-media-processors/src/test/java/org/apache/nifi/processors/document/ExtractDocumentTextTest.java
##
@@ -0,0 +1,256 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.document;
+
+import org.apache.nifi.util.MockFlowFile;
+import org.apache.nifi.util.TestRunner;
+import org.apache.nifi.util.TestRunners;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+
+import java.io.FileInputStream;
+import java.io.FileNotFoundException;
+import java.io.UnsupportedEncodingException;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+
+public class ExtractDocumentTextTest {
+
+private TestRunner testRunner;
+
+@BeforeEach
+public void init() {
+testRunner = TestRunners.newTestRunner(ExtractDocumentText.class);
+}
+
+@Test
+public void processor_should_support_pdf_types_without_exception() {
+try {
+final String filename = "simple.pdf";
+MockFlowFile flowFile = testRunner.enqueue(new
FileInputStream("src/test/resources/" + filename));
+Map attrs = new HashMap() {{
put("filename", filename); }};
+flowFile.putAttributes(attrs);
+} catch (FileNotFoundException e) {
+e.printStackTrace();
+}
+
+testRunner.assertValid();
+testRunner.run();
+testRunner.assertTransferCount(ExtractDocumentText.REL_FAILURE, 0);
+
+List successFiles =
testRunner.getFlowFilesForRelationship(ExtractDocumentText.REL_SUCCESS);
+for (MockFlowFile mockFile : successFiles) {
+try {
+String result = new String(mockFile.toByteArray(), "UTF-8");
+String trimmedResult = result.trim();
+assertTrue(trimmedResult.startsWith("A Simple PDF File"));
+System.out.println("FILE:" + result);
+} catch (UnsupportedEncodingException e) {
+e.printStackTrace();
+}
+}
+}
+
+@Test
+public void processor_should_support_doc_types_without_exception() {
+try {
+final String filename = "simple.doc";
Review comment:
That's fair, perhaps reducing to a single binary document would be a
sufficient way to go.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Assigned] (NIFI-9699) NullPointerException at org.apache.nifi.web.api.OIDCAccessResource.oidcCallback(OIDCAccessResource.java:186)
[
https://issues.apache.org/jira/browse/NIFI-9699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nathan Gough reassigned NIFI-9699:
--
Assignee: Nathan Gough
> NullPointerException at
> org.apache.nifi.web.api.OIDCAccessResource.oidcCallback(OIDCAccessResource.java:186)
>
>
> Key: NIFI-9699
> URL: https://issues.apache.org/jira/browse/NIFI-9699
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.15.3
>Reporter: Hans Deragon
>Assignee: Nathan Gough
>Priority: Minor
>
> Using [Okta |https://www.okta.com/]as an OpenID provider, sometimes the error
> below occurs. Looking at the code of
> [OIDCAccessResource.java|https://github.com/apache/nifi/blob/main/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/OIDCAccessResource.java]
> line 186, seams that the response returned or its description is Null. Need
> to handle it properly.
> *Code snippet of
> [OIDCAccessResource.java|https://github.com/apache/nifi/blob/main/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/OIDCAccessResource.java]
> line 186:*
> {code:java}
> // report the unsuccessful login
> final AuthenticationErrorResponse errorOidcResponse
> =(AuthenticationErrorResponse) oidcResponse;
> forwardToLoginMessagePage(httpServletRequest, httpServletResponse,
> "Unsuccessful login attempt: "+
> errorOidcResponse.getErrorObject().getDescription()); // Not catching
> NULL here.{code}
> *Error with full stacktrace:*
> {code:java}
> 2022-02-16 12:00:34,692 ERROR [NiFi Web Server-38]
> o.a.nifi.web.api.config.ThrowableMapper An unexpected error has occurred:
> java.lang.NullPointerException. Returning Internal Server Error response.
>
>
>java.lang.NullPointerException: null
>
> at
> org.apache.nifi.web.api.OIDCAccessResource.oidcCallback(OIDCAccessResource.java:186)
>
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
>
> at
> org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52)
> at
> org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124)
> at
> org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167)
> at
> org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$VoidOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:159)
> at
> org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79)
> at
> org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475)
>
> at
> org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397)
>
>at
> org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81)
>
[GitHub] [nifi] MikeThomsen commented on a change in pull request #5732: NIFI-9647 Added ExtractDocumentText which is a processor based on Tim…
MikeThomsen commented on a change in pull request #5732:
URL: https://github.com/apache/nifi/pull/5732#discussion_r808384119
##
File path:
nifi-nar-bundles/nifi-media-bundle/nifi-media-processors/src/test/java/org/apache/nifi/processors/document/ExtractDocumentTextTest.java
##
@@ -0,0 +1,256 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.document;
+
+import org.apache.nifi.util.MockFlowFile;
+import org.apache.nifi.util.TestRunner;
+import org.apache.nifi.util.TestRunners;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+
+import java.io.FileInputStream;
+import java.io.FileNotFoundException;
+import java.io.UnsupportedEncodingException;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+
+public class ExtractDocumentTextTest {
+
+private TestRunner testRunner;
+
+@BeforeEach
+public void init() {
+testRunner = TestRunners.newTestRunner(ExtractDocumentText.class);
+}
+
+@Test
+public void processor_should_support_pdf_types_without_exception() {
+try {
+final String filename = "simple.pdf";
+MockFlowFile flowFile = testRunner.enqueue(new
FileInputStream("src/test/resources/" + filename));
+Map attrs = new HashMap() {{
put("filename", filename); }};
+flowFile.putAttributes(attrs);
+} catch (FileNotFoundException e) {
+e.printStackTrace();
+}
+
+testRunner.assertValid();
+testRunner.run();
+testRunner.assertTransferCount(ExtractDocumentText.REL_FAILURE, 0);
+
+List successFiles =
testRunner.getFlowFilesForRelationship(ExtractDocumentText.REL_SUCCESS);
+for (MockFlowFile mockFile : successFiles) {
+try {
+String result = new String(mockFile.toByteArray(), "UTF-8");
+String trimmedResult = result.trim();
+assertTrue(trimmedResult.startsWith("A Simple PDF File"));
+System.out.println("FILE:" + result);
+} catch (UnsupportedEncodingException e) {
+e.printStackTrace();
+}
+}
+}
+
+@Test
+public void processor_should_support_doc_types_without_exception() {
+try {
+final String filename = "simple.doc";
Review comment:
I'm going to skip this one. I think this provides a really good and
cheap way to do a meaningful functional test of the processor for anyone who
has to step in and fix it or update it later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (NIFI-8492) IPLookupService not working since 1.8.0?
[
https://issues.apache.org/jira/browse/NIFI-8492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17493410#comment-17493410
]
Joe Witt commented on NIFI-8492:
ooof yeah I'm sure someone will get to this one. Not sure how it has been able
to be broken for so long or how we dont have a test that exposes it
> IPLookupService not working since 1.8.0?
>
>
> Key: NIFI-8492
> URL: https://issues.apache.org/jira/browse/NIFI-8492
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.11.4, 1.13.2
>Reporter: Benjamin Charron
>Priority: Major
> Labels: IPLookupService
>
> IPLookupService, like its cousin GeoEnrichIP, uses maxmind/DatabaseReader.java
> However, because they are in different NARs, they each have their own
> "DatabaseReader.java
> ". GeoEnrichIP's version had a bug fixed back in 1.9.0
> (https://issues.apache.org/jira/browse/NIFI-5814), but IPLookupService did
> not.
> As far as I can tell, IPLookupService has also been broken since 1.8.0. We
> get the following error with the IPLookupService in 1.11.4:
> {code}
> Caused by: java.lang.UnsupportedOperationException: null
> at java.util.Collections$UnmodifiableMap.put(Collections.java:1457)
> at
> com.fasterxml.jackson.databind.node.ObjectNode.set(ObjectNode.java:370)
> at
> org.apache.nifi.lookup.maxmind.DatabaseReader.get(DatabaseReader.java:158)
> at
> org.apache.nifi.lookup.maxmind.DatabaseReader.city(DatabaseReader.java:194)
> at
> org.apache.nifi.lookup.maxmind.IPLookupService.doLookup(IPLookupService.java:262)
> {code}
> Should the fixes in GeoEnrichIP's DatabaseReader also be copied to
> IPLookupService's DatabaseReader? Or is there a way for them to use the same
> one without copying the file?
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (NIFI-9620) Add isStateful() method to Stateless DataFlow API
[ https://issues.apache.org/jira/browse/NIFI-9620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17493406#comment-17493406 ] ASF subversion and git services commented on NIFI-9620: --- Commit 4210e30047b7d7db17375caaba70d3465450d423 in nifi's branch refs/heads/main from Joe Gresock [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=4210e30 ] NIFI-9620: Adding isStateful to StatelessDataflow (#5703) * Adding isStateless method to reporting task and controller service > Add isStateful() method to Stateless DataFlow API > - > > Key: NIFI-9620 > URL: https://issues.apache.org/jira/browse/NIFI-9620 > Project: Apache NiFi > Issue Type: Improvement > Components: NiFi Stateless >Reporter: Joe Gresock >Priority: Major > Time Spent: 50m > Remaining Estimate: 0h > > Stateless NiFi's DataFlow interface could have a isStateful() method, which > returns true if any processors in the flow maintain state (by checking for > the presence of the @Stateful annotation). -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [nifi] markap14 commented on pull request #5703: NIFI-9620: Adding isStateful to StatelessDataflow
markap14 commented on pull request #5703: URL: https://github.com/apache/nifi/pull/5703#issuecomment-1041994403 Thanks @gresockj all good from my end. +1 merged to main -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] markap14 merged pull request #5703: NIFI-9620: Adding isStateful to StatelessDataflow
markap14 merged pull request #5703: URL: https://github.com/apache/nifi/pull/5703 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] markap14 commented on pull request #5745: NIFI-9492: Fix for Chinese VPCE - Tests included
markap14 commented on pull request #5745: URL: https://github.com/apache/nifi/pull/5745#issuecomment-1041987628 Thanks for the contribution @jynolen . I wonder, though, if there are other VPCE endpoints that may need to be used, other than just `.com` and `.com.cn`? We probably should not just slowly add them one at a time. Instead, perhaps it makes sense to check if the URL ends with ".vpce.amazonaws.com" followed by `.` and if so extract that whole suffix and pass that on. Thoughts? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] markap14 commented on pull request #5767: NIFI-9694 : Correct FetchS3Object.java for provenance data URL
markap14 commented on pull request #5767: URL: https://github.com/apache/nifi/pull/5767#issuecomment-1041975795 @lawye1973 looks like this pull request is based on the `master` branch. We need a Pull Request that is based off of the `main` branch - `master` was deprecated a few years ago. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (NIFI-9687) Add additional documentation for setting nifi.cluster.node.protocol.max.threads property
[ https://issues.apache.org/jira/browse/NIFI-9687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne resolved NIFI-9687. -- Fix Version/s: 1.16.0 Resolution: Fixed > Add additional documentation for setting > nifi.cluster.node.protocol.max.threads property > > > Key: NIFI-9687 > URL: https://issues.apache.org/jira/browse/NIFI-9687 > Project: Apache NiFi > Issue Type: Improvement > Components: Documentation Website >Reporter: Andrew M. Lim >Assignee: Andrew M. Lim >Priority: Major > Fix For: 1.16.0 > > Time Spent: 40m > Remaining Estimate: 0h > > Current documentation in Admin Guide does not help the user set a value for > this property. Will add: > When a request is made to one node, it must be forwarded to the coordinator. > The coordinator then replicates it to all nodes. There could be up to n+2 > threads for a given request, where n = number of nodes in your cluster. As an > example, if 4 requests are made, a 5 node cluster will use 4 * 7 = 28 threads. > to these existing sections: > https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#basic-cluster-setup > https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#cluster_node_properties -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (NIFI-9687) Add additional documentation for setting nifi.cluster.node.protocol.max.threads property
[ https://issues.apache.org/jira/browse/NIFI-9687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17493400#comment-17493400 ] ASF subversion and git services commented on NIFI-9687: --- Commit 6ad28046c3b4fa6cecb40088c42c95ff939be74f in nifi's branch refs/heads/main from Andrew Lim [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=6ad2804 ] NIFI-9687 Add additional documentation for nifi.cluster.node.protocol.max.threads property to Admin Guide (#5770) > Add additional documentation for setting > nifi.cluster.node.protocol.max.threads property > > > Key: NIFI-9687 > URL: https://issues.apache.org/jira/browse/NIFI-9687 > Project: Apache NiFi > Issue Type: Improvement > Components: Documentation Website >Reporter: Andrew M. Lim >Assignee: Andrew M. Lim >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > Current documentation in Admin Guide does not help the user set a value for > this property. Will add: > When a request is made to one node, it must be forwarded to the coordinator. > The coordinator then replicates it to all nodes. There could be up to n+2 > threads for a given request, where n = number of nodes in your cluster. As an > example, if 4 requests are made, a 5 node cluster will use 4 * 7 = 28 threads. > to these existing sections: > https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#basic-cluster-setup > https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#cluster_node_properties -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [nifi] markap14 merged pull request #5770: NIFI-9687 Add additional documentation for nifi.cluster.node.protocol…
markap14 merged pull request #5770: URL: https://github.com/apache/nifi/pull/5770 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] markap14 commented on pull request #5770: NIFI-9687 Add additional documentation for nifi.cluster.node.protocol…
markap14 commented on pull request #5770: URL: https://github.com/apache/nifi/pull/5770#issuecomment-1041974025 Thanks @andrewmlim merged to main. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] markap14 commented on pull request #5774: NIFI-9696 : DeleteS3Object don't have provenance
markap14 commented on pull request #5774: URL: https://github.com/apache/nifi/pull/5774#issuecomment-1041971783 @lawye1973 this PR is also set to merge into the `master` branch. But that branch hasn't been used in a long time. Please create with an upstream branch of `main`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-9700) NiFi Stateless Kafka connectors should not redefine 'name' property
Peter Turcsanyi created NIFI-9700: - Summary: NiFi Stateless Kafka connectors should not redefine 'name' property Key: NIFI-9700 URL: https://issues.apache.org/jira/browse/NIFI-9700 Project: Apache NiFi Issue Type: Improvement Reporter: Peter Turcsanyi Assignee: Peter Turcsanyi The connector's 'name' is a standard property available for all connectors by default (tested on Kafka 2.8.1). It is not needed to explicitly (re)define it so the following line should be removed: https://github.com/apache/nifi/blob/fe1139b8bb58acab3fa9a59dd64669848caf480a/nifi-external/nifi-kafka-connect/nifi-kafka-connector/src/main/java/org/apache/nifi/kafka/connect/StatelessKafkaConnectorUtil.java#L103 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (NIFI-9699) NullPointerException at org.apache.nifi.web.api.OIDCAccessResource.oidcCallback(OIDCAccessResource.java:186)
Hans Deragon created NIFI-9699:
--
Summary: NullPointerException at
org.apache.nifi.web.api.OIDCAccessResource.oidcCallback(OIDCAccessResource.java:186)
Key: NIFI-9699
URL: https://issues.apache.org/jira/browse/NIFI-9699
Project: Apache NiFi
Issue Type: Bug
Components: Core Framework
Affects Versions: 1.15.3
Reporter: Hans Deragon
Using [Okta |https://www.okta.com/]as an OpenID provider, sometimes the error
below occurs. Looking at the code of
[OIDCAccessResource.java|https://github.com/apache/nifi/blob/main/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/OIDCAccessResource.java]
line 186, seams that the response returned or its description is Null. Need
to handle it properly.
*Code snippet of
[OIDCAccessResource.java|https://github.com/apache/nifi/blob/main/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/OIDCAccessResource.java]
line 186:*
{code:java}
// report the unsuccessful login
final AuthenticationErrorResponse errorOidcResponse
=(AuthenticationErrorResponse) oidcResponse;
forwardToLoginMessagePage(httpServletRequest, httpServletResponse,
"Unsuccessful login attempt: "+
errorOidcResponse.getErrorObject().getDescription()); // Not catching
NULL here.{code}
*Error with full stacktrace:*
{code:java}
2022-02-16 12:00:34,692 ERROR [NiFi Web Server-38]
o.a.nifi.web.api.config.ThrowableMapper An unexpected error has occurred:
java.lang.NullPointerException. Returning Internal Server Error response.
java.lang.NullPointerException: null
at
org.apache.nifi.web.api.OIDCAccessResource.oidcCallback(OIDCAccessResource.java:186)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at
org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52)
at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124)
at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167)
at
org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$VoidOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:159)
at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79)
at
org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475)
at
org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397)
at
org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81)
at
org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255)
at
org.glassfish.jersey.internal.Errors$1.call(Errors.java:248)
at
[GitHub] [nifi] markap14 commented on pull request #5247: NIFI-8927 - Add option to start/stop all controllers
markap14 commented on pull request #5247: URL: https://github.com/apache/nifi/pull/5247#issuecomment-1041928487 Thanks for your thoughts @joewitt and @mcgilman. Here's my 2 cents: - I'm not the one to review the code because my JavaScript skills are non-existent, so I can't review from that perspective. - The calls that the UI makes are to the existing endpoints, which asynchronously enable/disable controller services. So I don't think we have concerns about high load causing timeouts, failures, etc. because it shouldn't block - I do agree with mcgilman that I'd like to see a UI that allows the controller services to be visualized with all referencing components so that users know exactly what they are doing when they Disable All. And when enabling, ideally it would be helpful to be able to choose to start referencing components as well. - That said, I think this feature is extremely useful as-is. The more powerful UI capability can still be provided in a future release of NiFi, but I think this is significantly more helpful than what we have currently. So while I'm not in a position to review the code I'm a +1 personally. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-9698) When creating avro schema from a RecordSchema, if array field has empty array as default, it fails
[ https://issues.apache.org/jira/browse/NIFI-9698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-9698: - Fix Version/s: 1.16.0 > When creating avro schema from a RecordSchema, if array field has empty array > as default, it fails > -- > > Key: NIFI-9698 > URL: https://issues.apache.org/jira/browse/NIFI-9698 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 1.16.0 > > Attachments: flow.json > > Time Spent: 10m > Remaining Estimate: 0h > > Reported via Apache Slack, the attached flow results in failing on the > ScriptedTransformRecord processor when attempting to create the schema for > the Avro Writer -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (NIFI-9698) When creating avro schema from a RecordSchema, if array field has empty array as default, it fails
[
https://issues.apache.org/jira/browse/NIFI-9698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17493353#comment-17493353
]
Mark Payne commented on NIFI-9698:
--
Of note, with this fix, the attached flow still fails in
ScriptedTransformRecord because it attempts to set an `int` field to an empty
array. This can be addressed by updating the ScriptedTransformRecord to use:
{code:java}
user_data["external_id"] = 123{code}
instead of:
{code:java}
user_data["external_id"] = record.getValue("a") {code}
> When creating avro schema from a RecordSchema, if array field has empty array
> as default, it fails
> --
>
> Key: NIFI-9698
> URL: https://issues.apache.org/jira/browse/NIFI-9698
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
> Attachments: flow.json
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Reported via Apache Slack, the attached flow results in failing on the
> ScriptedTransformRecord processor when attempting to create the schema for
> the Avro Writer
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (NIFI-9698) When creating avro schema from a RecordSchema, if array field has empty array as default, it fails
[ https://issues.apache.org/jira/browse/NIFI-9698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-9698: - Status: Patch Available (was: Open) > When creating avro schema from a RecordSchema, if array field has empty array > as default, it fails > -- > > Key: NIFI-9698 > URL: https://issues.apache.org/jira/browse/NIFI-9698 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Attachments: flow.json > > > Reported via Apache Slack, the attached flow results in failing on the > ScriptedTransformRecord processor when attempting to create the schema for > the Avro Writer -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [nifi] markap14 opened a new pull request #5776: NIFI-9698: When creating an Avro schema, ensure that any default valu…
markap14 opened a new pull request #5776:
URL: https://github.com/apache/nifi/pull/5776
…e is converted from what is returned by RecordField.getDefaultValue() to
what Avro requires.
Thank you for submitting a contribution to Apache NiFi.
Please provide a short description of the PR here:
Description of PR
_Enables X functionality; fixes bug NIFI-._
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [ ] Does your PR title start with **NIFI-** where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [ ] Has your PR been rebased against the latest commit within the target
branch (typically `main`)?
- [ ] Is your initial contribution a single, squashed commit? _Additional
commits in response to PR reviewer feedback should be made on this branch and
pushed to allow change tracking. Do not `squash` or use `--force` when pushing
to allow for clean monitoring of changes._
### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via `mvn
-Pcontrib-check clean install` at the root `nifi` folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] Have you verified that the full build is successful on JDK 8?
- [ ] Have you verified that the full build is successful on JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main
`LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main
`NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI for
build issues and submit an update to your PR as soon as possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (NIFI-9698) When creating avro schema from a RecordSchema, if array field has empty array as default, it fails
[ https://issues.apache.org/jira/browse/NIFI-9698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-9698: - Attachment: flow.json > When creating avro schema from a RecordSchema, if array field has empty array > as default, it fails > -- > > Key: NIFI-9698 > URL: https://issues.apache.org/jira/browse/NIFI-9698 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Attachments: flow.json > > > Reported via Apache Slack, the attached template results in failing on the > ScriptedTransformRecord processor when attempting to create the schema for > the Avro Writer -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (NIFI-9698) When creating avro schema from a RecordSchema, if array field has empty array as default, it fails
Mark Payne created NIFI-9698: Summary: When creating avro schema from a RecordSchema, if array field has empty array as default, it fails Key: NIFI-9698 URL: https://issues.apache.org/jira/browse/NIFI-9698 Project: Apache NiFi Issue Type: Bug Components: Extensions Reporter: Mark Payne Assignee: Mark Payne Attachments: flow.json Reported via Apache Slack, the attached template results in failing on the ScriptedTransformRecord processor when attempting to create the schema for the Avro Writer -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (NIFI-9698) When creating avro schema from a RecordSchema, if array field has empty array as default, it fails
[ https://issues.apache.org/jira/browse/NIFI-9698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-9698: - Description: Reported via Apache Slack, the attached flow results in failing on the ScriptedTransformRecord processor when attempting to create the schema for the Avro Writer (was: Reported via Apache Slack, the attached template results in failing on the ScriptedTransformRecord processor when attempting to create the schema for the Avro Writer) > When creating avro schema from a RecordSchema, if array field has empty array > as default, it fails > -- > > Key: NIFI-9698 > URL: https://issues.apache.org/jira/browse/NIFI-9698 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Attachments: flow.json > > > Reported via Apache Slack, the attached flow results in failing on the > ScriptedTransformRecord processor when attempting to create the schema for > the Avro Writer -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [nifi] Lehel44 commented on pull request #5752: NIFI-9657 Create MoveADLS processor
Lehel44 commented on pull request #5752: URL: https://github.com/apache/nifi/pull/5752#issuecomment-1041836918 Thanks @timeabarna LGTM+1 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-8492) IPLookupService not working since 1.8.0?
[
https://issues.apache.org/jira/browse/NIFI-8492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17493263#comment-17493263
]
Fabian Reiber commented on NIFI-8492:
-
I see the same error in the logs, if I enable it for the LookupRecord. Is there
any plan when it will be fixed? Maybe in the next release 1.16.0, if there is
already a fix in the other component. I am using Nifi 1.15.2 atm..
> IPLookupService not working since 1.8.0?
>
>
> Key: NIFI-8492
> URL: https://issues.apache.org/jira/browse/NIFI-8492
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.11.4, 1.13.2
>Reporter: Benjamin Charron
>Priority: Major
> Labels: IPLookupService
>
> IPLookupService, like its cousin GeoEnrichIP, uses maxmind/DatabaseReader.java
> However, because they are in different NARs, they each have their own
> "DatabaseReader.java
> ". GeoEnrichIP's version had a bug fixed back in 1.9.0
> (https://issues.apache.org/jira/browse/NIFI-5814), but IPLookupService did
> not.
> As far as I can tell, IPLookupService has also been broken since 1.8.0. We
> get the following error with the IPLookupService in 1.11.4:
> {code}
> Caused by: java.lang.UnsupportedOperationException: null
> at java.util.Collections$UnmodifiableMap.put(Collections.java:1457)
> at
> com.fasterxml.jackson.databind.node.ObjectNode.set(ObjectNode.java:370)
> at
> org.apache.nifi.lookup.maxmind.DatabaseReader.get(DatabaseReader.java:158)
> at
> org.apache.nifi.lookup.maxmind.DatabaseReader.city(DatabaseReader.java:194)
> at
> org.apache.nifi.lookup.maxmind.IPLookupService.doLookup(IPLookupService.java:262)
> {code}
> Should the fixes in GeoEnrichIP's DatabaseReader also be copied to
> IPLookupService's DatabaseReader? Or is there a way for them to use the same
> one without copying the file?
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Assigned] (NIFI-9690) UI - Update Connection Label to show when head of queue is penalized
[ https://issues.apache.org/jira/browse/NIFI-9690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Kocsis reassigned NIFI-9690: - Assignee: Adam Kocsis (was: Matt Gilman) > UI - Update Connection Label to show when head of queue is penalized > > > Key: NIFI-9690 > URL: https://issues.apache.org/jira/browse/NIFI-9690 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Reporter: Matt Gilman >Assignee: Adam Kocsis >Priority: Major > > Update the rendering of the Connection Label to indicate when the FlowFile at > the head of the queue is penalized. Without this indicator, it may be > confusing why the downstream Processor is not polling the FlowFile from the > queue. In NIFI-9689 the Connection response payload will include a new enum > that will describe the FlowFile availability. This field is > HEAD_OF_QUEUE_PENALIZED we should indicate this on the Connection label in > the row detailing the current queue size. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (NIFI-9692) Upgrade Apache Commons Lang3 to 3.12.0
[ https://issues.apache.org/jira/browse/NIFI-9692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17493253#comment-17493253 ] ASF subversion and git services commented on NIFI-9692: --- Commit fe1139b8bb58acab3fa9a59dd64669848caf480a in nifi's branch refs/heads/main from David Handermann [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=fe1139b ] NIFI-9692 Upgraded Apache Commons Lang3 to 3.12.0 - Moved commons-lang3 version management to root Maven configuration - Refactored limited usage of StringUtils is nifi-reporting-utils to remove commons-lang3 - Refactored limited usage of StringUtils in nifi-websocket-processors to remove commons-lang3 Signed-off-by: Matthew Burgess This closes #5773 > Upgrade Apache Commons Lang3 to 3.12.0 > -- > > Key: NIFI-9692 > URL: https://issues.apache.org/jira/browse/NIFI-9692 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Time Spent: 0.5h > Remaining Estimate: 0h > > A large number of components rely on various versions of Apache Commons Lang3 > from version 3.9 to version 3.12.0. These individual version references > should be replaced with a managed dependency version of 3.12.0 for all > components. Several NAR modules also contain copies of Commons Lang3 that > could be marked as provided and inherited from parent NAR modules to avoid > unnecessary binary duplication. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (NIFI-9692) Upgrade Apache Commons Lang3 to 3.12.0
[ https://issues.apache.org/jira/browse/NIFI-9692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-9692: --- Fix Version/s: 1.16.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Upgrade Apache Commons Lang3 to 3.12.0 > -- > > Key: NIFI-9692 > URL: https://issues.apache.org/jira/browse/NIFI-9692 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Fix For: 1.16.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > A large number of components rely on various versions of Apache Commons Lang3 > from version 3.9 to version 3.12.0. These individual version references > should be replaced with a managed dependency version of 3.12.0 for all > components. Several NAR modules also contain copies of Commons Lang3 that > could be marked as provided and inherited from parent NAR modules to avoid > unnecessary binary duplication. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [nifi] mattyb149 closed pull request #5773: NIFI-9692 Upgrade Apache Commons Lang3 to 3.12.0
mattyb149 closed pull request #5773: URL: https://github.com/apache/nifi/pull/5773 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] mattyb149 commented on pull request #5773: NIFI-9692 Upgrade Apache Commons Lang3 to 3.12.0
mattyb149 commented on pull request #5773: URL: https://github.com/apache/nifi/pull/5773#issuecomment-1041548594 +1 LGTM, thanks for the upgrade and improvements! Merging to main -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on a change in pull request #5301: NIFI-8209 Added Neo4J 4.X and 3.X clients by splitting the current co…
exceptionfactory commented on a change in pull request #5301:
URL: https://github.com/apache/nifi/pull/5301#discussion_r807997553
##
File path:
nifi-nar-bundles/nifi-graph-bundle/nifi-neo4j-3x-cypher-service/src/test/java/org/apache/nifi/graph/MockProcessor.java
##
@@ -0,0 +1,47 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.graph;
+
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.exception.ProcessException;
+
+import java.util.Arrays;
+import java.util.List;
+
+public class MockProcessor extends AbstractProcessor {
Review comment:
This class can be removed and replaced with `NoOpProcessor` from
`nifi-mock`.
##
File path:
nifi-nar-bundles/nifi-graph-bundle/nifi-neo4j-3x-cypher-service/src/main/java/org/apache/nifi/graph/Neo4JCypher3xClientService.java
##
@@ -0,0 +1,318 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.graph;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnDisabled;
+import org.apache.nifi.annotation.lifecycle.OnEnabled;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.controller.AbstractControllerService;
+import org.apache.nifi.controller.ConfigurationContext;
+import org.apache.nifi.expression.ExpressionLanguageScope;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.ssl.SSLContextService;
+import org.neo4j.driver.internal.InternalNode;
+import org.neo4j.driver.v1.AuthTokens;
+import org.neo4j.driver.v1.Config;
+import org.neo4j.driver.v1.Driver;
+import org.neo4j.driver.v1.GraphDatabase;
+import org.neo4j.driver.v1.Record;
+import org.neo4j.driver.v1.Session;
+import org.neo4j.driver.v1.StatementResult;
+import org.neo4j.driver.v1.summary.ResultSummary;
+import org.neo4j.driver.v1.summary.SummaryCounters;
+
+import java.io.File;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.TimeUnit;
+
+@Deprecated
+@Tags({ "graph", "neo4j", "cypher" })
+@CapabilityDescription("Provides a client service for managing connections to
a Neo4J 3.X database. Configuration information for " +
+"the Neo4J driver that corresponds to most of the settings for this
service can be found here: " +
+
"https://neo4j.com/docs/driver-manual/current/client-applications/#driver-configuration;)
+public class Neo4JCypher3xClientService extends AbstractControllerService
implements GraphClientService {
+public static final PropertyDescriptor CONNECTION_URL = new
PropertyDescriptor.Builder()
+.name("neo4j-connection-url")
+.displayName("Neo4j Connection URL")
+.description("Neo4J endpoing to connect to.")
+.required(true)
+.defaultValue("bolt://localhost:7687")
+
.expressionLanguageSupported(ExpressionLanguageScope.VARIABLE_REGISTRY)
+
[GitHub] [nifi] nuttybrewer opened a new pull request #5775: NIFI-8641: AWS WebIdentity for Nifi
nuttybrewer opened a new pull request #5775:
URL: https://github.com/apache/nifi/pull/5775
Thank you for submitting a contribution to Apache NiFi.
Please provide a short description of the PR here:
Description of PR
_Enables X functionality; fixes bug NIFI-._
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [ ] Does your PR title start with **NIFI-** where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [ ] Has your PR been rebased against the latest commit within the target
branch (typically `main`)?
- [ ] Is your initial contribution a single, squashed commit? _Additional
commits in response to PR reviewer feedback should be made on this branch and
pushed to allow change tracking. Do not `squash` or use `--force` when pushing
to allow for clean monitoring of changes._
### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via `mvn
-Pcontrib-check clean install` at the root `nifi` folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] Have you verified that the full build is successful on JDK 8?
- [ ] Have you verified that the full build is successful on JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main
`LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main
`NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI for
build issues and submit an update to your PR as soon as possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] nuttybrewer commented on pull request #5768: NIFI-8641: AWS WebIdentity for Nifi
nuttybrewer commented on pull request #5768: URL: https://github.com/apache/nifi/pull/5768#issuecomment-1041427582 Need to fix some tests, will resubmit -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] nuttybrewer closed pull request #5768: NIFI-8641: AWS WebIdentity for Nifi
nuttybrewer closed pull request #5768: URL: https://github.com/apache/nifi/pull/5768 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] lawye1973 commented on pull request #5774: NIFI-9696 : DeleteS3Object don't have provenance
lawye1973 commented on pull request #5774: URL: https://github.com/apache/nifi/pull/5774#issuecomment-1041401598 > @lawye1973 Thank you for implementing this missing feature! > > `SEND` event is for indicating a provenance event for sending data to an external system. Please use `REMOTE_INVOCATION` instead, like in case of [DeleteHDFS](https://github.com/apache/nifi/blob/67880765365e033d0d772f5c7b5c2e2186a1086c/nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/DeleteHDFS.java#L179) or [DeleteAzureDatalakeStorege](https://github.com/apache/nifi/blob/67880765365e033d0d772f5c7b5c2e2186a1086c/nifi-nar-bundles/nifi-azure-bundle/nifi-azure-processors/src/main/java/org/apache/nifi/processors/azure/storage/DeleteAzureDataLakeStorage.java#L104). Thanks! @turcsanyip , thanks for the guidance. I'd updated the code. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-9696) DeleteS3Object do not have provenance data
[ https://issues.apache.org/jira/browse/NIFI-9696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17493186#comment-17493186 ] Peter Turcsanyi commented on NIFI-9696: --- [~lawye1973] In order to assign the jira to yourself (and your PR to be merged in the end), you need to request "Jira contributor access" as described in the [Contributor Guide|https://cwiki.apache.org/confluence/display/NIFI/Contributor+Guide]. Please follow those steps and then assign the jira to yourself. Thanks! > DeleteS3Object do not have provenance data > -- > > Key: NIFI-9696 > URL: https://issues.apache.org/jira/browse/NIFI-9696 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Law Yaw Eng >Priority: Trivial > Time Spent: 10m > Remaining Estimate: 0h > > DeleteS3Object processor do not have any provenance data. While it's true > that it doesn't change any flowfile or attribute, it does an external action. > > Perhaps a SEND provenance data with transitUri value of "DELETE " will > be suitable -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (NIFI-9697) Azure Key Vault Parameter Provider
Joe Gresock created NIFI-9697: - Summary: Azure Key Vault Parameter Provider Key: NIFI-9697 URL: https://issues.apache.org/jira/browse/NIFI-9697 Project: Apache NiFi Issue Type: Improvement Components: Extensions Reporter: Joe Gresock Once NIFI-9003 is complete, implement an Azure Key Vault Parameter Provider. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [nifi] timeabarna commented on pull request #5752: NIFI-9657 Create MoveADLS processor
timeabarna commented on pull request #5752: URL: https://github.com/apache/nifi/pull/5752#issuecomment-1041380972 Thanks @Lehel44 for your review. I've modified the code based on our discussion. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] lawye1973 opened a new pull request #5774: NIFI-9696 : DeleteS3Object don't have provenance
lawye1973 opened a new pull request #5774:
URL: https://github.com/apache/nifi/pull/5774
Changed from AmazonS3 to AmazonS3client.
Add SEND provenance event with transitUri value of "DELETE "
Thank you for submitting a contribution to Apache NiFi.
Please provide a short description of the PR here:
Description of PR
_Enables X functionality; fixes bug NIFI-._
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [ ] Does your PR title start with **NIFI-** where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [ ] Has your PR been rebased against the latest commit within the target
branch (typically `main`)?
- [ ] Is your initial contribution a single, squashed commit? _Additional
commits in response to PR reviewer feedback should be made on this branch and
pushed to allow change tracking. Do not `squash` or use `--force` when pushing
to allow for clean monitoring of changes._
### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via `mvn
-Pcontrib-check clean install` at the root `nifi` folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] Have you verified that the full build is successful on JDK 8?
- [ ] Have you verified that the full build is successful on JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main
`LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main
`NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI for
build issues and submit an update to your PR as soon as possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Created] (NIFI-9696) DeleteS3Object do not have provenance data
Law Yaw Eng created NIFI-9696: - Summary: DeleteS3Object do not have provenance data Key: NIFI-9696 URL: https://issues.apache.org/jira/browse/NIFI-9696 Project: Apache NiFi Issue Type: Improvement Reporter: Law Yaw Eng DeleteS3Object processor do not have any provenance data. While it's true that it doesn't change any flowfile or attribute, it does an external action. Perhaps a SEND provenance data with transitUri value of "DELETE " will be suitable -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (NIFI-9695) Load balancing not working
[
https://issues.apache.org/jira/browse/NIFI-9695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
davide updated NIFI-9695:
-
Labels: Cluster balancer nifi (was: )
> Load balancing not working
> --
>
> Key: NIFI-9695
> URL: https://issues.apache.org/jira/browse/NIFI-9695
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.13.2
>Reporter: davide
>Priority: Major
> Labels: Cluster, balancer, nifi
> Attachments: activeThread.png, cores.png, flow.png, heapUse.png,
> version.png
>
>
> I have a 3 nodes cluster(as you can see on attached images) with a connection
> with a Load Balance Strategy set to Partition by attribute, sometimes some
> flowfiles get stuck in this connection (see image1). Furthermore we have this
> log error in bulletin board
> {\{10:56:28 UTC ERROR -xxx-xx-nifi0.:8443 }}
> {{Failed to receive FlowFiles for Load Balancing due to
> org.apache.nifi.controller.queue.clustered.server.TransactionAbortedException:
> Attempted to receive FlowFiles from Peer -xxx-xx-nifi2. for
> Connection with ID 0c20343f-f63d-1ef1-8da0-6d3cb9fc670d but no Connection
> exists with that ID}}
> {{10:56:28 UTC ERROR -xxx-xx-nifi0.:8443}}
> {{{}Failed to communicate with Peer -xxx-xx-nifi2.:8443 when load
> balancing data for Connection with ID c12f3c02-9f05-1232--f8d925a9
> due to java.io.IOException: Failed to decrypt data from Peer
> -xxx-xx-nifi2.:8443 because Peer unexpectedly closed
> connection{}}}}}{}}}
>
> The weird thing is that only some flowfiles are stucked other flowfiles goes
> forward in the flows.
> I think this bug is the same of
> https://issues.apache.org/jira/browse/NIFI-9574, I post a comment in
> NIFI-9574 but I have got no answer.
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (NIFI-9695) Load balancing not working
[
https://issues.apache.org/jira/browse/NIFI-9695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
davide updated NIFI-9695:
-
Description:
I have a 3 nodes cluster(as you can see on attached images) with a connection
with a Load Balance Strategy set to Partition by attribute, sometimes some
flowfiles get stuck in this connection (see image1). Furthermore we have this
log error in bulletin board
{\{10:56:28 UTC ERROR -xxx-xx-nifi0.:8443 }}
{{Failed to receive FlowFiles for Load Balancing due to
org.apache.nifi.controller.queue.clustered.server.TransactionAbortedException:
Attempted to receive FlowFiles from Peer -xxx-xx-nifi2. for Connection
with ID 0c20343f-f63d-1ef1-8da0-6d3cb9fc670d but no Connection exists with that
ID}}
{{10:56:28 UTC ERROR -xxx-xx-nifi0.:8443}}
{{{}Failed to communicate with Peer -xxx-xx-nifi2.:8443 when load
balancing data for Connection with ID c12f3c02-9f05-1232--f8d925a9 due
to java.io.IOException: Failed to decrypt data from Peer
-xxx-xx-nifi2.:8443 because Peer unexpectedly closed
connection{}}}}}{}}}
The weird thing is that only some flowfiles are stucked other flowfiles goes
forward in the flows.
I think this bug is the same of
https://issues.apache.org/jira/browse/NIFI-9574, I post a comment in NIFI-9574
but I have got no answer.
was:
I have a 3 nodes cluster(as you can see on attached images) with a connection
with a Load Balance Strategy set to Partition by attribute, sometimes some
flowfiles get stuck in this connection (see image1). Furthermore we have this
log error in bulletin board
{\{10:56:28 UTC ERROR -xxx-xx-nifi0.:8443 }}
{{Failed to receive FlowFiles for Load Balancing due to
org.apache.nifi.controller.queue.clustered.server.TransactionAbortedException:
Attempted to receive FlowFiles from Peer -xxx-xx-nifi2. for Connection
with ID 0c20343f-f63d-1ef1-8da0-6d3cb9fc670d but no Connection exists with that
ID}}
{{10:56:28 UTC ERROR -xxx-xx-nifi0.:8443}}
{{{}Failed to communicate with Peer -xxx-xx-nifi2.:8443 when load
balancing data for Connection with ID c12f3c02-9f05-1232--f8d925a9 due
to java.io.IOException: Failed to decrypt data from Peer
-xxx-xx-nifi2.:8443 because Peer unexpectedly closed
connection{}}}}}{}}}
The weird thing is that only some flowfiles are stucked other flowfiles goes
forward in the flows.
Is the same problem of https://issues.apache.org/jira/browse/NIFI-9574, I post
a comment in NIFI-9574 but I have got no answer.
> Load balancing not working
> --
>
> Key: NIFI-9695
> URL: https://issues.apache.org/jira/browse/NIFI-9695
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.13.2
>Reporter: davide
>Priority: Major
> Attachments: activeThread.png, cores.png, flow.png, heapUse.png,
> version.png
>
>
> I have a 3 nodes cluster(as you can see on attached images) with a connection
> with a Load Balance Strategy set to Partition by attribute, sometimes some
> flowfiles get stuck in this connection (see image1). Furthermore we have this
> log error in bulletin board
> {\{10:56:28 UTC ERROR -xxx-xx-nifi0.:8443 }}
> {{Failed to receive FlowFiles for Load Balancing due to
> org.apache.nifi.controller.queue.clustered.server.TransactionAbortedException:
> Attempted to receive FlowFiles from Peer -xxx-xx-nifi2. for
> Connection with ID 0c20343f-f63d-1ef1-8da0-6d3cb9fc670d but no Connection
> exists with that ID}}
> {{10:56:28 UTC ERROR -xxx-xx-nifi0.:8443}}
> {{{}Failed to communicate with Peer -xxx-xx-nifi2.:8443 when load
> balancing data for Connection with ID c12f3c02-9f05-1232--f8d925a9
> due to java.io.IOException: Failed to decrypt data from Peer
> -xxx-xx-nifi2.:8443 because Peer unexpectedly closed
> connection{}}}}}{}}}
>
> The weird thing is that only some flowfiles are stucked other flowfiles goes
> forward in the flows.
> I think this bug is the same of
> https://issues.apache.org/jira/browse/NIFI-9574, I post a comment in
> NIFI-9574 but I have got no answer.
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Created] (NIFI-9695) Load balancing not working
davide created NIFI-9695:
Summary: Load balancing not working
Key: NIFI-9695
URL: https://issues.apache.org/jira/browse/NIFI-9695
Project: Apache NiFi
Issue Type: Bug
Affects Versions: 1.13.2
Reporter: davide
Attachments: activeThread.png, cores.png, flow.png, heapUse.png,
version.png
I have a 3 nodes cluster(as you can see on attached images) with a connection
with a Load Balance Strategy set to Partition by attribute, sometimes some
flowfiles get stuck in this connection (see image1). Furthermore we have this
log error in bulletin board
{\{10:56:28 UTC ERROR -xxx-xx-nifi0.:8443 }}
{{Failed to receive FlowFiles for Load Balancing due to
org.apache.nifi.controller.queue.clustered.server.TransactionAbortedException:
Attempted to receive FlowFiles from Peer -xxx-xx-nifi2. for Connection
with ID 0c20343f-f63d-1ef1-8da0-6d3cb9fc670d but no Connection exists with that
ID}}
{{10:56:28 UTC ERROR -xxx-xx-nifi0.:8443}}
{{{}Failed to communicate with Peer -xxx-xx-nifi2.:8443 when load
balancing data for Connection with ID c12f3c02-9f05-1232--f8d925a9 due
to java.io.IOException: Failed to decrypt data from Peer
-xxx-xx-nifi2.:8443 because Peer unexpectedly closed
connection{}}}}}{}}}
The weird thing is that only some flowfiles are stucked other flowfiles goes
forward in the flows.
Is the same problem of https://issues.apache.org/jira/browse/NIFI-9574, I post
a comment in NIFI-9574 but I have got no answer.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
