[jira] [Updated] (NIFI-10623) Several Related Flaky Tests in TestHttpClient
[
https://issues.apache.org/jira/browse/NIFI-10623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yiteng Zhang updated NIFI-10623:
Description:
These tests:
{code:java}
org.apache.nifi.remote.client.http.TestHttpClient.testSendLargeFileHTTP
org.apache.nifi.remote.client.http.TestHttpClient.testSendLargeFileHTTPS
org.apache.nifi.remote.client.http.TestHttpClient.testSendLargeFileHTTPSWithProxy
org.apache.nifi.remote.client.http.TestHttpClient.testSendLargeFileHTTPSWithProxyAuth
org.apache.nifi.remote.client.http.TestHttpClient.testSendLargeFileHTTPWithProxy
org.apache.nifi.remote.client.http.TestHttpClient.testSendLargeFileHTTPWithProxyAuth
org.apache.nifi.remote.client.http.TestHttpClient.testSendSlowClientSuccess
org.apache.nifi.remote.client.http.TestHttpClient.testSendSuccess
org.apache.nifi.remote.client.http.TestHttpClient.testSendSuccessCompressed
org.apache.nifi.remote.client.http.TestHttpClient.testSendSuccessHTTPS
org.apache.nifi.remote.client.http.TestHttpClient.testSendSuccessMultipleUrls
org.apache.nifi.remote.client.http.TestHttpClient.testSendSuccessWithProxy
org.apache.nifi.remote.client.http.TestHttpClient.testSendSuccessWithProxyAuth
{code}
can pass using the normal maven-test, but show non-deterministic behaviors
under [NonDex|[https://github.com/TestingResearchIllinois/NonDex]] and thus
failed.
An example of the error message is below:
{code:java}
[ERROR] Errors:
[ERROR]
TestHttpClient.testSendLargeFileHTTPS:1024->testSendLargeFile:954->testSendIgnoreProxyError:940
» IO Failed to confirm transaction with
Peer[url=https://localhost:33385/nifi-api] due to java.io.IOException:
[Url=https://localhost:33385/nifi-api, TransferDirection=SEND,
State=DATA_EXCHANGED] Sent data to peer
Peer[url=https://localhost:33385/nifi-api] but calculated CRC32 Checksum as
2387509971 while peer calculated CRC32 Checksum as 1527414060; canceling
transaction and rolling back session {code}
It seems like two peers calculate a different checksum, therefore the
transaction was rolled back, and thus the test failed.
*Steps to reproduce the failure:*
# install [NonDex|[https://github.com/TestingResearchIllinois/NonDex]]
# run the following command in nifi
{code:java}
MODULE=nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-site-to-site
TEST=org.apache.nifi.remote.protocol.http.TestHttpFlowFileServerProtocol#testTransferTwoFiles
mvn install -pl $MODULE -am -DskipTests
mvn -pl $MODULE edu.illinois:nondex-maven-plugin:1.1.2:nondex -Dtest=$TEST{code}
3. the result will be saved under the package folder in .nondex
h4.
was:
These tests:
{code:java}
org.apache.nifi.remote.client.http.TestHttpClient.testSendLargeFileHTTP
org.apache.nifi.remote.client.http.TestHttpClient.testSendLargeFileHTTPS
org.apache.nifi.remote.client.http.TestHttpClient.testSendLargeFileHTTPSWithProxy
org.apache.nifi.remote.client.http.TestHttpClient.testSendLargeFileHTTPSWithProxyAuth
org.apache.nifi.remote.client.http.TestHttpClient.testSendLargeFileHTTPWithProxy
org.apache.nifi.remote.client.http.TestHttpClient.testSendLargeFileHTTPWithProxyAuth
org.apache.nifi.remote.client.http.TestHttpClient.testSendSuccessWithProxyAuth
org.apache.nifi.remote.client.http.TestHttpClient.testSendSlowClientSuccess
org.apache.nifi.remote.client.http.TestHttpClient.testSendSuccessCompressed{code}
can pass using the normal maven-test, but show non-deterministic behaviors
under [NonDex|[https://github.com/TestingResearchIllinois/NonDex]] and thus
failed.
An example of the error message is below:
{code:java}
[ERROR] Errors:
[ERROR]
TestHttpClient.testSendLargeFileHTTPS:1024->testSendLargeFile:954->testSendIgnoreProxyError:940
» IO Failed to confirm transaction with
Peer[url=https://localhost:33385/nifi-api] due to java.io.IOException:
[Url=https://localhost:33385/nifi-api, TransferDirection=SEND,
State=DATA_EXCHANGED] Sent data to peer
Peer[url=https://localhost:33385/nifi-api] but calculated CRC32 Checksum as
2387509971 while peer calculated CRC32 Checksum as 1527414060; canceling
transaction and rolling back session {code}
It seems like two peers calculate a different checksum, therefore the
transaction was rolled back, and thus the test failed.
*Steps to reproduce the failure:*
# install [NonDex|[https://github.com/TestingResearchIllinois/NonDex]]
# run the following command in nifi
{code:java}
MODULE=nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-site-to-site
TEST=org.apache.nifi.remote.protocol.http.TestHttpFlowFileServerProtocol#testTransferTwoFiles
mvn install -pl $MODULE -am -DskipTests
mvn -pl $MODULE edu.illinois:nondex-maven-plugin:1.1.2:nondex -Dtest=$TEST{code}
3. the result will be saved under the package folder in .nondex
h4.
> Several Related Flaky Tests in TestHttpClient
> -
>
> Key: NIFI-10623
> URL:
[GitHub] [nifi] Kerr0220 opened a new pull request, #6508: NIFI-10623 fix flaky tests in TestHttpClient
Kerr0220 opened a new pull request, #6508: URL: https://github.com/apache/nifi/pull/6508 # Summary [NIFI-10623](https://issues.apache.org/jira/browse/NIFI-10623) These flaky tests were caused by the feature of Map since the order of iterating map by _enhanced for loop_ is non-deterministic. Therefore, checksums calculated based on the stream were also non-deterministic and thus failed when using NonDex to test. After changing the order to deterministic by first sorting keys, and adjusting the values of checksums in test cases, these tests are no longer flaky. # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [x] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [x] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [x] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [x] Pull Request based on current revision of the `main` branch - [x] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [x] JDK 8 - [ ] JDK 11 - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] Kerr0220 commented on pull request #6508: NIFI-10623 fix flaky tests in TestHttpClient
Kerr0220 commented on PR #6508: URL: https://github.com/apache/nifi/pull/6508#issuecomment-1344843580 @exceptionfactory Hi, could you please reopen this PR? I've pushed a new commit to this PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] martinzink commented on a diff in pull request #1424: MINIFICPP-1862 use std::filesystem::path instead of std::string where appropriate
martinzink commented on code in PR #1424:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1424#discussion_r1044871408
##
extensions/jni/jvm/JniReferenceObjects.h:
##
@@ -299,24 +278,26 @@ class JniSessionFactory : public core::WeakReference {
}
}
- jobject getJavaReference() const {
+ [[nodiscard]] jobject getJavaReference() const {
return java_object_;
}
- std::shared_ptr getServicer() const {
+ [[nodiscard]] std::shared_ptr getServicer() const {
return servicer_;
}
- std::shared_ptr getFactory() const {
+ [[nodiscard]] std::shared_ptr getFactory()
const {
return factory_;
}
/**
*/
- JniSession *addSession(std::shared_ptr session) {
+ JniSession *addSession(const std::shared_ptr& session) {
std::lock_guard guard(session_mutex_);
-sessions_.erase(std::remove_if(sessions_.begin(), sessions_.end(),
check_empty()), sessions_.end());
+const auto check_empty = [](const std::shared_ptr& session) {
return session->prune(); };
+
+sessions_.erase(std::remove_if(sessions_.begin(), sessions_.end(),
check_empty), sessions_.end());
Review Comment:
good idea, I've simplified this and the empty() function in this file
https://github.com/apache/nifi-minifi-cpp/pull/1424/commits/1b8eb634cf2055efc79d7f49f780715c6730a6dc
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] martinzink commented on a diff in pull request #1424: MINIFICPP-1862 use std::filesystem::path instead of std::string where appropriate
martinzink commented on code in PR #1424:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1424#discussion_r1044842683
##
libminifi/test/unit/FileStreamTests.cpp:
##
@@ -301,32 +267,25 @@ TEST_CASE("Existing file read/write test") {
stream.seek(0);
}
-#if !defined(WIN32) || defined(USE_BOOST)
-// This could be simplified with C++17 std::filesystem
TEST_CASE("Opening file without permission creates error logs") {
TestController test_controller;
auto dir = test_controller.createTempDirectory();
- std::string path_to_permissionless_file(utils::file::concat_path(dir,
"permissionless_file.txt"));
+ auto path_to_permissionless_file = dir / "permissionless_file.txt";
{
std::ofstream outfile(path_to_permissionless_file);
outfile << "this file has been just created" << std::endl;
outfile.close();
-#ifndef WIN32
utils::file::FileUtils::set_permissions(path_to_permissionless_file, 0);
-#else
-boost::filesystem::permissions(path_to_permissionless_file,
boost::filesystem::no_perms);
-#endif
}
- minifi::io::FileStream stream(path_to_permissionless_file, 0, false);
+ minifi::io::FileStream stream(path_to_permissionless_file, 0, true);
Review Comment:
sure thing,
https://github.com/apache/nifi-minifi-cpp/pull/1424/commits/eb00f1b247b813448334cb1b9a019164e7e07b28#diff-c3a81806f45337794ecf51dd1a4507b18fd009557635c9775d29d1a5c77c7800R280
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] martinzink commented on a diff in pull request #1424: MINIFICPP-1862 use std::filesystem::path instead of std::string where appropriate
martinzink commented on code in PR #1424:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1424#discussion_r1044840784
##
extensions/sftp/tests/ListThenFetchSFTPTests.cpp:
##
@@ -121,7 +120,7 @@ class ListThenFetchSFTPTestsFixture {
plan->setProperty(list_sftp, "Minimum File Size", "0 B");
plan->setProperty(list_sftp, "Target System Timestamp Precision",
"Seconds");
plan->setProperty(list_sftp, "Remote Path", "nifi_test/");
-plan->setProperty(list_sftp, "State File", src_dir + "/state");
+plan->setProperty(list_sftp, "State File", src_dir.string() + "/state");
Review Comment:
good idea,
https://github.com/apache/nifi-minifi-cpp/pull/1424/commits/eb00f1b247b813448334cb1b9a019164e7e07b28#diff-dfaf51b7f8862c8ffdda0f89bd11d14b8fa83ef70ba0d9e5e686acb1afa51e5fR123
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] martinzink commented on a diff in pull request #1424: MINIFICPP-1862 use std::filesystem::path instead of std::string where appropriate
martinzink commented on code in PR #1424:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1424#discussion_r1044839048
##
extensions/sftp/tests/PutSFTPTests.cpp:
##
@@ -244,7 +240,7 @@ TEST_CASE_METHOD(PutSFTPTestsFixture, "PutSFTP bad
password", "[PutSFTP][authent
}
TEST_CASE_METHOD(PutSFTPTestsFixture, "PutSFTP public key authentication
success", "[PutSFTP][authentication]") {
- plan->setProperty(put, "Private Key Path",
utils::file::concat_path(get_sftp_test_dir(), "resources/id_rsa"));
+ plan->setProperty(put, "Private Key Path", get_sftp_test_dir() /
"resources/id_rsa");
Review Comment:
I've changed these and the ones in ListSFTPTests in
https://github.com/apache/nifi-minifi-cpp/pull/1424/commits/b7fce3d7cf85180b6054df874a0ad7b5b6f2e31e
I havent changed the occurrences where the root is a string aswell (e.g.
["nifi_test/tstFile.ext"](https://github.com/martinzink/nifi-minifi-cpp/blob/MINIFICPP-1862-2/extensions/sftp/tests/ListSFTPTests.cpp#L209),
because I think that could hinder the readability and std::filesystem is
capable to correctly parse those forward slashes even on windows, but I am
curious about your opinion.
##
extensions/sftp/tests/ListSFTPTests.cpp:
##
@@ -187,7 +180,7 @@ TEST_CASE_METHOD(ListSFTPTestsFixture, "ListSFTP list one
file", "[ListSFTP][bas
TEST_CASE_METHOD(ListSFTPTestsFixture, "ListSFTP public key authentication",
"[ListSFTP][basic]") {
plan->setProperty(list_sftp, "Remote File", "nifi_test/tstFile.ext");
- plan->setProperty(list_sftp, "Private Key Path",
utils::file::FileUtils::concat_path(get_sftp_test_dir(), "resources/id_rsa"));
+ plan->setProperty(list_sftp, "Private Key Path", get_sftp_test_dir() /
"resources/id_rsa");
Review Comment:
I've changed these and the ones in PutSFTPTests in
https://github.com/apache/nifi-minifi-cpp/pull/1424/commits/b7fce3d7cf85180b6054df874a0ad7b5b6f2e31e
I havent changed the occurrences where the root is a string aswell (e.g.
["nifi_test/tstFile.ext"](https://github.com/martinzink/nifi-minifi-cpp/blob/MINIFICPP-1862-2/extensions/sftp/tests/ListSFTPTests.cpp#L209),
because I think that could hinder the readability and std::filesystem is
capable to correctly parse those forward slashes even on windows, but I am
curious about your opinion.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Resolved] (NIFI-10453) Cannot set AnnotationData while processor is running
[
https://issues.apache.org/jira/browse/NIFI-10453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Payne resolved NIFI-10453.
---
Resolution: Fixed
> Cannot set AnnotationData while processor is running
>
>
> Key: NIFI-10453
> URL: https://issues.apache.org/jira/browse/NIFI-10453
> Project: Apache NiFi
> Issue Type: Bug
> Components: Flow Versioning
>Affects Versions: 1.17.0, 1.16.3
> Environment: Linux debian
>Reporter: hipotures
>Priority: Major
>
> *Action*: Change version
> *Problem*: Can't change to newer version of flow
> *Web UI error popup*:
> Change Version
> Failed to perform update flow request due to Cannot set AnnotationData while
> processor is running
> *Description*: Updated flow is in state STOPPED. Controller is active (this
> controller is shared between many other flows).
> *Workaround*: Stop all flows and controllers.
> *Error Log*:
> {code:java}
> 2022-09-07 10:43:05,380 ERROR [Process Group Update Thread-1]
> o.apache.nifi.web.api.FlowUpdateResource Failed to perform update flow
> request
> java.lang.IllegalStateException: Cannot set AnnotationData while processor is
> running
> at
> org.apache.nifi.controller.StandardProcessorNode.setAnnotationData(StandardProcessorNode.java:1370)
> at
> org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.updateProcessor(StandardVersionedComponentSynchronizer.java:2529)
> at
> org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronizeProcessors(StandardVersionedComponentSynchronizer.java:880)
> at
> org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronize(StandardVersionedComponentSynchronizer.java:407)
> at
> org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronizeChildGroups(StandardVersionedComponentSynchronizer.java:489)
> at
> org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronize(StandardVersionedComponentSynchronizer.java:401)
> at
> org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronizeChildGroups(StandardVersionedComponentSynchronizer.java:489)
> at
> org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronize(StandardVersionedComponentSynchronizer.java:401)
> at
> org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.lambda$synchronize$0(StandardVersionedComponentSynchronizer.java:248)
> at
> org.apache.nifi.controller.flow.AbstractFlowManager.withParameterContextResolution(AbstractFlowManager.java:464)
> at
> org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronize(StandardVersionedComponentSynchronizer.java:246)
> at
> org.apache.nifi.groups.StandardProcessGroup.synchronizeFlow(StandardProcessGroup.java:3825)
> at
> org.apache.nifi.groups.StandardProcessGroup.updateFlow(StandardProcessGroup.java:3805)
> at
> org.apache.nifi.web.dao.impl.StandardProcessGroupDAO.updateProcessGroupFlow(StandardProcessGroupDAO.java:431)
> at
> org.apache.nifi.web.dao.impl.StandardProcessGroupDAO$$FastClassBySpringCGLIB$$10a99b47.invoke()
> at
> org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:793)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
> at
> org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:89)
> at
> org.apache.nifi.audit.ProcessGroupAuditor.updateProcessGroupFlowAdvice(ProcessGroupAuditor.java:313)
> at sun.reflect.GeneratedMethodAccessor1323.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:634)
> at
> org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:624)
> at
> org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:72)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
> at
>
[jira] [Updated] (NIFI-10877) Refactor nifi-framework-bundle to use JUnit 5 Part 2
[ https://issues.apache.org/jira/browse/NIFI-10877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Stieglitz updated NIFI-10877: Status: Patch Available (was: In Progress) > Refactor nifi-framework-bundle to use JUnit 5 Part 2 > > > Key: NIFI-10877 > URL: https://issues.apache.org/jira/browse/NIFI-10877 > Project: Apache NiFi > Issue Type: Sub-task >Reporter: Daniel Stieglitz >Assignee: Daniel Stieglitz >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > Will make changes in the following directory > nifi-framework-bundle/nifi-framework/nifi-framework-core -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] dan-s1 opened a new pull request, #6775: [NIFI-10877] Conversion of nifi-framework-core tests from Junit4 to J…
dan-s1 opened a new pull request, #6775: URL: https://github.com/apache/nifi/pull/6775 …unit5 # Summary [NIFI-0](https://issues.apache.org/jira/browse/NIFI-0) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 8 - [ ] JDK 11 - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] exceptionfactory closed pull request #6517: NIFI-10576 - ParquetRecordSetWriter doesn't write avro schema to attribute
exceptionfactory closed pull request #6517: NIFI-10576 - ParquetRecordSetWriter doesn't write avro schema to attribute URL: https://github.com/apache/nifi/pull/6517 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (NIFI-10576) ParquetRecordSetWriter doesn't write avro schema
[ https://issues.apache.org/jira/browse/NIFI-10576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann resolved NIFI-10576. - Fix Version/s: 1.20.0 Resolution: Fixed > ParquetRecordSetWriter doesn't write avro schema > > > Key: NIFI-10576 > URL: https://issues.apache.org/jira/browse/NIFI-10576 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.15.2 >Reporter: DEOM Damien >Assignee: Nathan Gough >Priority: Critical > Fix For: 1.20.0 > > Time Spent: 10m > Remaining Estimate: 0h > > ParquetRecordSetWrite ignores Set 'avro.schema' Attribute option -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-10576) ParquetRecordSetWriter doesn't write avro schema
[ https://issues.apache.org/jira/browse/NIFI-10576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17645478#comment-17645478 ] ASF subversion and git services commented on NIFI-10576: Commit 2fa82179a81877c975333ba1315f9431330780f6 in nifi's branch refs/heads/main from Nathan Gough [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=2fa82179a8 ] NIFI-10576 Added onFinishRecordSet implementation for WriteParquetResult This closes #6517 Co-authored-by: David Handermann Signed-off-by: David Handermann > ParquetRecordSetWriter doesn't write avro schema > > > Key: NIFI-10576 > URL: https://issues.apache.org/jira/browse/NIFI-10576 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.15.2 >Reporter: DEOM Damien >Assignee: Nathan Gough >Priority: Critical > Time Spent: 10m > Remaining Estimate: 0h > > ParquetRecordSetWrite ignores Set 'avro.schema' Attribute option -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (NIFI-9894) Enhance InvokeHTTP process - Send content through body variable
[ https://issues.apache.org/jira/browse/NIFI-9894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17645470#comment-17645470 ] David Handermann edited comment on NIFI-9894 at 12/9/22 7:43 PM: - No new activity right now. This is an issue where it would be helpful to get some additional input. [~markap14] or [~pvillard] do either of you have additional comments on this feature? It could be introduced through a new strategy property, although InvokeHTTP already has a large number of properties. As indicated by the duplicated issues, however, this has come up a number of times. was (Author: exceptionfactory): No new activity right now. This is an issue where it would be helpful to get some additional input. [~markap14] or [~pvillard]] do either of you have additional comments on this feature? It could be introduced through a new strategy property, although InvokeHTTP already has a large number of properties. As indicated by the duplicated issues, however, this has come up a number of times. > Enhance InvokeHTTP process - Send content through body variable > --- > > Key: NIFI-9894 > URL: https://issues.apache.org/jira/browse/NIFI-9894 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 1.15.3 >Reporter: Denis Jakupovic >Priority: Trivial > > Hey, > could you please implement an attribute in the InvokeHTTP processor where the > processor sends a "body attribute" as raw content instead of the current > content triggering an InvokeHTTP processor. > The process of sending JSON files through the processor currently is: > 1. ReplaceText with content if needed > 2. Call Invoke HTTP > 3. Read Reponse variable or the new content created > 3. ReplaceText with enriched content > Yes I know in Nifi 1.16 there is are two new processor for enrichment which > needs then to be merged again. However, by working with a lot of RESTful > services its quite a pain always to replace the content. The data lineage and > provenance increases a lot as well. > imho better process in some use case: > # InvokeHTTP with attibute as body > # Read the "original" route instead of the response and read response adde > attribute. (feature already available) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-9894) Enhance InvokeHTTP process - Send content through body variable
[ https://issues.apache.org/jira/browse/NIFI-9894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17645470#comment-17645470 ] David Handermann commented on NIFI-9894: No new activity right now. This is an issue where it would be helpful to get some additional input. [~markap14] or [~pvillard]] do either of you have additional comments on this feature? It could be introduced through a new strategy property, although InvokeHTTP already has a large number of properties. As indicated by the duplicated issues, however, this has come up a number of times. > Enhance InvokeHTTP process - Send content through body variable > --- > > Key: NIFI-9894 > URL: https://issues.apache.org/jira/browse/NIFI-9894 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 1.15.3 >Reporter: Denis Jakupovic >Priority: Trivial > > Hey, > could you please implement an attribute in the InvokeHTTP processor where the > processor sends a "body attribute" as raw content instead of the current > content triggering an InvokeHTTP processor. > The process of sending JSON files through the processor currently is: > 1. ReplaceText with content if needed > 2. Call Invoke HTTP > 3. Read Reponse variable or the new content created > 3. ReplaceText with enriched content > Yes I know in Nifi 1.16 there is are two new processor for enrichment which > needs then to be merged again. However, by working with a lot of RESTful > services its quite a pain always to replace the content. The data lineage and > provenance increases a lot as well. > imho better process in some use case: > # InvokeHTTP with attibute as body > # Read the "original" route instead of the response and read response adde > attribute. (feature already available) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-10877) Refactor nifi-framework-bundle to use JUnit 5 Part 2
[ https://issues.apache.org/jira/browse/NIFI-10877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17645467#comment-17645467 ] David Handermann commented on NIFI-10877: - Thanks for the update and work on this [~dstiegli1]! Yes, feel free to go ahead and submit a pull request, it sounds like those integration tests have latent problems that can be addressed separately. > Refactor nifi-framework-bundle to use JUnit 5 Part 2 > > > Key: NIFI-10877 > URL: https://issues.apache.org/jira/browse/NIFI-10877 > Project: Apache NiFi > Issue Type: Sub-task >Reporter: Daniel Stieglitz >Assignee: Daniel Stieglitz >Priority: Minor > > Will make changes in the following directory > nifi-framework-bundle/nifi-framework/nifi-framework-core -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] NissimShiman commented on pull request #6725: NIFI-10888: When inferring a schema using a Record Reader, buffer up …
NissimShiman commented on PR #6725: URL: https://github.com/apache/nifi/pull/6725#issuecomment-1344673531 This is a really nice enhancement. Tested with csv files being converted to json files before and after fix for csv files of 21 and 63 bytes, there was 25 -30% speed up in processing (19/20 seconds to 15 seconds) for csv files of 549 bytes, I saw a 33%+ speed up (37 to 23 seconds) This was compiled/run with java openjdk 1.8.0_332 on linux kernel 3.10.0-1160 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-10958) Documentation is not generated for some components in Java 17
[ https://issues.apache.org/jira/browse/NIFI-10958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-10958: Affects Version/s: 1.19.1 Status: Patch Available (was: Open) > Documentation is not generated for some components in Java 17 > - > > Key: NIFI-10958 > URL: https://issues.apache.org/jira/browse/NIFI-10958 > Project: Apache NiFi > Issue Type: Bug > Components: Documentation Website >Affects Versions: 1.19.1, 1.19.0 > Environment: Java 17 >Reporter: Mark Bean >Assignee: David Handermann >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > When building with Java 17, the documentation for some components is not > generated properly. Affected components are identified by the following > warning messages which appear in nifi-app.log when docs are first created: > > 2022-12-07 09:13:17,236 WARN [main] o.apache.nifi.documentation.DocGenerator > Documentation generation failed: Component Class [class > org.apache.nifi.processors.script.ExecuteScript] > 2022-12-07 09:13:17,610 WARN [main] o.apache.nifi.documentation.DocGenerator > Documentation generation failed: Component Class [class > org.apache.nifi.processors.script.InvokeScriptedProcessor] > 2022-12-07 09:13:18,071 WARN [main] o.apache.nifi.documentation.DocGenerator > Documentation generation failed: Component Class [class > org.apache.nifi.record.sink.script.ScriptedRecordSink] > 2022-12-07 09:13:18,074 WARN [main] o.apache.nifi.documentation.DocGenerator > Documentation generation failed: Component Class [class > org.apache.nifi.rules.engine.script.ScriptedRulesEngine] > 2022-12-07 09:13:18,078 WARN [main] o.apache.nifi.documentation.DocGenerator > Documentation generation failed: Component Class [class > org.apache.nifi.record.script.ScriptedReader] > 2022-12-07 09:13:18,082 WARN [main] o.apache.nifi.documentation.DocGenerator > Documentation generation failed: Component Class [class > org.apache.nifi.rules.handlers.script.ScriptedActionHandler] > 2022-12-07 09:13:18,088 WARN [main] o.apache.nifi.documentation.DocGenerator > Documentation generation failed: Component Class [class > org.apache.nifi.lookup.script.SimpleScriptedLookupService] > 2022-12-07 09:13:18,094 WARN [main] o.apache.nifi.documentation.DocGenerator > Documentation generation failed: Component Class [class > org.apache.nifi.lookup.script.ScriptedLookupService] > 2022-12-07 09:13:18,098 WARN [main] o.apache.nifi.documentation.DocGenerator > Documentation generation failed: Component Class [class > org.apache.nifi.record.script.ScriptedRecordSetWriter] > 2022-12-07 09:13:18,262 WARN [main] o.apache.nifi.documentation.DocGenerator > Documentation generation failed: Component Class [class > org.apache.nifi.reporting.script.ScriptedReportingTask] > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] exceptionfactory opened a new pull request, #6774: NIFI-10958 Adjust Script Engine handling to avoid errors on Java 17
exceptionfactory opened a new pull request, #6774: URL: https://github.com/apache/nifi/pull/6774 # Summary [NIFI-10958](https://issues.apache.org/jira/browse/NIFI-10958) Adjusts Script Engine property handling in `ScriptingComponentsHelper` to avoid exceptions on Java 17 during documentation generation. Additional work is necessary to support ECMAScript and the Nashorn engine on Java 17, but introduced an empty check during the Script Engine introspection process avoids complete documentation generation failures. Current unit tests for `createResources()` work on Java 17 because the test environment has available classes. Further work on ECMAScript support using the Nashorn engine dependency on Java 17 is described in [NIFI-9383](https://issues.apache.org/jira/browse/NIFI-9383). # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 8 - [ ] JDK 11 - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] fgerlits commented on a diff in pull request #1424: MINIFICPP-1862 use std::filesystem::path instead of std::string where appropriate
fgerlits commented on code in PR #1424:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1424#discussion_r1044702728
##
extensions/sftp/tests/ListThenFetchSFTPTests.cpp:
##
@@ -121,7 +120,7 @@ class ListThenFetchSFTPTestsFixture {
plan->setProperty(list_sftp, "Minimum File Size", "0 B");
plan->setProperty(list_sftp, "Target System Timestamp Precision",
"Seconds");
plan->setProperty(list_sftp, "Remote Path", "nifi_test/");
-plan->setProperty(list_sftp, "State File", src_dir + "/state");
+plan->setProperty(list_sftp, "State File", src_dir.string() + "/state");
Review Comment:
why not `(src_dir / "state").string()`? (at line 142, too)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] thenatog commented on pull request #6517: NIFI-10576 - ParquetRecordSetWriter doesn't write avro schema to attribute
thenatog commented on PR #6517: URL: https://github.com/apache/nifi/pull/6517#issuecomment-1344598862 Thanks, if the avro.schema attribute is coming through on flowfiles coming out of the runner, then that sounds good to me. The simplest test would be to see that the attributes are now returned from the method I added, but I noticed that at run time it didn't come through until I added a schema cache service. Entirely possible I didn't understand how the schemas worked, and that setting it to use schema name(?) should work. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Comment Edited] (NIFI-10877) Refactor nifi-framework-bundle to use JUnit 5 Part 2
[ https://issues.apache.org/jira/browse/NIFI-10877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17645405#comment-17645405 ] Daniel Stieglitz edited comment on NIFI-10877 at 12/9/22 5:25 PM: -- [~exceptionfactory] [~mike.thomsen] I have almost completed converting all of the unit and integration tests in nifi-framework-bundle/nifi-framework/nifi-framework-core. There are six integration tests which fail even without my changes. The failing integration tests are: # ITZooKeeperStateServerTLS # StandardProcessorNodeIT # ITZooKeeperStateProvider # ParametersIT # ProcessorParameterTokenIT # ClusteredSwapFileIT Do you want me to submit a patch including my changes to these files anyway or not? was (Author: JIRAUSER294662): [~exceptionfactory] I have almost completed converting all of the unit and integration tests in nifi-framework-bundle/nifi-framework/nifi-framework-core. There are six integration tests which fail even without my changes. The failing integration tests are: # ITZooKeeperStateServerTLS # StandardProcessorNodeIT # ITZooKeeperStateProvider # ParametersIT # ProcessorParameterTokenIT # ClusteredSwapFileIT Do you want me to submit a patch including my changes to these files anyway or not? > Refactor nifi-framework-bundle to use JUnit 5 Part 2 > > > Key: NIFI-10877 > URL: https://issues.apache.org/jira/browse/NIFI-10877 > Project: Apache NiFi > Issue Type: Sub-task >Reporter: Daniel Stieglitz >Assignee: Daniel Stieglitz >Priority: Minor > > Will make changes in the following directory > nifi-framework-bundle/nifi-framework/nifi-framework-core -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2010) The systemd extension shouldn't be available on mac
[ https://issues.apache.org/jira/browse/MINIFICPP-2010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz updated MINIFICPP-2010: Resolution: Fixed Status: Resolved (was: Patch Available) > The systemd extension shouldn't be available on mac > --- > > Key: MINIFICPP-2010 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2010 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Affects Versions: 0.13.0 >Reporter: Marton Szasz >Assignee: Marton Szasz >Priority: Major > Fix For: 0.14.0 > > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi-minifi-cpp] fgerlits commented on a diff in pull request #1424: MINIFICPP-1862 use std::filesystem::path instead of std::string where appropriate
fgerlits commented on code in PR #1424:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1424#discussion_r1044661369
##
libminifi/test/unit/FileStreamTests.cpp:
##
@@ -301,32 +267,25 @@ TEST_CASE("Existing file read/write test") {
stream.seek(0);
}
-#if !defined(WIN32) || defined(USE_BOOST)
-// This could be simplified with C++17 std::filesystem
TEST_CASE("Opening file without permission creates error logs") {
TestController test_controller;
auto dir = test_controller.createTempDirectory();
- std::string path_to_permissionless_file(utils::file::concat_path(dir,
"permissionless_file.txt"));
+ auto path_to_permissionless_file = dir / "permissionless_file.txt";
{
std::ofstream outfile(path_to_permissionless_file);
outfile << "this file has been just created" << std::endl;
outfile.close();
-#ifndef WIN32
utils::file::FileUtils::set_permissions(path_to_permissionless_file, 0);
-#else
-boost::filesystem::permissions(path_to_permissionless_file,
boost::filesystem::no_perms);
-#endif
}
- minifi::io::FileStream stream(path_to_permissionless_file, 0, false);
+ minifi::io::FileStream stream(path_to_permissionless_file, 0, true);
Review Comment:
thanks, makes sense
can you add this as a comment here, please?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] fgerlits commented on a diff in pull request #1424: MINIFICPP-1862 use std::filesystem::path instead of std::string where appropriate
fgerlits commented on code in PR #1424:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1424#discussion_r1044660504
##
libminifi/test/rocksdb-tests/RocksDBTests.cpp:
##
@@ -232,9 +231,9 @@ void
withDefaultEnv(minifi::internal::Writable& db_opts) {
}
TEST_CASE_METHOD(RocksDBTest, "Error is logged if different encryption keys
are used", "[rocksDBTest10]") {
- utils::Path home_dir{createTempDirectory()};
- utils::file::FileUtils::create_dir((home_dir / "conf").str());
- std::ofstream{(home_dir / "conf" / "bootstrap.conf").str()}
+ auto home_dir = createTempDirectory();
+ utils::file::FileUtils::create_dir((home_dir / "conf"));
+ std::ofstream{(home_dir / "conf" / "bootstrap.conf")}
Review Comment:
same here; line 235 still has the double parens
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] fgerlits commented on a diff in pull request #1424: MINIFICPP-1862 use std::filesystem::path instead of std::string where appropriate
fgerlits commented on code in PR #1424:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1424#discussion_r1044659717
##
libminifi/test/rocksdb-tests/EncryptionTests.cpp:
##
@@ -77,8 +75,8 @@ TEST_CASE_METHOD(FFRepoFixture, "FlowFileRepository creates
checkpoint and loads
// pass
}
SECTION("With encryption") {
-utils::file::FileUtils::create_dir((home_ / "conf").str());
-std::ofstream{(home_ / "conf" / "bootstrap.conf").str()}
+utils::file::FileUtils::create_dir((home_ / "conf"));
+std::ofstream{(home_ / "conf" / "bootstrap.conf")}
Review Comment:
the double parens in line 78 are still there; are those needed?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (NIFI-10877) Refactor nifi-framework-bundle to use JUnit 5 Part 2
[ https://issues.apache.org/jira/browse/NIFI-10877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17645405#comment-17645405 ] Daniel Stieglitz commented on NIFI-10877: - [~exceptionfactory] I have almost completed converting all of the unit and integration tests in nifi-framework-bundle/nifi-framework/nifi-framework-core. There are six integration tests which fail even without my changes. The failing integration tests are: # ITZooKeeperStateServerTLS # StandardProcessorNodeIT # ITZooKeeperStateProvider # ParametersIT # ProcessorParameterTokenIT # ClusteredSwapFileIT Do you want me to submit a patch including my changes to these files anyway or not? > Refactor nifi-framework-bundle to use JUnit 5 Part 2 > > > Key: NIFI-10877 > URL: https://issues.apache.org/jira/browse/NIFI-10877 > Project: Apache NiFi > Issue Type: Sub-task >Reporter: Daniel Stieglitz >Assignee: Daniel Stieglitz >Priority: Minor > > Will make changes in the following directory > nifi-framework-bundle/nifi-framework/nifi-framework-core -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (NIFI-10942) TestGenerateTableFetch.testMultipleColumnTypeMissing is a flaky test
[
https://issues.apache.org/jira/browse/NIFI-10942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann reassigned NIFI-10942:
---
Assignee: Haoran Jiang
> TestGenerateTableFetch.testMultipleColumnTypeMissing is a flaky test
>
>
> Key: NIFI-10942
> URL: https://issues.apache.org/jira/browse/NIFI-10942
> Project: Apache NiFi
> Issue Type: Bug
> Components: NiFi Registry
> Environment: Apache Maven 3.6.0;
> openjdk version "1.8.0_342";
> OpenJDK Runtime Environment (build 1.8.0_342-8u342-b07-0ubuntu1~20.04-b07);
> OpenJDK 64-Bit Server VM (build 25.342-b07, mixed mode);
>Reporter: Haoran Jiang
>Assignee: Haoran Jiang
>Priority: Trivial
> Fix For: 1.20.0
>
> Attachments:
> org.apache.nifi.processors.standard.TestGenerateTableFetch..rtf
>
>
> {code:java}
> org.apache.nifi.processors.standard.TestGenerateTableFetch.testMultipleColumnTypeMissing
> {code}
> This is the flaky test. It can pass maven-test while testing failed under
> [NonDex|[https://github.com/TestingResearchIllinois/NonDex]].
> This is the log for the result.
> {code:java}
> [INFO] Results:
> [INFO]
> [ERROR] Failures:
> [ERROR] TestGenerateTableFetch.testMultipleColumnTypeMissing:1487 expected:
> <2> but was: <1>
> [INFO]
> [ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0 {code}
> *Steps to reproduce the failure:*
> 1.intall [Nondex|[https://github.com/TestingResearchIllinois/NonDex]]
> 2.Run the command in nifi:
> {code:java}
> mvn -pl nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors
> edu.illinois:nondex-maven-plugin:1.1.2:nondex
> -Dtest=org.apache.nifi.processors.standard.TestGenerateTableFetch#testMultipleColumnTypeMissing
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (NIFI-10637) TestParseCEF.testCustomValidator is a flaky test
[
https://issues.apache.org/jira/browse/NIFI-10637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann reassigned NIFI-10637:
---
Assignee: Haoran Jiang
> TestParseCEF.testCustomValidator is a flaky test
>
>
> Key: NIFI-10637
> URL: https://issues.apache.org/jira/browse/NIFI-10637
> Project: Apache NiFi
> Issue Type: Bug
> Components: NiFi Registry
> Environment: Apache Maven 3.6.0;
> openjdk version "1.8.0_342";
> OpenJDK Runtime Environment (build 1.8.0_342-8u342-b07-0ubuntu1~20.04-b07);
> OpenJDK 64-Bit Server VM (build 25.342-b07, mixed mode);
>Reporter: Haoran Jiang
>Assignee: Haoran Jiang
>Priority: Trivial
> Fix For: 1.19.0
>
> Attachments:
> org.apache.nifi.processors.standard.TestParseCEF.testCustomValidator-log.txt
>
> Original Estimate: 0h
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> {code:java}
> org.apache.nifi.processors.standard.TestParseCEF.testCustomValidator {code}
> This is the flaky test. it can pass maven-test while testing failed under
> [NonDex|[https://github.com/TestingResearchIllinois/NonDex]].
> This is the log for the result.
> {code:java}
> Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.46 s <<<
> FAILURE! - in org.apache.nifi.processors.standard.TestParseCEF
> org.apache.nifi.processors.standard.TestParseCEF.testCustomValidator Time
> elapsed: 0.391 s <<< FAILURE!
> org.opentest4j.AssertionFailedError: Processor appears to be valid but
> expected it to be invalid ==> expected: but was: {code}
> *Steps to reproduce the failure:*
> 1.intall [Nondex|https://github.com/TestingResearchIllinois/NonDex]
> 2.Run the command in nifi:
> {code:java}
> mvn install -pl
> nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors -am
> -DskipTests
> mvn -pl nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors
> edu.illinois:nondex-maven-plugin:1.1.2:nondex
> -Dtest=org.apache.nifi.processors.standard.TestParseCEF#testCustomValidator
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (NIFI-10942) TestGenerateTableFetch.testMultipleColumnTypeMissing is a flaky test
[
https://issues.apache.org/jira/browse/NIFI-10942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann resolved NIFI-10942.
-
Fix Version/s: 1.20.0
(was: 1.19.0)
Resolution: Fixed
Resolved in commit:
https://github.com/apache/nifi/commit/ac8e3dae58339c91177d5b8565e28fd3d5d74a83
> TestGenerateTableFetch.testMultipleColumnTypeMissing is a flaky test
>
>
> Key: NIFI-10942
> URL: https://issues.apache.org/jira/browse/NIFI-10942
> Project: Apache NiFi
> Issue Type: Bug
> Components: NiFi Registry
> Environment: Apache Maven 3.6.0;
> openjdk version "1.8.0_342";
> OpenJDK Runtime Environment (build 1.8.0_342-8u342-b07-0ubuntu1~20.04-b07);
> OpenJDK 64-Bit Server VM (build 25.342-b07, mixed mode);
>Reporter: Haoran Jiang
>Priority: Trivial
> Fix For: 1.20.0
>
> Attachments:
> org.apache.nifi.processors.standard.TestGenerateTableFetch..rtf
>
>
> {code:java}
> org.apache.nifi.processors.standard.TestGenerateTableFetch.testMultipleColumnTypeMissing
> {code}
> This is the flaky test. It can pass maven-test while testing failed under
> [NonDex|[https://github.com/TestingResearchIllinois/NonDex]].
> This is the log for the result.
> {code:java}
> [INFO] Results:
> [INFO]
> [ERROR] Failures:
> [ERROR] TestGenerateTableFetch.testMultipleColumnTypeMissing:1487 expected:
> <2> but was: <1>
> [INFO]
> [ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0 {code}
> *Steps to reproduce the failure:*
> 1.intall [Nondex|[https://github.com/TestingResearchIllinois/NonDex]]
> 2.Run the command in nifi:
> {code:java}
> mvn -pl nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors
> edu.illinois:nondex-maven-plugin:1.1.2:nondex
> -Dtest=org.apache.nifi.processors.standard.TestGenerateTableFetch#testMultipleColumnTypeMissing
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-10637) TestParseCEF.testCustomValidator is a flaky test
[
https://issues.apache.org/jira/browse/NIFI-10637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17645393#comment-17645393
]
ASF subversion and git services commented on NIFI-10637:
Commit ac8e3dae58339c91177d5b8565e28fd3d5d74a83 in nifi's branch
refs/heads/main from hj28
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=ac8e3dae58 ]
NIFI-10637 Deterministic TestGenerateTableFetch testMultipleColumnTypeMissing
This closes #6754
Signed-off-by: David Handermann
> TestParseCEF.testCustomValidator is a flaky test
>
>
> Key: NIFI-10637
> URL: https://issues.apache.org/jira/browse/NIFI-10637
> Project: Apache NiFi
> Issue Type: Bug
> Components: NiFi Registry
> Environment: Apache Maven 3.6.0;
> openjdk version "1.8.0_342";
> OpenJDK Runtime Environment (build 1.8.0_342-8u342-b07-0ubuntu1~20.04-b07);
> OpenJDK 64-Bit Server VM (build 25.342-b07, mixed mode);
>Reporter: Haoran Jiang
>Priority: Trivial
> Fix For: 1.19.0
>
> Attachments:
> org.apache.nifi.processors.standard.TestParseCEF.testCustomValidator-log.txt
>
> Original Estimate: 0h
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> {code:java}
> org.apache.nifi.processors.standard.TestParseCEF.testCustomValidator {code}
> This is the flaky test. it can pass maven-test while testing failed under
> [NonDex|[https://github.com/TestingResearchIllinois/NonDex]].
> This is the log for the result.
> {code:java}
> Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.46 s <<<
> FAILURE! - in org.apache.nifi.processors.standard.TestParseCEF
> org.apache.nifi.processors.standard.TestParseCEF.testCustomValidator Time
> elapsed: 0.391 s <<< FAILURE!
> org.opentest4j.AssertionFailedError: Processor appears to be valid but
> expected it to be invalid ==> expected: but was: {code}
> *Steps to reproduce the failure:*
> 1.intall [Nondex|https://github.com/TestingResearchIllinois/NonDex]
> 2.Run the command in nifi:
> {code:java}
> mvn install -pl
> nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors -am
> -DskipTests
> mvn -pl nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors
> edu.illinois:nondex-maven-plugin:1.1.2:nondex
> -Dtest=org.apache.nifi.processors.standard.TestParseCEF#testCustomValidator
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi] exceptionfactory closed pull request #6754: NIFI-10637 Fixing Flaky Test In TestGenerateTableFetch
exceptionfactory closed pull request #6754: NIFI-10637 Fixing Flaky Test In TestGenerateTableFetch URL: https://github.com/apache/nifi/pull/6754 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on a diff in pull request #6480: NIFI-10585: Add GenerateFakeRecord processor
exceptionfactory commented on code in PR #6480:
URL: https://github.com/apache/nifi/pull/6480#discussion_r1044616560
##
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/GenerateFakeRecord.java:
##
@@ -0,0 +1,1021 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+import com.github.javafaker.Faker;
+import org.apache.avro.Schema;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.avro.AvroSchemaValidator;
+import org.apache.nifi.avro.AvroTypeUtil;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.expression.ExpressionLanguageScope;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.schema.access.SchemaNotFoundException;
+import org.apache.nifi.serialization.RecordSetWriter;
+import org.apache.nifi.serialization.RecordSetWriterFactory;
+import org.apache.nifi.serialization.SimpleRecordSchema;
+import org.apache.nifi.serialization.WriteResult;
+import org.apache.nifi.serialization.record.DataType;
+import org.apache.nifi.serialization.record.MapRecord;
+import org.apache.nifi.serialization.record.Record;
+import org.apache.nifi.serialization.record.RecordField;
+import org.apache.nifi.serialization.record.RecordFieldType;
+import org.apache.nifi.serialization.record.RecordSchema;
+import org.apache.nifi.serialization.record.type.ArrayDataType;
+import org.apache.nifi.serialization.record.type.ChoiceDataType;
+import org.apache.nifi.serialization.record.type.DecimalDataType;
+import org.apache.nifi.serialization.record.type.EnumDataType;
+import org.apache.nifi.serialization.record.type.MapDataType;
+import org.apache.nifi.serialization.record.type.RecordDataType;
+import org.apache.nifi.util.StringUtils;
+
+import java.math.BigInteger;
+import java.text.DateFormat;
+import java.text.SimpleDateFormat;
+import java.util.ArrayList;
+import java.util.Date;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.stream.Collectors;
+
+@SupportsBatching
+@Tags({"test", "random", "generate", "fake"})
+@InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN)
+@WritesAttributes({
+@WritesAttribute(attribute = "mime.type", description = "Sets the
mime.type attribute to the MIME Type specified by the Record Writer"),
+@WritesAttribute(attribute = "record.count", description = "The number
of records in the FlowFile"),
+})
+@CapabilityDescription("This processor creates FlowFiles with records having
random value for the specified fields. GenerateFakeRecord is useful " +
+"for testing, configuration, and simulation. It uses either
user-defined properties to define a record schema or a provided schema and
generates the specified number of records using " +
+"random data for the fields in the schema.")
+public class GenerateFakeRecord extends AbstractProcessor {
+
+static final AllowableValue FT_ADDRESS = new AllowableValue("Address",
"Address", "A full address including street, city, state, etc.");
+static final AllowableValue FT_AIRCRAFT = new AllowableValue("Aircraft",
"Aircraft", "The name of a
[GitHub] [nifi-minifi-cpp] arpadboda closed pull request #1470: MINIFICPP-2010 Disable systemd on non-linux
arpadboda closed pull request #1470: MINIFICPP-2010 Disable systemd on non-linux URL: https://github.com/apache/nifi-minifi-cpp/pull/1470 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] martinzink commented on a diff in pull request #1424: MINIFICPP-1862 use std::filesystem::path instead of std::string where appropriate
martinzink commented on code in PR #1424:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1424#discussion_r1044596070
##
extensions/sftp/tests/ListSFTPTests.cpp:
##
@@ -139,15 +139,8 @@ class ListSFTPTestsFixture {
std::fstream file;
std::stringstream ss;
ss << src_dir << "/vfs/" << relative_path;
-auto full_path = ss.str();
-std::deque parent_dirs;
-std::string parent_dir = full_path;
-while (!(parent_dir =
utils::file::FileUtils::get_parent_path(parent_dir)).empty()) {
- parent_dirs.push_front(parent_dir);
-}
-for (const auto& dir : parent_dirs) {
- utils::file::FileUtils::create_dir(dir);
-}
+std::filesystem::path full_path = ss.str();
Review Comment:
Good idea changed this in
[extensions/sftp/tests/ListSFTPTests.cpp](https://github.com/apache/nifi-minifi-cpp/pull/1424/commits/6c775d116236c92105c7be38bb284f309c27887c#diff-ac90fb77395d946dce409066d0015e9308fb4151efdb0d2cc146edc1abfe1552R155)
[extensions/sftp/tests/ListThenFetchSFTPTests.cpp](https://github.com/apache/nifi-minifi-cpp/pull/1424/commits/6c775d116236c92105c7be38bb284f309c27887c#diff-dfaf51b7f8862c8ffdda0f89bd11d14b8fa83ef70ba0d9e5e686acb1afa51e5fR191)
##
extensions/standard-processors/tests/unit/ProcessorTests.cpp:
##
@@ -197,10 +196,8 @@ TEST_CASE("Test GetFile Ignore", "[getfileCreate3]") {
REQUIRE(record == nullptr);
REQUIRE(records.empty());
- const std::string hidden_file_name = [&] {
-std::stringstream ss;
-ss << dir << utils::file::FileUtils::get_separator() <<
".filewithoutanext";
-return ss.str();
+ const std::filesystem::path hidden_file_name = [&] {
+return dir / ".filewithoutanext";
}();
Review Comment:
:sweat_smile:
https://github.com/apache/nifi-minifi-cpp/pull/1424/commits/6c775d116236c92105c7be38bb284f309c27887c#diff-8caa2f65fe058d943caa4cd2c876226de0a63bea73ce1f7ffcb0550be1d7ed4fR199
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] martinzink commented on a diff in pull request #1424: MINIFICPP-1862 use std::filesystem::path instead of std::string where appropriate
martinzink commented on code in PR #1424:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1424#discussion_r1044593149
##
extensions/standard-processors/tests/unit/TailFileTests.cpp:
##
@@ -99,18 +89,16 @@ TEST_CASE("TailFile reads the file until the first
delimiter", "[simple]") {
plan->addProcessor("LogAttribute", "logattribute",
core::Relationship("success", "description"), true);
auto dir = testController.createTempDirectory();
- std::stringstream temp_file;
- temp_file << dir << utils::file::get_separator() << TMP_FILE;
+ auto temp_file_path = dir / TMP_FILE;
std::ofstream tmpfile;
- tmpfile.open(temp_file.str(), std::ios::out | std::ios::binary);
+ tmpfile.open(temp_file_path, std::ios::out | std::ios::binary);
tmpfile << NEWLINE_FILE;
tmpfile.close();
- std::stringstream state_file;
- state_file << dir << utils::file::get_separator() << STATE_FILE;
+ auto state_file_path = dir / STATE_FILE;
Review Comment:
you are right :+1:
https://github.com/apache/nifi-minifi-cpp/pull/1424/commits/6c775d116236c92105c7be38bb284f309c27887c#diff-0f2d0f56e01853fa6b0ed53098d1da12deaef25aca6b36c29ee8d3174a5556f5L99-L100
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] martinzink commented on a diff in pull request #1424: MINIFICPP-1862 use std::filesystem::path instead of std::string where appropriate
martinzink commented on code in PR #1424:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1424#discussion_r1044592592
##
libminifi/src/core/ProcessSessionReadCallback.cpp:
##
@@ -22,18 +22,19 @@
#include
#include
#include
+#include
#include "core/logging/LoggerConfiguration.h"
#include "utils/gsl.h"
namespace org::apache::nifi::minifi::core {
-ProcessSessionReadCallback::ProcessSessionReadCallback(const std::string
,
- std::string destFile,
+ProcessSessionReadCallback::ProcessSessionReadCallback(std::filesystem::path
tmpFile,
+ std::filesystem::path
destFile,
std::shared_ptr logger)
: logger_(std::move(logger)),
_tmpFileOs(tmpFile, std::ios::binary),
-_tmpFile(tmpFile),
+_tmpFile(std::move(tmpFile)),
_destFile(std::move(destFile)) {
Review Comment:
sure thing,
https://github.com/apache/nifi-minifi-cpp/pull/1424/commits/6c775d116236c92105c7be38bb284f309c27887c#diff-80c661dbc9b1b934d1581bb3e000acad44a92a93a18726f09b1a6f85bae27d38R32-R38
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] martinzink commented on a diff in pull request #1424: MINIFICPP-1862 use std::filesystem::path instead of std::string where appropriate
martinzink commented on code in PR #1424:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1424#discussion_r1044592071
##
libminifi/test/rocksdb-tests/EncryptionTests.cpp:
##
@@ -77,8 +75,8 @@ TEST_CASE_METHOD(FFRepoFixture, "FlowFileRepository creates
checkpoint and loads
// pass
}
SECTION("With encryption") {
-utils::file::FileUtils::create_dir((home_ / "conf").str());
-std::ofstream{(home_ / "conf" / "bootstrap.conf").str()}
+utils::file::FileUtils::create_dir((home_ / "conf"));
+std::ofstream{(home_ / "conf" / "bootstrap.conf")}
Review Comment:
you are right :+1:
https://github.com/apache/nifi-minifi-cpp/pull/1424/commits/6c775d116236c92105c7be38bb284f309c27887c#diff-ec160d83c141c2946b29793ff7e36a270fcd74259377bc1615e408b95ef70065R79
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] martinzink commented on a diff in pull request #1424: MINIFICPP-1862 use std::filesystem::path instead of std::string where appropriate
martinzink commented on code in PR #1424:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1424#discussion_r1044591718
##
libminifi/test/rocksdb-tests/RocksDBTests.cpp:
##
@@ -232,9 +231,9 @@ void
withDefaultEnv(minifi::internal::Writable& db_opts) {
}
TEST_CASE_METHOD(RocksDBTest, "Error is logged if different encryption keys
are used", "[rocksDBTest10]") {
- utils::Path home_dir{createTempDirectory()};
- utils::file::FileUtils::create_dir((home_dir / "conf").str());
- std::ofstream{(home_dir / "conf" / "bootstrap.conf").str()}
+ auto home_dir = createTempDirectory();
+ utils::file::FileUtils::create_dir((home_dir / "conf"));
+ std::ofstream{(home_dir / "conf" / "bootstrap.conf")}
Review Comment:
:+1: fixed in
https://github.com/apache/nifi-minifi-cpp/pull/1424/commits/6c775d116236c92105c7be38bb284f309c27887c#diff-8c2ecfdba8d618439b859597953f3011d805da3004421c49631fb947488b555fR236
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] martinzink commented on a diff in pull request #1424: MINIFICPP-1862 use std::filesystem::path instead of std::string where appropriate
martinzink commented on code in PR #1424:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1424#discussion_r1044590030
##
minifi_main/MainHelper.cpp:
##
@@ -133,43 +128,41 @@ std::string determineMinifiHome(const
std::shared_ptr& logger)
return "";
}();
- if (minifiHome.empty()) {
+ if (minifi_home.empty()) {
logger->log_error("No " MINIFI_HOME_ENV_KEY " could be inferred. "
"Please set " MINIFI_HOME_ENV_KEY " or run minifi from a
valid location.");
return "";
}
/* Verify that MINIFI_HOME is valid */
- bool minifiHomeValid = false;
- if (validHome(minifiHome)) {
-minifiHomeValid = true;
+ bool minifi_home_is_valid = false;
+ if (validHome(minifi_home)) {
+minifi_home_is_valid = true;
} else {
-logger->log_info("%s is not a valid " MINIFI_HOME_ENV_KEY ", because there
is no " DEFAULT_NIFI_PROPERTIES_FILE " file in it.", minifiHome);
-
-std::string minifiHomeWithoutBin;
-std::string binDir;
-std::tie(minifiHomeWithoutBin, binDir) =
minifi::utils::file::split_path(minifiHome);
-if (!minifiHomeWithoutBin.empty() && (binDir == "bin" || binDir ==
std::string("bin") + minifi::utils::file::get_separator())) {
- if (validHome(minifiHomeWithoutBin)) {
-logger->log_info("%s is a valid " MINIFI_HOME_ENV_KEY ", falling back
to it.", minifiHomeWithoutBin);
-minifiHomeValid = true;
-minifiHome = std::move(minifiHomeWithoutBin);
+logger->log_info("%s is not a valid %s, because there is no %s file in
it.", minifi_home.string(), MINIFI_HOME_ENV_KEY,
DEFAULT_NIFI_PROPERTIES_FILE.string());
+
+auto minifi_home_without_bin = minifi_home.parent_path();
+auto bin_dir = minifi_home.filename();
+if (!minifi_home_without_bin.empty() && (bin_dir ==
std::filesystem::path("bin") || bin_dir == (std::filesystem::path("bin")/""))) {
Review Comment:
makes sense, fixed it in
https://github.com/apache/nifi-minifi-cpp/pull/1424/commits/6c775d116236c92105c7be38bb284f309c27887c#diff-4ceccfbc1ea6c4b068f7f198d1fc53d5650ec2c057f2145e5bf41040325262c0R146
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] martinzink commented on a diff in pull request #1424: MINIFICPP-1862 use std::filesystem::path instead of std::string where appropriate
martinzink commented on code in PR #1424:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1424#discussion_r1044589812
##
minifi_main/MainHelper.cpp:
##
@@ -133,43 +128,41 @@ std::string determineMinifiHome(const
std::shared_ptr& logger)
return "";
}();
- if (minifiHome.empty()) {
+ if (minifi_home.empty()) {
logger->log_error("No " MINIFI_HOME_ENV_KEY " could be inferred. "
"Please set " MINIFI_HOME_ENV_KEY " or run minifi from a
valid location.");
return "";
}
/* Verify that MINIFI_HOME is valid */
- bool minifiHomeValid = false;
- if (validHome(minifiHome)) {
-minifiHomeValid = true;
+ bool minifi_home_is_valid = false;
+ if (validHome(minifi_home)) {
+minifi_home_is_valid = true;
} else {
-logger->log_info("%s is not a valid " MINIFI_HOME_ENV_KEY ", because there
is no " DEFAULT_NIFI_PROPERTIES_FILE " file in it.", minifiHome);
-
-std::string minifiHomeWithoutBin;
-std::string binDir;
-std::tie(minifiHomeWithoutBin, binDir) =
minifi::utils::file::split_path(minifiHome);
-if (!minifiHomeWithoutBin.empty() && (binDir == "bin" || binDir ==
std::string("bin") + minifi::utils::file::get_separator())) {
- if (validHome(minifiHomeWithoutBin)) {
-logger->log_info("%s is a valid " MINIFI_HOME_ENV_KEY ", falling back
to it.", minifiHomeWithoutBin);
-minifiHomeValid = true;
-minifiHome = std::move(minifiHomeWithoutBin);
+logger->log_info("%s is not a valid %s, because there is no %s file in
it.", minifi_home.string(), MINIFI_HOME_ENV_KEY,
DEFAULT_NIFI_PROPERTIES_FILE.string());
+
+auto minifi_home_without_bin = minifi_home.parent_path();
+auto bin_dir = minifi_home.filename();
+if (!minifi_home_without_bin.empty() && (bin_dir ==
std::filesystem::path("bin") || bin_dir == (std::filesystem::path("bin")/""))) {
+ if (validHome(minifi_home_without_bin)) {
+logger->log_info("%s is a valid %s, falling back to it.",
minifi_home_without_bin.string());
Review Comment:
good catch :+1: , fixed it in
https://github.com/apache/nifi-minifi-cpp/pull/1424/commits/6c775d116236c92105c7be38bb284f309c27887c#diff-4ceccfbc1ea6c4b068f7f198d1fc53d5650ec2c057f2145e5bf41040325262c0R148
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] martinzink commented on a diff in pull request #1424: MINIFICPP-1862 use std::filesystem::path instead of std::string where appropriate
martinzink commented on code in PR #1424:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1424#discussion_r1044587720
##
libminifi/test/unit/FileStreamTests.cpp:
##
@@ -301,32 +267,25 @@ TEST_CASE("Existing file read/write test") {
stream.seek(0);
}
-#if !defined(WIN32) || defined(USE_BOOST)
-// This could be simplified with C++17 std::filesystem
TEST_CASE("Opening file without permission creates error logs") {
TestController test_controller;
auto dir = test_controller.createTempDirectory();
- std::string path_to_permissionless_file(utils::file::concat_path(dir,
"permissionless_file.txt"));
+ auto path_to_permissionless_file = dir / "permissionless_file.txt";
{
std::ofstream outfile(path_to_permissionless_file);
outfile << "this file has been just created" << std::endl;
outfile.close();
-#ifndef WIN32
utils::file::FileUtils::set_permissions(path_to_permissionless_file, 0);
-#else
-boost::filesystem::permissions(path_to_permissionless_file,
boost::filesystem::no_perms);
-#endif
}
- minifi::io::FileStream stream(path_to_permissionless_file, 0, false);
+ minifi::io::FileStream stream(path_to_permissionless_file, 0, true);
Review Comment:
It never worked on windows (with boost enabled), this is due to the fact
that the test and (iirc filesystem::permissions aswell) only sets the old
windows permissions (which doesnt have the concept of an unreadable file) so
setting a file to permission only sets the read-only flag on windows.
So for the FileStream to fail due to permission issues we must try to write
to it on windows.
Maybe we could test the newer ACL permission model on windows aswell, but to
be honest I am not sure if that worth the effort.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (MINIFICPP-2010) The systemd extension shouldn't be available on mac
[ https://issues.apache.org/jira/browse/MINIFICPP-2010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz updated MINIFICPP-2010: Affects Version/s: 0.13.0 > The systemd extension shouldn't be available on mac > --- > > Key: MINIFICPP-2010 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2010 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Affects Versions: 0.13.0 >Reporter: Marton Szasz >Assignee: Marton Szasz >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2010) The systemd extension shouldn't be available on mac
[ https://issues.apache.org/jira/browse/MINIFICPP-2010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz updated MINIFICPP-2010: Fix Version/s: 0.14.0 > The systemd extension shouldn't be available on mac > --- > > Key: MINIFICPP-2010 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2010 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Affects Versions: 0.13.0 >Reporter: Marton Szasz >Assignee: Marton Szasz >Priority: Major > Fix For: 0.14.0 > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2010) The systemd extension shouldn't be available on mac
[ https://issues.apache.org/jira/browse/MINIFICPP-2010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz updated MINIFICPP-2010: Status: Patch Available (was: Open) https://github.com/apache/nifi-minifi-cpp/pull/1470 > The systemd extension shouldn't be available on mac > --- > > Key: MINIFICPP-2010 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2010 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Marton Szasz >Assignee: Marton Szasz >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi-minifi-cpp] szaszm opened a new pull request, #1470: MINIFICPP-2010 Disable systemd on non-linux
szaszm opened a new pull request, #1470: URL: https://github.com/apache/nifi-minifi-cpp/pull/1470 Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with MINIFICPP- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically main)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [x] If applicable, have you updated the LICENSE file? - [x] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [x] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check GitHub Actions CI results for build issues and submit an update to your PR as soon as possible. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] martinzink commented on a diff in pull request #1424: MINIFICPP-1862 use std::filesystem::path instead of std::string where appropriate
martinzink commented on code in PR #1424:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1424#discussion_r1044576534
##
libminifi/include/core/ConfigurableComponent.h:
##
@@ -59,6 +59,18 @@ class ConfigurableComponent {
template
bool getProperty(const std::string name, T ) const;
+ template
+ std::enable_if_t::value, std::optional>
+ getProperty(const std::string& property_name) const {
Review Comment:
It would be nicer I agree, I tried
https://github.com/apache/nifi-minifi-cpp/pull/1424/commits/6c775d116236c92105c7be38bb284f309c27887c#diff-15b7e52fd974b055c6f71955af2eb1742390e7916dd98c549b9031a9c4b77e9bR62-R63
but it turns out concepts on apple clang are only partially supported (source:
https://en.cppreference.com/w/cpp/compiler_support), [here is the failed
CI](https://github.com/martinzink/nifi-minifi-cpp/actions/runs/3647371508/jobs/6159525208)
maybe with some time we could make it work :man_shrugging: , but I dont have
macOS workstation and didnt want to put too much effort into this so I just
reverted the change in
https://github.com/apache/nifi-minifi-cpp/pull/1424/commits/a1a262a18fbfbdfd6cd0b3c6bf0dd53e94f4af87
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Created] (MINIFICPP-2010) The systemd extension shouldn't be available on mac
Marton Szasz created MINIFICPP-2010: --- Summary: The systemd extension shouldn't be available on mac Key: MINIFICPP-2010 URL: https://issues.apache.org/jira/browse/MINIFICPP-2010 Project: Apache NiFi MiNiFi C++ Issue Type: Bug Reporter: Marton Szasz Assignee: Marton Szasz -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi-minifi-cpp] martinzink commented on a diff in pull request #1424: MINIFICPP-1862 use std::filesystem::path instead of std::string where appropriate
martinzink commented on code in PR #1424:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1424#discussion_r1044570304
##
extensions/standard-processors/tests/unit/GenerateFlowFileTests.cpp:
##
@@ -57,9 +57,9 @@ TEST_CASE("GenerateFlowFileTest", "[generateflowfiletest]") {
std::vector file_contents;
- auto lambda = [_contents](const std::string& path, const std::string&
filename) -> bool {
-std::ifstream is(path + utils::file::get_separator() + filename,
std::ifstream::binary);
-file_contents.push_back(std::string((std::istreambuf_iterator(is)),
std::istreambuf_iterator()));
+ auto lambda = [_contents](const std::filesystem::path& path, const
std::filesystem::path& filename) -> bool {
+std::ifstream is(path / filename, std::ifstream::binary);
+file_contents.emplace_back((std::istreambuf_iterator(is)),
std::istreambuf_iterator());
Review Comment:
fixed them in
https://github.com/apache/nifi-minifi-cpp/pull/1424/commits/6c775d116236c92105c7be38bb284f309c27887c#diff-5aa458ae662cf99a08e2c71b3db9668cff5b9343a864e20a16b19c37388e7bf6R62
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] martinzink commented on a diff in pull request #1424: MINIFICPP-1862 use std::filesystem::path instead of std::string where appropriate
martinzink commented on code in PR #1424:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1424#discussion_r1044569637
##
extensions/standard-processors/processors/GetFile.cpp:
##
@@ -154,36 +154,32 @@ void GetFile::onTrigger(core::ProcessContext*
/*context*/, core::ProcessSession*
return;
}
- std::queue list_of_file_names = pollListing(request_.batchSize);
+ std::queue list_of_file_names =
pollListing(request_.batchSize);
while (!list_of_file_names.empty()) {
-std::string file_name = list_of_file_names.front();
+auto file_name = list_of_file_names.front();
list_of_file_names.pop();
getSingleFile(*session, file_name);
}
}
-void GetFile::getSingleFile(core::ProcessSession& session, const std::string&
file_name) const {
- logger_->log_info("GetFile process %s", file_name);
+void GetFile::getSingleFile(core::ProcessSession& session, const
std::filesystem::path& file_path) const {
+ logger_->log_info("GetFile process %s", file_path.string());
auto flow_file = session.create();
gsl_Expects(flow_file);
- std::string path;
- std::string name;
- std::tie(path, name) = utils::file::split_path(file_name);
- flow_file->setAttribute(core::SpecialFlowAttribute::FILENAME, name);
- flow_file->setAttribute(core::SpecialFlowAttribute::PATH, path);
- flow_file->addAttribute(core::SpecialFlowAttribute::ABSOLUTE_PATH,
file_name);
+ flow_file->setAttribute(core::SpecialFlowAttribute::FILENAME,
file_path.filename().string());
+ flow_file->setAttribute(core::SpecialFlowAttribute::PATH,
(file_path.parent_path() / "").string());
+ flow_file->addAttribute(core::SpecialFlowAttribute::ABSOLUTE_PATH,
file_path.string());
try {
-session.write(flow_file, utils::FileReaderCallback{file_name});
+session.write(flow_file, utils::FileReaderCallback{file_path});
session.transfer(flow_file, Success);
if (!request_.keepSourceFile) {
- auto remove_status = remove(file_name.c_str());
- if (remove_status != 0) {
-logger_->log_error("GetFile could not delete file '%s', error %d: %s",
file_name, errno, strerror(errno));
+ if (!std::filesystem::remove(file_path)) {
+logger_->log_error("GetFile could not delete file '%s', error %d: %s",
file_path.string(), errno, strerror(errno));
Review Comment:
Nope, that wasnt intentional, good thing you caught it. Fixed it in
https://github.com/apache/nifi-minifi-cpp/pull/1424/commits/6c775d116236c92105c7be38bb284f309c27887c#diff-3f192172f343b26308cbdbb6f28836bcd8ca11f0484d3aa95450dde45970fa1fR175-R177
##
extensions/standard-processors/processors/TailFile.cpp:
##
@@ -530,21 +530,20 @@ bool
TailFile::getStateFromStateManager(std::map _ta
readOptionalInt64(state_map, "file." + std::to_string(i) +
".last_read_time")
}};
-std::string fileLocation;
-std::string fileName;
-if (utils::file::getFileNameAndPath(current, fileLocation, fileName)) {
- logger_->log_debug("Received path %s, file %s", fileLocation,
fileName);
- new_tail_states.emplace(current, TailState{fileLocation, fileName,
position, last_read_time, checksum});
+std::filesystem::path file_path = current;
+if (file_path.has_filename() && file_path.has_parent_path()) {
+ logger_->log_debug("Received path %s, file %s",
file_path.parent_path().string(), file_path.filename().string());
+ new_tail_states.emplace(current, TailState{file_path.parent_path(),
file_path.filename(), position, last_read_time, checksum});
} else {
- new_tail_states.emplace(current, TailState{fileLocation, current,
position, last_read_time, checksum});
+ new_tail_states.emplace(current, TailState{file_path.parent_path(),
current, position, last_read_time, checksum});
Review Comment:
fixed it in
https://github.com/apache/nifi-minifi-cpp/pull/1424/commits/6c775d116236c92105c7be38bb284f309c27887c#diff-16a8df4d25f85ca2144b3e61d418c99e06ffc578461f288db8d32cd02d45307dR550
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] martinzink commented on a diff in pull request #1424: MINIFICPP-1862 use std::filesystem::path instead of std::string where appropriate
martinzink commented on code in PR #1424: URL: https://github.com/apache/nifi-minifi-cpp/pull/1424#discussion_r1044566428 ## extensions/sftp/processors/FetchSFTP.cpp: ## @@ -87,12 +87,11 @@ void FetchSFTP::onTrigger(const std::shared_ptr , return; } - /* Parse processor-specific properties */ - std::string remote_file; - std::string move_destination_directory; - - context->getProperty(RemoteFile, remote_file, flow_file); - context->getProperty(MoveDestinationDirectory, move_destination_directory, flow_file); + std::string path_str; + context->getProperty(RemoteFile, path_str, flow_file); + auto remote_file = std::filesystem::path(path_str, std::filesystem::path::format::generic_format); + context->getProperty(MoveDestinationDirectory, path_str, flow_file); Review Comment: great catch, reworked this in https://github.com/apache/nifi-minifi-cpp/pull/1424/commits/6c775d116236c92105c7be38bb284f309c27887c#diff-1681c2dbfafc5cbbb437de97c560fabdb639b547d6a368f3e65c8e6c5c03d3ecR90-R98 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] martinzink commented on a diff in pull request #1424: MINIFICPP-1862 use std::filesystem::path instead of std::string where appropriate
martinzink commented on code in PR #1424:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1424#discussion_r1044565861
##
extensions/script/tests/TestExecuteScriptProcessorWithPythonScript.cpp:
##
@@ -295,124 +291,119 @@ TEST_CASE("Python: Test Update Attribute",
"[executescriptPythonUpdateAttribute]
session.transfer(flow_file, REL_SUCCESS)
)");
- auto getFileDir = testController.createTempDirectory();
- plan->setProperty(getFile, minifi::processors::GetFile::Directory.getName(),
getFileDir);
+ auto get_file_dir = test_controller.createTempDirectory();
+ plan->setProperty(getFile, minifi::processors::GetFile::Directory.getName(),
get_file_dir.string());
- utils::putFileToDir(getFileDir, "tstFile.ext", "tempFile");
+ utils::putFileToDir(get_file_dir, "tstFile.ext", "tempFile");
- testController.runSession(plan, false);
- testController.runSession(plan, false);
- testController.runSession(plan, false);
+ test_controller.runSession(plan, false);
+ test_controller.runSession(plan, false);
+ test_controller.runSession(plan, false);
REQUIRE(LogTestController::getInstance().contains("key:test_attr value:2"));
- logTestController.reset();
+ log_test_controller.reset();
}
TEST_CASE("Python: Test Get Context Property",
"[executescriptPythonGetContextProperty]") {
- TestController testController;
+ TestController test_controller;
- LogTestController = LogTestController::getInstance();
- logTestController.setDebug();
- logTestController.setDebug();
- logTestController.setDebug();
+ LogTestController& log_test_controller = LogTestController::getInstance();
+ log_test_controller.setDebug();
+ log_test_controller.setDebug();
+ log_test_controller.setDebug();
- auto plan = testController.createPlan();
+ auto plan = test_controller.createPlan();
- auto getFile = plan->addProcessor("GetFile", "getFile");
- auto executeScript = plan->addProcessor("ExecuteScript",
+ auto get_file = plan->addProcessor("GetFile", "getFile");
+ auto execute_script = plan->addProcessor("ExecuteScript",
"executeScript",
core::Relationship("success",
"description"),
true);
- auto logAttribute = plan->addProcessor("LogAttribute", "logAttribute",
+ auto log_attribute = plan->addProcessor("LogAttribute", "logAttribute",
core::Relationship("success",
"description"),
true);
- plan->setProperty(executeScript,
minifi::processors::ExecuteScript::ScriptBody.getName(), R"(
+ plan->setProperty(execute_script,
minifi::processors::ExecuteScript::ScriptBody.getName(), R"(
def onTrigger(context, session):
script_engine = context.getProperty('Script Engine')
log.info('got Script Engine property: %s' % script_engine)
)");
- auto getFileDir = testController.createTempDirectory();
- plan->setProperty(getFile, minifi::processors::GetFile::Directory.getName(),
getFileDir);
+ auto get_file_dir = test_controller.createTempDirectory();
+ plan->setProperty(get_file,
minifi::processors::GetFile::Directory.getName(), get_file_dir.string());
- utils::putFileToDir(getFileDir, "tstFile.ext", "tempFile");
+ utils::putFileToDir(get_file_dir, "tstFile.ext", "tempFile");
- testController.runSession(plan, false);
- testController.runSession(plan, false);
- testController.runSession(plan, false);
+ test_controller.runSession(plan, false);
+ test_controller.runSession(plan, false);
+ test_controller.runSession(plan, false);
REQUIRE(LogTestController::getInstance().contains("[info] got Script Engine
property: python"));
- logTestController.reset();
+ log_test_controller.reset();
}
TEST_CASE("Python: Test Module Directory property",
"[executescriptPythonModuleDirectoryProperty]") {
- using org::apache::nifi::minifi::utils::file::concat_path;
using org::apache::nifi::minifi::utils::file::get_executable_dir;
- TestController testController;
+ TestController test_controller;
- LogTestController = LogTestController::getInstance();
- logTestController.setDebug();
- logTestController.setDebug();
+ LogTestController& log_test_controller = LogTestController::getInstance();
+ log_test_controller.setDebug();
+ log_test_controller.setDebug();
- std::string path;
- std::string filename;
- minifi::utils::file::getFileNameAndPath(__FILE__, path, filename);
- const std::string SCRIPT_FILES_DIRECTORY =
minifi::utils::file::getFullPath(concat_path(path, "test_python_scripts"));
+ const auto script_files_directory =
std::filesystem::path(__FILE__).parent_path() / "test_python_scripts";
- auto getScriptFullPath = [_FILES_DIRECTORY](const std::string&
script_file_name) {
-return concat_path(SCRIPT_FILES_DIRECTORY, script_file_name);
- };
+ auto plan = test_controller.createPlan();
- auto plan =
[GitHub] [nifi-minifi-cpp] martinzink commented on a diff in pull request #1424: MINIFICPP-1862 use std::filesystem::path instead of std::string where appropriate
martinzink commented on code in PR #1424:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1424#discussion_r1044561423
##
extensions/script/tests/TestExecuteScriptProcessorWithLuaScript.cpp:
##
@@ -421,8 +412,10 @@ TEST_CASE("Lua: Test Module Directory property",
"[executescriptLuaModuleDirecto
true);
plan->setProperty(executeScript,
minifi::processors::ExecuteScript::ScriptEngine.getName(), "lua");
- plan->setProperty(executeScript,
minifi::processors::ExecuteScript::ScriptFile.getName(),
getScriptFullPath("foo_bar_processor.lua"));
- plan->setProperty(executeScript,
minifi::processors::ExecuteScript::ModuleDirectory.getName(),
getScriptFullPath("foo_modules/foo.lua") + "," +
getScriptFullPath("bar_modules"));
+ plan->setProperty(executeScript,
minifi::processors::ExecuteScript::ScriptFile.getName(), script_files_directory
/ "foo_bar_processor.lua");
+ plan->setProperty(executeScript,
+
minifi::processors::ExecuteScript::ModuleDirectory.getName(),
+(script_files_directory / "foo_modules/foo.lua").string()
+ "," + (script_files_directory / "bar_modules").string());
Review Comment:
Looks like this was fixed in
https://github.com/apache/nifi-minifi-cpp/commit/9a3616ff1557eb9d35a8a3c477b8ae6954808021#diff-8838aae70901899c9da68b92dbe0a0209a9aaeb6565af72cefbe088b6a377f51R422
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] martinzink commented on a diff in pull request #1424: MINIFICPP-1862 use std::filesystem::path instead of std::string where appropriate
martinzink commented on code in PR #1424:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1424#discussion_r1044558281
##
extensions/script/python/PythonCreator.h:
##
@@ -140,26 +136,18 @@ class PythonCreator : public minifi::core::CoreComponent {
return python_package;
}
- std::string getPath(const std::string ) {
-return std::filesystem::path(pythonscript).parent_path().string();
+ std::filesystem::path getFileName(const std::filesystem::path&
python_script) {
+return std::filesystem::path(python_script).filename();
Review Comment:
:+1: good idea, went with the later approach
https://github.com/apache/nifi-minifi-cpp/pull/1424/commits/6c775d116236c92105c7be38bb284f309c27887c#diff-0a5612bbb1831ede9a5b11ebe105ddb1fd97b399d44f51c34cebb87832769145R66
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] martinzink commented on a diff in pull request #1424: MINIFICPP-1862 use std::filesystem::path instead of std::string where appropriate
martinzink commented on code in PR #1424:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1424#discussion_r1044557769
##
extensions/jni/jvm/JniReferenceObjects.h:
##
@@ -244,23 +228,24 @@ class JniSession : public core::WeakReference {
}
bool prune() {
-global_ff_objects_.erase(std::remove_if(global_ff_objects_.begin(),
global_ff_objects_.end(), check_empty_ff()), global_ff_objects_.end());
+const auto check_empty_ff = [](const std::shared_ptr&
flow_file) { return flow_file->empty(); };
+global_ff_objects_.erase(std::remove_if(global_ff_objects_.begin(),
global_ff_objects_.end(), check_empty_ff), global_ff_objects_.end());
Review Comment:
good idea, changed it in
https://github.com/apache/nifi-minifi-cpp/pull/1424/commits/6c775d116236c92105c7be38bb284f309c27887c#diff-a981b563c7f8e990402f120b51a94b4330574c37633c40b192bf944e69a95af0R232
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] martinzink commented on a diff in pull request #1424: MINIFICPP-1862 use std::filesystem::path instead of std::string where appropriate
martinzink commented on code in PR #1424:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1424#discussion_r1044557458
##
extensions/http-curl/tests/C2VerifyServeResults.cpp:
##
@@ -82,8 +82,8 @@ class VerifyC2Server : public HTTPIntegrationBase {
}
protected:
- std::string dir;
- std::stringstream ss;
+ std::filesystem::path dir_;
+ std::filesystem::path path_;
Review Comment:
good idea, changed it in
[extensions/http-curl/tests/C2VerifyServeResults.cpp](https://github.com/apache/nifi-minifi-cpp/pull/1424/commits/6c775d116236c92105c7be38bb284f309c27887c#diff-69e06ea0333f54d2cc4e70adc182466ab3a05c7aca36ed64b973bb31c6c042b8R86)
[extensions/http-curl/tests/HttpPostIntegrationTest.cpp](https://github.com/apache/nifi-minifi-cpp/pull/1424/commits/6c775d116236c92105c7be38bb284f309c27887c#diff-1c8f5d88f728bfc1c2ef82eed5ef340df4422e4b2d3365f33a0725577db754edR82)
[extensions/http-curl/tests/SiteToSiteRestTest.cpp](https://github.com/apache/nifi-minifi-cpp/pull/1424/commits/6c775d116236c92105c7be38bb284f309c27887c#diff-1fc35039eb1d5a8bb80ebe997ef71780d7752ac4933bd4f3cbe53de0df28dd01R104)
##
extensions/http-curl/tests/HttpPostIntegrationTest.cpp:
##
@@ -78,9 +78,9 @@ class HttpTestHarness : public HTTPIntegrationBase {
}
protected:
- std::string dir;
- std::stringstream ss;
- TestController testController;
+ std::filesystem::path dir_;
+ std::filesystem::path path_;
Review Comment:
good idea, changed it in
[extensions/http-curl/tests/C2VerifyServeResults.cpp](https://github.com/apache/nifi-minifi-cpp/pull/1424/commits/6c775d116236c92105c7be38bb284f309c27887c#diff-69e06ea0333f54d2cc4e70adc182466ab3a05c7aca36ed64b973bb31c6c042b8R86)
[extensions/http-curl/tests/HttpPostIntegrationTest.cpp](https://github.com/apache/nifi-minifi-cpp/pull/1424/commits/6c775d116236c92105c7be38bb284f309c27887c#diff-1c8f5d88f728bfc1c2ef82eed5ef340df4422e4b2d3365f33a0725577db754edR82)
[extensions/http-curl/tests/SiteToSiteRestTest.cpp](https://github.com/apache/nifi-minifi-cpp/pull/1424/commits/6c775d116236c92105c7be38bb284f309c27887c#diff-1fc35039eb1d5a8bb80ebe997ef71780d7752ac4933bd4f3cbe53de0df28dd01R104)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] martinzink commented on a diff in pull request #1424: MINIFICPP-1862 use std::filesystem::path instead of std::string where appropriate
martinzink commented on code in PR #1424:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1424#discussion_r1044555660
##
extensions/expression-language/tests/ExpressionLanguageTests.cpp:
##
@@ -218,7 +216,7 @@ TEST_CASE("GetFile PutFile dynamic attribute",
"[expressionLanguageTestGetFilePu
plan->setProperty(extract_text,
minifi::processors::ExtractText::Attribute.getName(), "extracted_attr_name");
plan->addProcessor("LogAttribute", "LogAttribute",
core::Relationship("success", "description"), true);
auto put_file = plan->addProcessor("PutFile", "PutFile",
core::Relationship("success", "description"), true);
- plan->setProperty(put_file,
minifi::processors::PutFile::Directory.getName(), out_dir +
"/${extracted_attr_name}");
+ plan->setProperty(put_file,
minifi::processors::PutFile::Directory.getName(), out_dir.string() +
"/${extracted_attr_name}");
Review Comment:
:+1: changed it in
https://github.com/apache/nifi-minifi-cpp/pull/1424/commits/6c775d116236c92105c7be38bb284f309c27887c#diff-f6418e5a4d678b633f2a69e64bc97df66407255f1ad86dd4e6052845b5e31b51R219
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] martinzink commented on a diff in pull request #1424: MINIFICPP-1862 use std::filesystem::path instead of std::string where appropriate
martinzink commented on code in PR #1424: URL: https://github.com/apache/nifi-minifi-cpp/pull/1424#discussion_r1044555022 ## Windows.md: ## @@ -93,7 +93,7 @@ A basic working CMake configuration can be inferred from the `win_build_vs.bat`. ``` mkdir build cd build -cmake -G "Visual Studio 16 2019" -DINSTALLER_MERGE_MODULES=OFF -DENABLE_SQL=OFF -DCMAKE_BUILD_TYPE_INIT=Release -DCMAKE_BUILD_TYPE=Release -DWIN32=WIN32 -DENABLE_LIBRDKAFKA=OFF -DENABLE_JNI=OFF -DOPENSSL_OFF=OFF -DENABLE_COAP=OFF -DUSE_SHARED_LIBS=OFF -DDISABLE_CONTROLLER=ON -DBUILD_ROCKSDB=ON -DFORCE_WINDOWS=ON -DUSE_SYSTEM_UUID=OFF -DDISABLE_LIBARCHIVE=OFF -DDISABLE_SCRIPTING=ON -DEXCLUDE_BOOST=ON -DENABLE_WEL=TRUE -DFAIL_ON_WARNINGS=OFF -DSKIP_TESTS=OFF .. Review Comment: Good catch, removed it in https://github.com/apache/nifi-minifi-cpp/pull/1424/commits/6c775d116236c92105c7be38bb284f309c27887c#diff-cca060aa16d15639b5869bc21923825c304b01e4f22c181aaa0c087a586113f5R123 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] NissimShiman commented on pull request #6750: NIFI-10934 Adding uniqueness check for Regisrty Client creation
NissimShiman commented on PR #6750: URL: https://github.com/apache/nifi/pull/6750#issuecomment-1344334227 reviewing/testing pr... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on a diff in pull request #6589: NIFI-10710 implement processor for AWS Polly, Textract, Translate, Tr…
exceptionfactory commented on code in PR #6589:
URL: https://github.com/apache/nifi/pull/6589#discussion_r1044467172
##
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/ml/polly/GetAwsPollyJobStatus.java:
##
@@ -0,0 +1,115 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.aws.ml.polly;
+
+import com.amazonaws.ClientConfiguration;
+import com.amazonaws.auth.AWSCredentialsProvider;
+import com.amazonaws.services.polly.AmazonPollyClient;
+import com.amazonaws.services.polly.AmazonPollyClientBuilder;
+import com.amazonaws.services.polly.model.GetSpeechSynthesisTaskRequest;
+import com.amazonaws.services.polly.model.GetSpeechSynthesisTaskResult;
+import com.amazonaws.services.polly.model.TaskStatus;
+import com.amazonaws.services.textract.model.ThrottlingException;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.SeeAlso;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processors.aws.ml.AwsMachineLearningJobStatusProcessor;
+
+@Tags({"Amazon", "AWS", "ML", "Machine Learning", "Polly"})
+@CapabilityDescription("Retrieves the current status of an AWS Polly job.")
+@SeeAlso({StartAwsPollyJob.class})
+@WritesAttributes({
+@WritesAttribute(attribute = "PollyS3OutputBucket", description = "The
bucket name where polly output will be located."),
+@WritesAttribute(attribute = "PollyS3OutputKey", description = "Object
key of polly output."),
+@WritesAttribute(attribute = "outputLocation", description = "S3
path-style output location of the result.")
+})
+public class GetAwsPollyJobStatus extends
AwsMachineLearningJobStatusProcessor {
+private static final String BUCKET = "bucket";
+private static final String KEY = "key";
+private static final Pattern S3_PATH =
Pattern.compile("https://s3.*amazonaws.com/(?<" + BUCKET + ">[^/]+)/(?<" + KEY
+ ">.*)");
+private static final String AWS_S3_BUCKET = "PollyS3OutputBucket";
+private static final String AWS_S3_KEY = "PollyS3OutputKey";
Review Comment:
That's a very good point @KalmanJantner. In that case, keeping the
attributes sounds good. It might make sense to use the `filename` attribute for
the key portion in that case, since that is the default input value for
`FetchS3Object`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Assigned] (NIFI-9677) LookUpRecord record path evaluation is "breaking" the next evaluation in case data is missing
[
https://issues.apache.org/jira/browse/NIFI-9677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tamas Horvath reassigned NIFI-9677:
---
Assignee: (was: Tamas Horvath)
> LookUpRecord record path evaluation is "breaking" the next evaluation in case
> data is missing
> -
>
> Key: NIFI-9677
> URL: https://issues.apache.org/jira/browse/NIFI-9677
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.16.0
> Environment: Apache NiFi custom built from Github repo.
>Reporter: Peter Molnar
>Priority: Major
> Attachments: LookUpRecord_empty_array_data_issue.xml,
> image-2022-02-11-12-30-53-134.png, image-2022-02-11-12-32-01-833.png,
> image-2022-02-11-12-33-23-283.png
>
>
> Input JSON generated by GenerateFlowFile processor looks like this (actually
> I just added a currencies array under each record in addition to the "Record
> Update Strategy - Replace Existing Values" example here
> [https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.15.0/org.apache.nifi.processors.standard.LookupRecord/additionalDetails.html).]
> *Note:* for the first record currencies array is empty.
>
> {code:java}
> [
> {
> "locales": [
> {
> "region": "FR",
> "language": "fr"
> }, {
> "region": "US",
> "language": "en"
> }
> ],
> "currencies": []
> }, {
> "locales": [
> {
> "region": "CA",
> "language": "fr"
> },
> {
> "region": "JP",
> "language": "ja"
> }
> ],
> "currencies": [
> {
> "currency": "CAD"
> }, {
> "currency": "JPY"
> }
> ]
> }
> ]{code}
>
> SimpleKeyValueLookUp service contains the following values:
> !image-2022-02-11-12-33-23-283.png!
>
> LookUpRecord processor is configured as follows:
> !image-2022-02-11-12-30-53-134.png!
> Once I execute the LookUpRecord processor for the flow file, language look up
> works fine, but the look up for currencies and regions do not work.
>
> !image-2022-02-11-12-32-01-833.png!
> *Note:* in case the 1st currencies array is not empty but contains \{
> "currency": "EUR" }, \{ "currency": "USD" }, all look up works fine. But a
> missing data seems to break the next evaluation of the record path.
> Please find the template for reproducing the issue enclosed as
> "LookUpRecord_empty_array_data_issue.xml".
> Thank you.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (NIFI-10885) ConsumeMQTT should stop client threads
[
https://issues.apache.org/jira/browse/NIFI-10885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nandor Soma Abonyi reassigned NIFI-10885:
-
Assignee: Nandor Soma Abonyi
> ConsumeMQTT should stop client threads
> --
>
> Key: NIFI-10885
> URL: https://issues.apache.org/jira/browse/NIFI-10885
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.18.0
>Reporter: Peter Turcsanyi
>Assignee: Nandor Soma Abonyi
>Priority: Minor
>
> When {{ConsumeMQTT}} is stopped, some client threads keep running which
> prevents NiFi to stop cleanly.
> {code:java}
> 2022-11-28 13:14:01,126 INFO [main] org.apache.nifi.bootstrap.Command NiFi
> PID [13987] shutdown in progress...
> 2022-11-28 13:14:03,140 INFO [main] org.apache.nifi.bootstrap.Command NiFi
> PID [13987] shutdown in progress...
> 2022-11-28 13:14:05,152 INFO [main] org.apache.nifi.bootstrap.Command NiFi
> PID [13987] shutdown in progress...
> 2022-11-28 13:14:05,162 WARN [main] org.apache.nifi.bootstrap.Command NiFi
> PID [13987] shutdown not completed after 20 seconds: Killing process{code}
> It affects the v5 client version and may be an issue in the underlying client
> library.
> Suspicious threads:
> {code:java}
> "com.hivemq.client.mqtt-1-1@20668" prio=10 tid=0x69 nid=NA runnable
> java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.KQueue.poll(KQueue.java:-1)
> at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:122)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:129)
> - locked <0x522a> (a sun.nio.ch.KQueueSelectorImpl)
> - locked <0x522b> (a io.netty.channel.nio.SelectedSelectionKeySet)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
> at
> io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
> at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883)
> at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
> at
> io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at
> java.lang.Thread.run(Thread.java:833)"RxComputationThreadPool-1@20859" daemon
> prio=5 tid=0x6a nid=NA waiting
> java.lang.Thread.State: WAITING
> at jdk.internal.misc.Unsafe.park(Unsafe.java:-1)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:341)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block(AbstractQueuedSynchronizer.java:506)
> at
> java.util.concurrent.ForkJoinPool.unmanagedBlock(ForkJoinPool.java:3463)
> at
> java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3434)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1623)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1170)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1062)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1122)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
> at java.lang.Thread.run(Thread.java:833)"RxSchedulerPurge-1@20122"
> daemon prio=5 tid=0x67 nid=NA waiting
> java.lang.Thread.State: WAITING
> at jdk.internal.misc.Unsafe.park(Unsafe.java:-1)
> at
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:252)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1672)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1062)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1122)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
> at
> java.lang.Thread.run(Thread.java:833)"RxCachedWorkerPoolEvictor-1@20139"
> daemon prio=5 tid=0x68 nid=NA waiting
> java.lang.Thread.State: WAITING
> at jdk.internal.misc.Unsafe.park(Unsafe.java:-1)
> at
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:252)
> at
>
[jira] [Updated] (NIFI-10929) NiFi generated certificates (e.g. Single User, or nifi-toolkit) are not compatible with OpenSSL 3.x+
[
https://issues.apache.org/jira/browse/NIFI-10929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Sampson updated NIFI-10929:
-
Affects Version/s: 1.19.1
> NiFi generated certificates (e.g. Single User, or nifi-toolkit) are not
> compatible with OpenSSL 3.x+
>
>
> Key: NIFI-10929
> URL: https://issues.apache.org/jira/browse/NIFI-10929
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.19.0, 1.19.1
>Reporter: Chris Sampson
>Priority: Major
>
> The certificates (keystore, truststore) generated by NiFi are not compatible
> with OpenSSL 3+ because they are created using old/insecure algorithms.
> When starting NiFi 1.19.0 (using the {{apache/nifi}} Docker image) and
> confguring a single-user auth, which is the default for NiFi, the generated
> {{truststore.p12}} and {{keystore.p12}} cannot be decrypted (e.g. to extract
> the public cert/private key into PEM format files so they can be used with
> {{curl}}) using the OpenSSL version present within the Docker Image.
> {quote}
> OpenSSL 3.0.2 15 Mar 2022 (Library: OpenSSL 3.0.2 15 Mar 2022)
> {quote}
> Attempted command:
> {code:bash}
> openssl pkcs12 -in "truststore.p12" -out "nifi-cert.pem" -cacerts -nokeys
> -password "pass:${TRUSTSTORE_PASSWORD}"
> {code}
> Error:
> {quote}
> 4047C0AF997F:error:0308010C:digital envelope
> routines:inner_evp_generic_fetch:unsupported:../crypto/evp/evp_fetch.c:349:Global
> default library context, Algorithm (RC2-40-CBC : 0), Properties ()
> {quote}
> This can be worked around by adding the {{-legacy}} flag to the {{openssl}}
> command (see [this Stack Overflow
> answer|https://stackoverflow.com/a/72600724]).
> Using the command suggested from teh linked article, we see that the NiFi
> generated truststure uses:
> {quote}
> PKCS7 Encrypted data: pbeWithSHA1And40BitRC2-CBC, Iteration 51200
> {quote}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-10932) NiFi Toolkit CLI cannot connect to NiFi - trustAnchors parameter must be non-empty
[
https://issues.apache.org/jira/browse/NIFI-10932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Sampson updated NIFI-10932:
-
Affects Version/s: 1.19.1
> NiFi Toolkit CLI cannot connect to NiFi - trustAnchors parameter must be
> non-empty
> --
>
> Key: NIFI-10932
> URL: https://issues.apache.org/jira/browse/NIFI-10932
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.19.0, 1.19.1
>Reporter: Chris Sampson
>Priority: Major
>
> When running NiFi 1.19.0 using the {{apache/nifi}} docker image, the NiFi
> Toolkit is not able to connect to the running NiFi instance (with the default
> Single User Auth enabled).
> Having updated the NiFi Toolkit CLI properties file with the Truststore and
> Keystore details generated by NiFi (found within {{nifi.properties}}), then
> attempting to run a command using the NiFi Toolkit command line such as:
> {code:bash}
> /opt/nifi/nifi-toolkit-current/bin/cli.sh session set nifi
> /opt/nifi/nifi-toolkit-current/bin/cli.sh nifi get-services
> {code}
> The following error is returned:
> {quote}
> ERROR: Error executing command 'get-services' : Unexpected error:
> java.security.InvalidAlgorithmParameterException: the trustAnchors parameter
> must be non-empty
> {quote}
> This suggests something isn't working correctly with the NiFi Toolkit JVM
> process/configuration or such (e.g. the correct truststore is not being used,
> see https://www.baeldung.com/java-trustanchors-parameter-must-be-non-empty)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
