[jira] [Assigned] (NIFI-9812) nifi-framework-nar-loading-utils native lib test cases fail on ARM platforms
[
https://issues.apache.org/jira/browse/NIFI-9812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kevin Doran reassigned NIFI-9812:
-
Assignee: Kevin Doran
> nifi-framework-nar-loading-utils native lib test cases fail on ARM platforms
>
>
> Key: NIFI-9812
> URL: https://issues.apache.org/jira/browse/NIFI-9812
> Project: Apache NiFi
> Issue Type: Sub-task
>Reporter: Kevin Doran
>Assignee: Kevin Doran
>Priority: Minor
>
> Looks like the JNI/native-lib loader test cases needs some work for arm64,
> which makes sense.
> {noformat}
> [INFO] --< org.apache.nifi:nifi-framework-nar-loading-utils
> >--
> [INFO] Building nifi-framework-nar-loading-utils 1.16.0-SNAPSHOT
> [94/642]
> [INFO] [ jar
> ]-
> [INFO]
> [INFO] --- maven-enforcer-plugin:3.0.0:enforce (enforce-maven-version) @
> nifi-framework-nar-loading-utils ---
> [INFO]
> [INFO] --- maven-enforcer-plugin:3.0.0:enforce (enforce-java-version) @
> nifi-framework-nar-loading-utils ---
> [INFO]
> [INFO] --- buildnumber-maven-plugin:1.4:create (default) @
> nifi-framework-nar-loading-utils ---
> [INFO]
> [INFO] --- maven-remote-resources-plugin:1.7.0:process
> (process-resource-bundles) @ nifi-framework-nar-loading-utils ---
> [INFO] Preparing remote bundle org.apache:apache-jar-resource-bundle:1.4
> [INFO] Copying 3 resources from 1 bundle.
> [INFO]
> [INFO] --- maven-resources-plugin:3.2.0:resources (default-resources) @
> nifi-framework-nar-loading-utils ---
> [INFO] Using 'UTF-8' encoding to copy filtered resources.
> [INFO] Using 'UTF-8' encoding to copy filtered properties files.
> [INFO] skip non existing resourceDirectory
> /Users/kdoran/dev/code/nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-nar-loading-utils/src/main/resources
> [INFO] Copying 3 resources
> [INFO]
> [INFO] --- maven-compiler-plugin:3.9.0:compile (default-compile) @
> nifi-framework-nar-loading-utils ---
> [INFO] Nothing to compile - all classes are up to date
> [INFO]
> [INFO] --- maven-resources-plugin:3.2.0:testResources (default-testResources)
> @ nifi-framework-nar-loading-utils ---
> [INFO] Using 'UTF-8' encoding to copy filtered resources.
> [INFO] Using 'UTF-8' encoding to copy filtered properties files.
> [INFO] Copying 19 resources
> [INFO] Copying 3 resources
> [INFO]
> [INFO] --- maven-compiler-plugin:3.9.0:testCompile (default-testCompile) @
> nifi-framework-nar-loading-utils ---
> [INFO] Nothing to compile - all classes are up to date
> [INFO]
> [INFO] --- maven-compiler-plugin:3.9.0:testCompile (groovy-tests) @
> nifi-framework-nar-loading-utils ---
> [INFO] Changes detected - recompiling the module!
> [INFO] Nothing to compile - all classes are up to date
> [INFO]
> [INFO] --- maven-surefire-plugin:3.0.0-M5:test (default-test) @
> nifi-framework-nar-loading-utils ---
> [INFO]
> [INFO] ---
> [INFO] T E S T S
> [INFO] ---
> [INFO] Running org.apache.nifi.nar.TestLoadNativeLibViaSystemProperty
> [ERROR] Tests run: 2, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 0.101
> s <<< FAILURE! - in org.apache.nifi.nar.TestLoadNativeLibViaSystemProperty
> [ERROR]
> org.apache.nifi.nar.TestLoadNativeLibViaSystemProperty.testLoadSameLibraryBy2InstanceClassLoadersFromSystemProperty
> Time elapsed: 0.066 s <<< ERROR!
> java.lang.UnsatisfiedLinkError:
> /private/var/folders/dj/1c85sd0d6dvcp1fltmwr5nl4gn/T/901ce22d-a226-479b-baa1-b08aace5ec73_6588104464077247053_testjni:
>
> dlopen(/private/var/folders/dj/1c85sd0d6dvcp1fltmwr5nl4gn/T/901ce22d-a226-479b-baa1-b08aace5ec73_6588104464077247053_testjni,
> 0x0001): tried:
> '/private/var/folders/dj/1c85sd0d6dvcp1fltmwr5nl4gn/T/901ce22d-a226-479b-baa1-b08aace5ec73_6588104464077247053_testjni'
> (mach-o file, but is an incompatible architecture (have 'x86_64', need
> 'arm64e')),
> '/usr/lib/901ce22d-a226-479b-baa1-b08aace5ec73_6588104464077247053_testjni'
> (no such file)
> at
> org.apache.nifi.nar.TestLoadNativeLibViaSystemProperty.testLoadSameLibraryBy2InstanceClassLoadersFromSystemProperty(TestLoadNativeLibViaSystemProperty.java:139)
> [ERROR]
> org.apache.nifi.nar.TestLoadNativeLibViaSystemProperty.testLoadSameLibraryByNarClassLoaderFromSystemProperty
> Time elapsed: 0.021 s <<< ERROR!
> java.lang.UnsatisfiedLinkError:
> /private/var/folders/dj/1c85sd0d6dvcp1fltmwr5nl4gn/T/nifi-nar_without_native_lib-1-1.0.nar-unpacked_7297978890074099742_testjni:
>
> dlopen(/private/var/folders/dj/1c85sd0d6dvcp1fltmwr5nl4gn/T/nifi-nar_without_native_lib-1-1.0.nar-unpacked_7297978890074099742_testjni,
> 0x0001): tried:
>
[jira] [Updated] (NIFI-9812) nifi-framework-nar-loading-utils native lib test cases fail on ARM platforms
[
https://issues.apache.org/jira/browse/NIFI-9812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kevin Doran updated NIFI-9812:
--
Summary: nifi-framework-nar-loading-utils native lib test cases fail on ARM
platforms (was: nifi-framework-nar-loading-utils unable to load native libs on
ARM platforms)
> nifi-framework-nar-loading-utils native lib test cases fail on ARM platforms
>
>
> Key: NIFI-9812
> URL: https://issues.apache.org/jira/browse/NIFI-9812
> Project: Apache NiFi
> Issue Type: Sub-task
>Reporter: Kevin Doran
>Priority: Minor
>
> Looks like the JNI/native-lib loader test cases needs some work for arm64,
> which makes sense.
> {noformat}
> [INFO] --< org.apache.nifi:nifi-framework-nar-loading-utils
> >--
> [INFO] Building nifi-framework-nar-loading-utils 1.16.0-SNAPSHOT
> [94/642]
> [INFO] [ jar
> ]-
> [INFO]
> [INFO] --- maven-enforcer-plugin:3.0.0:enforce (enforce-maven-version) @
> nifi-framework-nar-loading-utils ---
> [INFO]
> [INFO] --- maven-enforcer-plugin:3.0.0:enforce (enforce-java-version) @
> nifi-framework-nar-loading-utils ---
> [INFO]
> [INFO] --- buildnumber-maven-plugin:1.4:create (default) @
> nifi-framework-nar-loading-utils ---
> [INFO]
> [INFO] --- maven-remote-resources-plugin:1.7.0:process
> (process-resource-bundles) @ nifi-framework-nar-loading-utils ---
> [INFO] Preparing remote bundle org.apache:apache-jar-resource-bundle:1.4
> [INFO] Copying 3 resources from 1 bundle.
> [INFO]
> [INFO] --- maven-resources-plugin:3.2.0:resources (default-resources) @
> nifi-framework-nar-loading-utils ---
> [INFO] Using 'UTF-8' encoding to copy filtered resources.
> [INFO] Using 'UTF-8' encoding to copy filtered properties files.
> [INFO] skip non existing resourceDirectory
> /Users/kdoran/dev/code/nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-nar-loading-utils/src/main/resources
> [INFO] Copying 3 resources
> [INFO]
> [INFO] --- maven-compiler-plugin:3.9.0:compile (default-compile) @
> nifi-framework-nar-loading-utils ---
> [INFO] Nothing to compile - all classes are up to date
> [INFO]
> [INFO] --- maven-resources-plugin:3.2.0:testResources (default-testResources)
> @ nifi-framework-nar-loading-utils ---
> [INFO] Using 'UTF-8' encoding to copy filtered resources.
> [INFO] Using 'UTF-8' encoding to copy filtered properties files.
> [INFO] Copying 19 resources
> [INFO] Copying 3 resources
> [INFO]
> [INFO] --- maven-compiler-plugin:3.9.0:testCompile (default-testCompile) @
> nifi-framework-nar-loading-utils ---
> [INFO] Nothing to compile - all classes are up to date
> [INFO]
> [INFO] --- maven-compiler-plugin:3.9.0:testCompile (groovy-tests) @
> nifi-framework-nar-loading-utils ---
> [INFO] Changes detected - recompiling the module!
> [INFO] Nothing to compile - all classes are up to date
> [INFO]
> [INFO] --- maven-surefire-plugin:3.0.0-M5:test (default-test) @
> nifi-framework-nar-loading-utils ---
> [INFO]
> [INFO] ---
> [INFO] T E S T S
> [INFO] ---
> [INFO] Running org.apache.nifi.nar.TestLoadNativeLibViaSystemProperty
> [ERROR] Tests run: 2, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 0.101
> s <<< FAILURE! - in org.apache.nifi.nar.TestLoadNativeLibViaSystemProperty
> [ERROR]
> org.apache.nifi.nar.TestLoadNativeLibViaSystemProperty.testLoadSameLibraryBy2InstanceClassLoadersFromSystemProperty
> Time elapsed: 0.066 s <<< ERROR!
> java.lang.UnsatisfiedLinkError:
> /private/var/folders/dj/1c85sd0d6dvcp1fltmwr5nl4gn/T/901ce22d-a226-479b-baa1-b08aace5ec73_6588104464077247053_testjni:
>
> dlopen(/private/var/folders/dj/1c85sd0d6dvcp1fltmwr5nl4gn/T/901ce22d-a226-479b-baa1-b08aace5ec73_6588104464077247053_testjni,
> 0x0001): tried:
> '/private/var/folders/dj/1c85sd0d6dvcp1fltmwr5nl4gn/T/901ce22d-a226-479b-baa1-b08aace5ec73_6588104464077247053_testjni'
> (mach-o file, but is an incompatible architecture (have 'x86_64', need
> 'arm64e')),
> '/usr/lib/901ce22d-a226-479b-baa1-b08aace5ec73_6588104464077247053_testjni'
> (no such file)
> at
> org.apache.nifi.nar.TestLoadNativeLibViaSystemProperty.testLoadSameLibraryBy2InstanceClassLoadersFromSystemProperty(TestLoadNativeLibViaSystemProperty.java:139)
> [ERROR]
> org.apache.nifi.nar.TestLoadNativeLibViaSystemProperty.testLoadSameLibraryByNarClassLoaderFromSystemProperty
> Time elapsed: 0.021 s <<< ERROR!
> java.lang.UnsatisfiedLinkError:
> /private/var/folders/dj/1c85sd0d6dvcp1fltmwr5nl4gn/T/nifi-nar_without_native_lib-1-1.0.nar-unpacked_7297978890074099742_testjni:
>
>
[jira] [Updated] (NIFI-9812) nifi-framework-nar-loading-utils unable to load native libs on ARM platforms
[
https://issues.apache.org/jira/browse/NIFI-9812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kevin Doran updated NIFI-9812:
--
Description:
Looks like the JNI/native-lib loader test cases needs some work for arm64,
which makes sense.
{noformat}
[INFO] --< org.apache.nifi:nifi-framework-nar-loading-utils >--
[INFO] Building nifi-framework-nar-loading-utils 1.16.0-SNAPSHOT [94/642]
[INFO] [ jar ]-
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0:enforce (enforce-maven-version) @
nifi-framework-nar-loading-utils ---
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0:enforce (enforce-java-version) @
nifi-framework-nar-loading-utils ---
[INFO]
[INFO] --- buildnumber-maven-plugin:1.4:create (default) @
nifi-framework-nar-loading-utils ---
[INFO]
[INFO] --- maven-remote-resources-plugin:1.7.0:process
(process-resource-bundles) @ nifi-framework-nar-loading-utils ---
[INFO] Preparing remote bundle org.apache:apache-jar-resource-bundle:1.4
[INFO] Copying 3 resources from 1 bundle.
[INFO]
[INFO] --- maven-resources-plugin:3.2.0:resources (default-resources) @
nifi-framework-nar-loading-utils ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Using 'UTF-8' encoding to copy filtered properties files.
[INFO] skip non existing resourceDirectory
/Users/kdoran/dev/code/nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-nar-loading-utils/src/main/resources
[INFO] Copying 3 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.9.0:compile (default-compile) @
nifi-framework-nar-loading-utils ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- maven-resources-plugin:3.2.0:testResources (default-testResources) @
nifi-framework-nar-loading-utils ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Using 'UTF-8' encoding to copy filtered properties files.
[INFO] Copying 19 resources
[INFO] Copying 3 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.9.0:testCompile (default-testCompile) @
nifi-framework-nar-loading-utils ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- maven-compiler-plugin:3.9.0:testCompile (groovy-tests) @
nifi-framework-nar-loading-utils ---
[INFO] Changes detected - recompiling the module!
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- maven-surefire-plugin:3.0.0-M5:test (default-test) @
nifi-framework-nar-loading-utils ---
[INFO]
[INFO] ---
[INFO] T E S T S
[INFO] ---
[INFO] Running org.apache.nifi.nar.TestLoadNativeLibViaSystemProperty
[ERROR] Tests run: 2, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 0.101 s
<<< FAILURE! - in org.apache.nifi.nar.TestLoadNativeLibViaSystemProperty
[ERROR]
org.apache.nifi.nar.TestLoadNativeLibViaSystemProperty.testLoadSameLibraryBy2InstanceClassLoadersFromSystemProperty
Time elapsed: 0.066 s <<< ERROR!
java.lang.UnsatisfiedLinkError:
/private/var/folders/dj/1c85sd0d6dvcp1fltmwr5nl4gn/T/901ce22d-a226-479b-baa1-b08aace5ec73_6588104464077247053_testjni:
dlopen(/private/var/folders/dj/1c85sd0d6dvcp1fltmwr5nl4gn/T/901ce22d-a226-479b-baa1-b08aace5ec73_6588104464077247053_testjni,
0x0001): tried:
'/private/var/folders/dj/1c85sd0d6dvcp1fltmwr5nl4gn/T/901ce22d-a226-479b-baa1-b08aace5ec73_6588104464077247053_testjni'
(mach-o file, but is an incompatible architecture (have 'x86_64', need
'arm64e')),
'/usr/lib/901ce22d-a226-479b-baa1-b08aace5ec73_6588104464077247053_testjni' (no
such file)
at
org.apache.nifi.nar.TestLoadNativeLibViaSystemProperty.testLoadSameLibraryBy2InstanceClassLoadersFromSystemProperty(TestLoadNativeLibViaSystemProperty.java:139)
[ERROR]
org.apache.nifi.nar.TestLoadNativeLibViaSystemProperty.testLoadSameLibraryByNarClassLoaderFromSystemProperty
Time elapsed: 0.021 s <<< ERROR!
java.lang.UnsatisfiedLinkError:
/private/var/folders/dj/1c85sd0d6dvcp1fltmwr5nl4gn/T/nifi-nar_without_native_lib-1-1.0.nar-unpacked_7297978890074099742_testjni:
dlopen(/private/var/folders/dj/1c85sd0d6dvcp1fltmwr5nl4gn/T/nifi-nar_without_native_lib-1-1.0.nar-unpacked_7297978890074099742_testjni,
0x0001): tried:
'/private/var/folders/dj/1c85sd0d6dvcp1fltmwr5nl4gn/T/nifi-nar_without_native_lib-1-1.0.nar-unpacked_7297978890074099742_testjni'
(mach-o file, but is an incompatible architecture (have 'x86_64', need
'arm64e')),
'/usr/lib/nifi-nar_without_native_lib-1-1.0.nar-unpacked_7297978890074099742_testjni'
(no such file)
at
org.apache.nifi.nar.TestLoadNativeLibViaSystemProperty.testLoadSameLibraryByNarClassLoaderFromSystemProperty(TestLoadNativeLibViaSystemProperty.java:97)
[INFO] Running org.apache.nifi.nar.TestNarLoader
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.063 s
- in
[jira] [Assigned] (NIFI-9811) QuestDB does not work on ARM
[
https://issues.apache.org/jira/browse/NIFI-9811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kevin Doran reassigned NIFI-9811:
-
Assignee: Kevin Doran
> QuestDB does not work on ARM
>
>
> Key: NIFI-9811
> URL: https://issues.apache.org/jira/browse/NIFI-9811
> Project: Apache NiFi
> Issue Type: Sub-task
>Reporter: Kevin Doran
>Assignee: Kevin Doran
>Priority: Major
>
> Judging from this open issue, it does not appear they will be adding support:
> https://github.com/questdb/questdb/issues/569
> so for ARM, we will need to exclude this or find an alternative.
> Here is test failure output showing the runtime issues:
> {noformat}
> [INFO] Running
> org.apache.nifi.controller.status.history.EmbeddedQuestDbStatusHistoryRepositoryForComponentsTest
> [ERROR] Tests run: 3, Failures: 3, Errors: 0, Skipped: 0, Time elapsed: 0.026
> s <<< FAILURE! - in
> org.apache.nifi.controller.status.history.EmbeddedQuestDbStatusHistoryRepositoryForComponentsTest
> [ERROR]
> org.apache.nifi.controller.status.history.EmbeddedQuestDbStatusHistoryRepositoryForComponentsTest.testReadingEmptyRepository
> Time elapsed: 0.025 s <<< FAILURE!
> org.opentest4j.MultipleFailuresError:
> Multiple Failures (2 failures)
> io.questdb.log.LogError: Error creating log writer
> java.lang.NullPointerException:
> [ERROR]
> org.apache.nifi.controller.status.history.EmbeddedQuestDbStatusHistoryRepositoryForComponentsTest.testWritingThenReadingComponents
> Time elapsed: 0.001 s <<< FAILURE!
> org.opentest4j.MultipleFailuresError:
> Multiple Failures (2 failures)
> java.lang.NoClassDefFoundError: Could not initialize class
> io.questdb.cutlass.text.types.InputFormatConfiguration
> java.lang.NullPointerException:
> [ERROR]
> org.apache.nifi.controller.status.history.EmbeddedQuestDbStatusHistoryRepositoryForComponentsTest.testReadingLimitedByPreferredDataPoints
> Time elapsed: 0 s <<< FAILURE!
> org.opentest4j.MultipleFailuresError:
> Multiple Failures (2 failures)
> java.lang.NoClassDefFoundError: Could not initialize class
> io.questdb.cutlass.text.types.InputFormatConfiguration
> java.lang.NullPointerException:
> [INFO] Running
> org.apache.nifi.controller.status.history.EmbeddedQuestDbStatusHistoryRepositoryForNodeTest
> [ERROR] Tests run: 3, Failures: 2, Errors: 0, Skipped: 1, Time elapsed: 0 s
> <<< FAILURE! - in
> org.apache.nifi.controller.status.history.EmbeddedQuestDbStatusHistoryRepositoryForNodeTest
> [ERROR]
> org.apache.nifi.controller.status.history.EmbeddedQuestDbStatusHistoryRepositoryForNodeTest.testReadingEmptyRepository
> Time elapsed: 0 s <<< FAILURE!
> org.opentest4j.MultipleFailuresError:
> Multiple Failures (2 failures)
> java.lang.NoClassDefFoundError: Could not initialize class
> io.questdb.cutlass.text.types.InputFormatConfiguration
> java.lang.NullPointerException:
> [ERROR]
> org.apache.nifi.controller.status.history.EmbeddedQuestDbStatusHistoryRepositoryForNodeTest.testWritingThenReadingComponents
> Time elapsed: 0 s <<< FAILURE!
> org.opentest4j.MultipleFailuresError:
> Multiple Failures (2 failures)
> java.lang.NoClassDefFoundError: Could not initialize class
> io.questdb.cutlass.text.types.InputFormatConfiguration
> java.lang.NullPointerException:
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (NIFI-9811) QuestDB does not work on ARM
[
https://issues.apache.org/jira/browse/NIFI-9811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kevin Doran resolved NIFI-9811.
---
Resolution: Fixed
Nothing to do here, as upgrading the quested dep version recently solved this
> QuestDB does not work on ARM
>
>
> Key: NIFI-9811
> URL: https://issues.apache.org/jira/browse/NIFI-9811
> Project: Apache NiFi
> Issue Type: Sub-task
>Reporter: Kevin Doran
>Priority: Major
>
> Judging from this open issue, it does not appear they will be adding support:
> https://github.com/questdb/questdb/issues/569
> so for ARM, we will need to exclude this or find an alternative.
> Here is test failure output showing the runtime issues:
> {noformat}
> [INFO] Running
> org.apache.nifi.controller.status.history.EmbeddedQuestDbStatusHistoryRepositoryForComponentsTest
> [ERROR] Tests run: 3, Failures: 3, Errors: 0, Skipped: 0, Time elapsed: 0.026
> s <<< FAILURE! - in
> org.apache.nifi.controller.status.history.EmbeddedQuestDbStatusHistoryRepositoryForComponentsTest
> [ERROR]
> org.apache.nifi.controller.status.history.EmbeddedQuestDbStatusHistoryRepositoryForComponentsTest.testReadingEmptyRepository
> Time elapsed: 0.025 s <<< FAILURE!
> org.opentest4j.MultipleFailuresError:
> Multiple Failures (2 failures)
> io.questdb.log.LogError: Error creating log writer
> java.lang.NullPointerException:
> [ERROR]
> org.apache.nifi.controller.status.history.EmbeddedQuestDbStatusHistoryRepositoryForComponentsTest.testWritingThenReadingComponents
> Time elapsed: 0.001 s <<< FAILURE!
> org.opentest4j.MultipleFailuresError:
> Multiple Failures (2 failures)
> java.lang.NoClassDefFoundError: Could not initialize class
> io.questdb.cutlass.text.types.InputFormatConfiguration
> java.lang.NullPointerException:
> [ERROR]
> org.apache.nifi.controller.status.history.EmbeddedQuestDbStatusHistoryRepositoryForComponentsTest.testReadingLimitedByPreferredDataPoints
> Time elapsed: 0 s <<< FAILURE!
> org.opentest4j.MultipleFailuresError:
> Multiple Failures (2 failures)
> java.lang.NoClassDefFoundError: Could not initialize class
> io.questdb.cutlass.text.types.InputFormatConfiguration
> java.lang.NullPointerException:
> [INFO] Running
> org.apache.nifi.controller.status.history.EmbeddedQuestDbStatusHistoryRepositoryForNodeTest
> [ERROR] Tests run: 3, Failures: 2, Errors: 0, Skipped: 1, Time elapsed: 0 s
> <<< FAILURE! - in
> org.apache.nifi.controller.status.history.EmbeddedQuestDbStatusHistoryRepositoryForNodeTest
> [ERROR]
> org.apache.nifi.controller.status.history.EmbeddedQuestDbStatusHistoryRepositoryForNodeTest.testReadingEmptyRepository
> Time elapsed: 0 s <<< FAILURE!
> org.opentest4j.MultipleFailuresError:
> Multiple Failures (2 failures)
> java.lang.NoClassDefFoundError: Could not initialize class
> io.questdb.cutlass.text.types.InputFormatConfiguration
> java.lang.NullPointerException:
> [ERROR]
> org.apache.nifi.controller.status.history.EmbeddedQuestDbStatusHistoryRepositoryForNodeTest.testWritingThenReadingComponents
> Time elapsed: 0 s <<< FAILURE!
> org.opentest4j.MultipleFailuresError:
> Multiple Failures (2 failures)
> java.lang.NoClassDefFoundError: Could not initialize class
> io.questdb.cutlass.text.types.InputFormatConfiguration
> java.lang.NullPointerException:
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-10235) Provenance replay fails when repository encryption is enabled
[
https://issues.apache.org/jira/browse/NIFI-10235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17567396#comment-17567396
]
David Handermann commented on NIFI-10235:
-
Thanks for reporting this issue and providing the detailed background
[~p-kimberley].
After reproducing the problem, the issue appears to be related with replay
events not accounting for the encryption metadata, indicated by the 379 byte
difference noted.
The Provenance Replay action builds a Content Claim based on the unencrypted
file size, which appears to be causing the problem. I will continue evaluating
the issue and follow up regarding a potential solution.
> Provenance replay fails when repository encryption is enabled
> -
>
> Key: NIFI-10235
> URL: https://issues.apache.org/jira/browse/NIFI-10235
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework, Security
>Affects Versions: 1.16.3
> Environment: RHEL 8.5
>Reporter: Peter Kimberley
>Priority: Major
> Labels: encryption, provenance, replay
> Attachments: NiFi_Flow.json, error-base-install.log, error.log
>
>
> h3. Problem summary
> When repository encryption is enabled, replaying a DROP provenance record
> fails, with the following error appearing in the logs:
> {quote}org.apache.nifi.processor.exception.FlowFileAccessException: Failed to
> export
> StandardFlowFileRecord[uuid=df985fc5-23da-4094-8783-2e0186bcb92d,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1657864218374-23, container=default,
> section=23], offset=379,
> length=1048576],offset=0,name=b29633c4-324e-42fe-b3e8-1ea455fc3650,size=1048576]
> to /opt/nifi/nifi-current/data/store/.b29633c4-324e-42fe-b3e8-1ea455fc3650
> due to java.io.EOFException: *Attempted to copy {color:#ff8b00}1048576{color}
> bytes but only {color:#ff8b00}1048197{color} bytes were available*
> {quote}
>
> I've observed that the difference between the sizes mentioned in the log is
> {+}*always 379 bytes*{+}, regardless of the length of the input file.
>
> With repository encryption disabled, provenance replay works as expected.
> h3. Configuration
> # NiFi v1.16.3 running as a three-node cluster in Kubernetes.
> # Each node has up to 8GB memory and 4 CPUs available to it.
> # Testing has included both NFS and ephemeral (emptyDir) storage.
> # The encryption key was generated by the following command, using the same
> JDK version:
> ## keytool -genseckey -alias key-1 -keyalg AES -keysize 256 -keystore
> repository.p12 -storetype PKCS12
> h4. nifi.properties
> {quote}nifi.repository.encryption.protocol.version=1
> nifi.repository.encryption.key.id=key-1
> nifi.repository.encryption.key.provider=KEYSTORE
> nifi.repository.encryption.key.provider.keystore.location=conf/repository.p12
> nifi.repository.encryption.key.provider.keystore.password=
> {quote}
> h3. Processor group
> GenerateFlowFile processor generating 1MB random files every second to a
> PutFile processor. Have also tested with InvokeHTTP.
> h3. Other comments
> With repository encryption enabled, I am able to download files via the
> provenance UI (suggesting that encryption/decryption works). The processor
> group also performs all other actions as expected.
> Not having the ability to replay provenance records is a blocker for our
> deployment, which requires data to be encrypted at rest and in transit.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (NIFI-10235) Provenance replay fails when repository encryption is enabled
[
https://issues.apache.org/jira/browse/NIFI-10235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann reassigned NIFI-10235:
---
Assignee: David Handermann
> Provenance replay fails when repository encryption is enabled
> -
>
> Key: NIFI-10235
> URL: https://issues.apache.org/jira/browse/NIFI-10235
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework, Security
>Affects Versions: 1.16.3
> Environment: RHEL 8.5
>Reporter: Peter Kimberley
>Assignee: David Handermann
>Priority: Major
> Labels: encryption, provenance, replay
> Attachments: NiFi_Flow.json, error-base-install.log, error.log
>
>
> h3. Problem summary
> When repository encryption is enabled, replaying a DROP provenance record
> fails, with the following error appearing in the logs:
> {quote}org.apache.nifi.processor.exception.FlowFileAccessException: Failed to
> export
> StandardFlowFileRecord[uuid=df985fc5-23da-4094-8783-2e0186bcb92d,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1657864218374-23, container=default,
> section=23], offset=379,
> length=1048576],offset=0,name=b29633c4-324e-42fe-b3e8-1ea455fc3650,size=1048576]
> to /opt/nifi/nifi-current/data/store/.b29633c4-324e-42fe-b3e8-1ea455fc3650
> due to java.io.EOFException: *Attempted to copy {color:#ff8b00}1048576{color}
> bytes but only {color:#ff8b00}1048197{color} bytes were available*
> {quote}
>
> I've observed that the difference between the sizes mentioned in the log is
> {+}*always 379 bytes*{+}, regardless of the length of the input file.
>
> With repository encryption disabled, provenance replay works as expected.
> h3. Configuration
> # NiFi v1.16.3 running as a three-node cluster in Kubernetes.
> # Each node has up to 8GB memory and 4 CPUs available to it.
> # Testing has included both NFS and ephemeral (emptyDir) storage.
> # The encryption key was generated by the following command, using the same
> JDK version:
> ## keytool -genseckey -alias key-1 -keyalg AES -keysize 256 -keystore
> repository.p12 -storetype PKCS12
> h4. nifi.properties
> {quote}nifi.repository.encryption.protocol.version=1
> nifi.repository.encryption.key.id=key-1
> nifi.repository.encryption.key.provider=KEYSTORE
> nifi.repository.encryption.key.provider.keystore.location=conf/repository.p12
> nifi.repository.encryption.key.provider.keystore.password=
> {quote}
> h3. Processor group
> GenerateFlowFile processor generating 1MB random files every second to a
> PutFile processor. Have also tested with InvokeHTTP.
> h3. Other comments
> With repository encryption enabled, I am able to download files via the
> provenance UI (suggesting that encryption/decryption works). The processor
> group also performs all other actions as expected.
> Not having the ability to replay provenance records is a blocker for our
> deployment, which requires data to be encrypted at rest and in transit.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-10236) ByteArrayContentRepository in core framework
[ https://issues.apache.org/jira/browse/NIFI-10236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17567381#comment-17567381 ] Mark Payne commented on NIFI-10236: --- Yeah I think it makes more sense to move the ByteArrayContentRepository from the nifi-stateless-engine to the nifi-framework-components module. We don't want to have two implementations to maintain. The repository can definitely be updated to take an optional max size. With respect to the StandardContentClaim and caching: I don't think it makes sense to introduce a new CachableContentClaim interface at this time. The setLength() is needed by StandrdContentClaimWriteCache. However, the claim that it is working with is always one that is returned by StandardContentClaimWriteCache.getContentClaim(). So it would make more sense to leave the Content Repository returning whatever type of Content Claim it wants. And then update the StandardContentClaimWriteCache's getContentClaim() method to just wrap the ContentClaim in its own internal implementation that allows the Length to be set. > ByteArrayContentRepository in core framework > > > Key: NIFI-10236 > URL: https://issues.apache.org/jira/browse/NIFI-10236 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework >Affects Versions: 1.17.0 >Reporter: Matthieu RÉ >Assignee: Matthieu RÉ >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > As discussed in NIFI-8760 and by mail, the use case of using an in-memory > based implementation of the ContentRepository is not completed after 1.17.0 > and the removal of the old VolatileContentRepository (NIFI-9823) when the > ExecuteStateless is not a solution. > As proposed by mail, the ByteArrayContentRepository could be duplicated from > the nifi-stateless framework to the core one, but maybe with more > functionalities, for instance the follow-up of the space taken in memory, and > the limitation of its total usage. > Don't hesitate to comment, add or discuss requirements or the implementation > I will propose. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] greyp9 commented on pull request #6131: NIFI-9822 - ConsumeKafkaRecord allows writing out Kafka record key
greyp9 commented on PR #6131: URL: https://github.com/apache/nifi/pull/6131#issuecomment-1185836381 > Thanks for pushing this @greyp9 ! I think this gets us really far down field. I left some comments inline, mostly around terminology / naming conventions, and documentation. I also think the tests that were added are really helpful, but I feel like some of the code is super repetitive and some of the sets of assertions can be easily refactored into a method so that the code is a little cleaner & less verbose - unless I missed some discrepancy between very similar chunks of code that are actually different. Just pushed another commit; it should address the issues you noted. There were some efficiencies to be gained in the unit tests, but the variables needed to verify behavior made the suggested refactor difficult. Hope the updates reach the needed bar. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-9809) Disable Kudu Tests on arm64 platforms
[
https://issues.apache.org/jira/browse/NIFI-9809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kevin Doran updated NIFI-9809:
--
Status: Patch Available (was: Open)
> Disable Kudu Tests on arm64 platforms
> -
>
> Key: NIFI-9809
> URL: https://issues.apache.org/jira/browse/NIFI-9809
> Project: Apache NiFi
> Issue Type: Sub-task
>Reporter: Kevin Doran
>Assignee: Kevin Doran
>Priority: Minor
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The kudu-binary artifact is not available for aarch64 (arm64) platforms:
> {noformat}
> Could not resolve dependencies for project
> org.apache.nifi:nifi-kudu-controller-service:jar:1.17.0-SNAPSHOT:
> Could not transfer artifact
> org.apache.kudu:kudu-binary:jar:osx-aarch_64:1.16.0
> {noformat}
> AFAICT, we only use this dependency in test code. so excluding those tests
> from compiling / running when on those platforms should fix the kudu bundle
> build when running on that chip architecture.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi] kevdoran opened a new pull request, #6214: NIFI-9809 Disable Kudu tests on arm64
kevdoran opened a new pull request, #6214: URL: https://github.com/apache/nifi/pull/6214 This PR disables kudu tests on arm64 platforms, as they require the `kudu-binary` artifact that is not available on arm64 platforms (at least not yet, the Kudu community, from conversations I've had, is investigating it). With this PR, I'm happy to share that all build issues on arm64 are resolved, at least for M1 Macs where I've been testing. There are still some runtime issues to address before we can claim full arm64 compatibility, but by disabling tests, this can be built on an M1 Mac. E.g.: ``` mvn -T2C clean install -DskipTests ``` # Summary [NIFI-9809](https://issues.apache.org/jira/browse/NIFI-9809) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 8 - [ ] JDK 11 - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-10241) Add comments tooltip for Controller Services
Nissim Shiman created NIFI-10241: Summary: Add comments tooltip for Controller Services Key: NIFI-10241 URL: https://issues.apache.org/jira/browse/NIFI-10241 Project: Apache NiFi Issue Type: Improvement Reporter: Nissim Shiman Currently, both Processors and Processor Groups have a tooltip enabled icon that alerts nifi graph users that a comment exists for the given Processor/Processor Group. These are seen at their respective bottom right corners Add similar functionality for Controller Services as well. This should be next to the Usage (and Alert) icons that are visible when lists of available controller services are displayed (for instance Global Menu -> Controller Settings -> Reporting Task Controller Services ) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (NIFI-10241) Add comments tooltip for Controller Services
[ https://issues.apache.org/jira/browse/NIFI-10241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nissim Shiman reassigned NIFI-10241: Assignee: Nissim Shiman > Add comments tooltip for Controller Services > > > Key: NIFI-10241 > URL: https://issues.apache.org/jira/browse/NIFI-10241 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Nissim Shiman >Assignee: Nissim Shiman >Priority: Major > > Currently, both Processors and Processor Groups have a tooltip enabled icon > that alerts nifi graph users that a comment exists for the given > Processor/Processor Group. > These are seen at their respective bottom right corners > Add similar functionality for Controller Services as well. > This should be next to the Usage (and Alert) icons that are visible when > lists of available controller services are displayed (for instance Global > Menu -> Controller Settings -> Reporting Task Controller Services ) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-10239) Fix Curator dependencies in Hive3 module
[
https://issues.apache.org/jira/browse/NIFI-10239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann updated NIFI-10239:
Fix Version/s: 1.17.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Fix Curator dependencies in Hive3 module
>
>
> Key: NIFI-10239
> URL: https://issues.apache.org/jira/browse/NIFI-10239
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Peter Turcsanyi
>Assignee: Peter Turcsanyi
>Priority: Major
> Fix For: 1.17.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Error:
> {code:java}
> 2022-07-14 23:15:15,741 WARN [Timer-Driven Process Thread-6]
> o.a.n.controller.tasks.ConnectableTask Processing halted: uncaught exception
> in Component [SelectHive3QL[id=e22cea71-f161-381e-c2c8-f241c6f363f7]]
> java.lang.NoSuchMethodError:
> org.apache.curator.shaded.com.google.common.util.concurrent.MoreExecutors.sameThreadExecutor()Lorg/apache/curator/shaded/com/google/common/util/concurrent/ListeningExecutorService;
> at
> org.apache.curator.framework.listen.ListenerContainer.addListener(ListenerContainer.java:41)
> at
> org.apache.curator.framework.imps.CuratorFrameworkImpl.start(CuratorFrameworkImpl.java:257)
> at
> org.apache.hive.jdbc.ZooKeeperHiveClientHelper.getZkClient(ZooKeeperHiveClientHelper.java:92)
> at
> org.apache.hive.jdbc.ZooKeeperHiveClientHelper.configureConnParams(ZooKeeperHiveClientHelper.java:141)
> at
> org.apache.hive.jdbc.Utils.configureConnParamsFromZooKeeper(Utils.java:511)
> at org.apache.hive.jdbc.Utils.parseURL(Utils.java:334)
> at org.apache.hive.jdbc.HiveConnection.(HiveConnection.java:168)
> at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:107)
> {code}
> RCA:
> NIFI-10145 upgraded to Hadoop 3.3.3.
> In Hive3 module, it brings curator-client:4.2.0 in but hive-common:3.1.3
> still depends on curator-framework 2.12.0 which would need a compatible
> version of curator-client.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-10239) Fix Curator dependencies in Hive3 module
[
https://issues.apache.org/jira/browse/NIFI-10239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17567350#comment-17567350
]
ASF subversion and git services commented on NIFI-10239:
Commit 14a31c81dd17f6d5fd3df38317673850a7e86b26 in nifi's branch
refs/heads/main from Peter Turcsanyi
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=14a31c81dd ]
NIFI-10239 Upgraded Curator from 4.2.0 to 5.3.0 for Hive3
- Corrected NoSuchMethodError for ZooKeeper discovery mode with Hive3 JDBC and
mismatch between Hadoop 3.3.3 and Curator 4.2.0
This closes #6210
Signed-off-by: David Handermann
> Fix Curator dependencies in Hive3 module
>
>
> Key: NIFI-10239
> URL: https://issues.apache.org/jira/browse/NIFI-10239
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Peter Turcsanyi
>Assignee: Peter Turcsanyi
>Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Error:
> {code:java}
> 2022-07-14 23:15:15,741 WARN [Timer-Driven Process Thread-6]
> o.a.n.controller.tasks.ConnectableTask Processing halted: uncaught exception
> in Component [SelectHive3QL[id=e22cea71-f161-381e-c2c8-f241c6f363f7]]
> java.lang.NoSuchMethodError:
> org.apache.curator.shaded.com.google.common.util.concurrent.MoreExecutors.sameThreadExecutor()Lorg/apache/curator/shaded/com/google/common/util/concurrent/ListeningExecutorService;
> at
> org.apache.curator.framework.listen.ListenerContainer.addListener(ListenerContainer.java:41)
> at
> org.apache.curator.framework.imps.CuratorFrameworkImpl.start(CuratorFrameworkImpl.java:257)
> at
> org.apache.hive.jdbc.ZooKeeperHiveClientHelper.getZkClient(ZooKeeperHiveClientHelper.java:92)
> at
> org.apache.hive.jdbc.ZooKeeperHiveClientHelper.configureConnParams(ZooKeeperHiveClientHelper.java:141)
> at
> org.apache.hive.jdbc.Utils.configureConnParamsFromZooKeeper(Utils.java:511)
> at org.apache.hive.jdbc.Utils.parseURL(Utils.java:334)
> at org.apache.hive.jdbc.HiveConnection.(HiveConnection.java:168)
> at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:107)
> {code}
> RCA:
> NIFI-10145 upgraded to Hadoop 3.3.3.
> In Hive3 module, it brings curator-client:4.2.0 in but hive-common:3.1.3
> still depends on curator-framework 2.12.0 which would need a compatible
> version of curator-client.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi] exceptionfactory closed pull request #6210: NIFI-10239: Fixed Curator dependencies in Hive3 module
exceptionfactory closed pull request #6210: NIFI-10239: Fixed Curator dependencies in Hive3 module URL: https://github.com/apache/nifi/pull/6210 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] NissimShiman commented on pull request #6151: NIFI-10154 ReplaceText AdminYielding on long line
NissimShiman commented on PR #6151: URL: https://github.com/apache/nifi/pull/6151#issuecomment-1185732426 @markobean Thank you very much for taking time tp look at this and for your very thought out and preceptive observations. I plan to take care of the first one. The second one is also valid but it looks like it fixing it will end up exposing other issues that may start to push this into a larger scope. Specifically, making this change will cause https://github.com/apache/nifi/blob/rel/nifi-1.16.3/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestReplaceText.java#L785 to fail There seems to be historical support for ignoring buffer sizes when regex of .* is used. See https://github.com/apache/nifi/blob/rel/nifi-1.16.3/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ReplaceText.java#L276 which seems to allude to this as well. In reality, the maximum buffer size is still looked at just like any other (i.e. and a minimum of 8KB is always assumed here as well), so the underlying code is currently not supporting this functionality, so your wondering about the 8KB default buffer size is valid, but perhaps resolving that should be done in another ticket (to handle the cascading need for some reimplementation of support for the .* regex) The third observation is also valid and I plan to update the javadocs to capture your point, although I think we should keep the IOException as it can be thrown from https://github.com/apache/nifi/blob/rel/nifi-1.16.3/nifi-commons/nifi-utils/src/main/java/org/apache/nifi/stream/io/util/AbstractTextDemarcator.java#L129 Thank you again @markobean ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-10240) UI: Processor is invalid because of a property not shown in UI
Andrew M. Lim created NIFI-10240: Summary: UI: Processor is invalid because of a property not shown in UI Key: NIFI-10240 URL: https://issues.apache.org/jira/browse/NIFI-10240 Project: Apache NiFi Issue Type: Bug Components: Core UI Reporter: Andrew M. Lim Attachments: TCP.png, UDP.png Can illustrate this issue with the ListenSyslog processor. Configure a ListenSyslog with the protocol TCP, enter a Port and set a controller service for the SSL Context Service property. This configuration is valid: !TCP.png! Now change the protocol to UDP. This configuration is considered invalid because of the SSL Context Service that is not needed or shown by the UI: !UDP.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] exceptionfactory commented on pull request #6190: NIFI-10210 Processor supporting Apache iotdb database
exceptionfactory commented on PR #6190: URL: https://github.com/apache/nifi/pull/6190#issuecomment-1185723848 > Thanks for pointing out the problem. Next, I will change the query interface mode of JDBC to the session mode of iotdb, and I will remove the JDBC insertion mode. I hope to discuss the options and general design on the associate JIRA issue would be a great next step Thanks for the response @CloudWise-Lukemiao, that sounds like a good next step. Including unit tests for a new processor is also important. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (NIFI-9758) PutEmail processor dying when SMTP server does not respond
[ https://issues.apache.org/jira/browse/NIFI-9758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann resolved NIFI-9758. Fix Version/s: 1.17.0 Resolution: Fixed > PutEmail processor dying when SMTP server does not respond > -- > > Key: NIFI-9758 > URL: https://issues.apache.org/jira/browse/NIFI-9758 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.14.0 > Environment: Windows Server 2016 > JDK 11 >Reporter: Javier Fonseca >Assignee: Emilio Setiadarma >Priority: Major > Labels: easyfix, features > Fix For: 1.17.0 > > Time Spent: 2.5h > Remaining Estimate: 0h > > *Bug* > PutEmail processor dyes after _"the socked connection is established, if the > server response is lost for some reason (or if the server doesn't respond at > all) the processor thread gets stuck forever._ > _Apparently this is kind of by design, since the default value of the > mail.smtp.timeout property is infinite, as it can be see > [here|https://javaee.github.io/javamail/docs/api/com/sun/mail/smtp/package-summary.html]._ > _I had a look at the PutEmail processor source code and unfortunately I don't > think there is a way currently to set this in the processor configuration."_ > Processor must be terminated/restart in these cases. > > *Potential Solution* > Add a configurable property TimeOut to set the __ value of the > _mail.smtp.timeout_ property, allowing the processor to fail after certain > amount of time. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-9758) PutEmail processor dying when SMTP server does not respond
[ https://issues.apache.org/jira/browse/NIFI-9758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17567325#comment-17567325 ] ASF subversion and git services commented on NIFI-9758: --- Commit cb0753b03cc7fe55b86cc795d9b7cd328c783fc0 in nifi's branch refs/heads/main from Emilio Setiadarma [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=cb0753b03c ] NIFI-9758 Added Dynamic Properties to PutEmail This closes #6204 Signed-off-by: David Handermann > PutEmail processor dying when SMTP server does not respond > -- > > Key: NIFI-9758 > URL: https://issues.apache.org/jira/browse/NIFI-9758 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.14.0 > Environment: Windows Server 2016 > JDK 11 >Reporter: Javier Fonseca >Assignee: Emilio Setiadarma >Priority: Major > Labels: easyfix, features > Time Spent: 2.5h > Remaining Estimate: 0h > > *Bug* > PutEmail processor dyes after _"the socked connection is established, if the > server response is lost for some reason (or if the server doesn't respond at > all) the processor thread gets stuck forever._ > _Apparently this is kind of by design, since the default value of the > mail.smtp.timeout property is infinite, as it can be see > [here|https://javaee.github.io/javamail/docs/api/com/sun/mail/smtp/package-summary.html]._ > _I had a look at the PutEmail processor source code and unfortunately I don't > think there is a way currently to set this in the processor configuration."_ > Processor must be terminated/restart in these cases. > > *Potential Solution* > Add a configurable property TimeOut to set the __ value of the > _mail.smtp.timeout_ property, allowing the processor to fail after certain > amount of time. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] exceptionfactory closed pull request #6204: NIFI-9758 implemented additional Dynamic Properties for JavaMail session
exceptionfactory closed pull request #6204: NIFI-9758 implemented additional Dynamic Properties for JavaMail session URL: https://github.com/apache/nifi/pull/6204 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Closed] (NIFI-10033) ScriptedLookupService will not stay running
[ https://issues.apache.org/jira/browse/NIFI-10033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Pascoe closed NIFI-10033. -- > ScriptedLookupService will not stay running > --- > > Key: NIFI-10033 > URL: https://issues.apache.org/jira/browse/NIFI-10033 > Project: Apache NiFi > Issue Type: Bug > Environment: RHEL 7.9 >Reporter: Paul Pascoe >Priority: Minor > > Any time NiFi is closed, and sometimes randomly, I have to disable and enable > the ScriptedLookupService controller or it will throw an error "Failed to > Lookup" the field it is suppsoed to find. It is using the CSVReader and > CSVRecordWriter, and I have had the same issue on NiFi 1.11.4, 1.13.2, and > 1.15.1. The ScriptedLookupService controller is runing a GROOVY script. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] exceptionfactory commented on a diff in pull request #6192: NIFI-10212 added ListSmb processor and SmbConnectionPoolService
exceptionfactory commented on code in PR #6192:
URL: https://github.com/apache/nifi/pull/6192#discussion_r922298770
##
nifi-nar-bundles/nifi-smb-bundle/nifi-smb-processors/src/main/java/org/apache/nifi/processors/smb/NiFiSmbClient.java:
##
@@ -0,0 +1,156 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.smb;
+
+import static java.util.Arrays.asList;
+import static java.util.stream.StreamSupport.stream;
+
+import com.hierynomus.msdtyp.AccessMask;
+import com.hierynomus.msfscc.FileAttributes;
+import com.hierynomus.msfscc.fileinformation.FileIdBothDirectoryInformation;
+import com.hierynomus.mssmb2.SMB2CreateDisposition;
+import com.hierynomus.mssmb2.SMB2CreateOptions;
+import com.hierynomus.mssmb2.SMB2ShareAccess;
+import com.hierynomus.mssmb2.SMBApiException;
+import com.hierynomus.smbj.session.Session;
+import com.hierynomus.smbj.share.Directory;
+import com.hierynomus.smbj.share.DiskShare;
+import com.hierynomus.smbj.share.File;
+import java.io.IOException;
+import java.io.OutputStream;
+import java.io.UncheckedIOException;
+import java.util.EnumSet;
+import java.util.List;
+import java.util.stream.Stream;
+
+public class NiFiSmbClient {
+
+private static final List SPECIAL_DIRECTORIES = asList(".", "..");
Review Comment:
Thanks for the reply, that's a good point. Leaving the variable name as
`SPECIAL_DIRECTORIES` sounds good.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on pull request #6192: NIFI-10212 added ListSmb processor and SmbConnectionPoolService
exceptionfactory commented on PR #6192: URL: https://github.com/apache/nifi/pull/6192#issuecomment-1185675042 > @kulikg - Did you have the intention of refactoring the existing GetSmb and PutSmb processors to utilise the client? Or do you think this is a body of work for later? As @kulikg mentioned, updating the existing processors to leverage the Controller Service could be something to consider in a separate issue after this one is completed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on a diff in pull request #6192: NIFI-10212 added ListSmb processor and SmbConnectionPoolService
exceptionfactory commented on code in PR #6192:
URL: https://github.com/apache/nifi/pull/6192#discussion_r922296919
##
nifi-nar-bundles/nifi-smb-bundle/nifi-smb-connection-pool/src/main/java/org/apache/nifi/services/smb/SmbjSessionProviderService.java:
##
@@ -0,0 +1,175 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.services.smb;
+
+import static java.util.Arrays.asList;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static
org.apache.nifi.expression.ExpressionLanguageScope.FLOWFILE_ATTRIBUTES;
+import static
org.apache.nifi.processor.util.StandardValidators.INTEGER_VALIDATOR;
+import static
org.apache.nifi.processor.util.StandardValidators.NON_BLANK_VALIDATOR;
+import static
org.apache.nifi.processor.util.StandardValidators.NON_EMPTY_VALIDATOR;
+import static org.apache.nifi.processor.util.StandardValidators.PORT_VALIDATOR;
+
+import com.hierynomus.smbj.SMBClient;
+import com.hierynomus.smbj.SmbConfig;
+import com.hierynomus.smbj.auth.AuthenticationContext;
+import com.hierynomus.smbj.session.Session;
+import com.hierynomus.smbj.transport.tcp.async.AsyncDirectTcpTransportFactory;
+import java.io.IOException;
+import java.io.UncheckedIOException;
+import java.net.URI;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.List;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnDisabled;
+import org.apache.nifi.annotation.lifecycle.OnEnabled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.controller.AbstractControllerService;
+import org.apache.nifi.controller.ConfigurationContext;
+
+@Tags({"microsoft", "samba"})
+@CapabilityDescription("Provides connection pool for ListSmb processor. ")
+public class SmbjSessionProviderService extends AbstractControllerService
implements SmbSessionProviderService {
+
+public static final PropertyDescriptor HOSTNAME = new
PropertyDescriptor.Builder()
+.displayName("Hostname")
+.name("hostname")
+.description("The network host of the SMB file server.")
+.required(false)
+.expressionLanguageSupported(FLOWFILE_ATTRIBUTES)
+.addValidator(NON_BLANK_VALIDATOR)
+.build();
+public static final PropertyDescriptor DOMAIN = new
PropertyDescriptor.Builder()
+.displayName("Domain")
+.name("domain")
+.description(
+"The domain used for authentication. Optional, in most
cases username and password is sufficient.")
+.required(false)
+.addValidator(NON_EMPTY_VALIDATOR)
+.build();
+public static final PropertyDescriptor USERNAME = new
PropertyDescriptor.Builder()
+.displayName("Username")
+.name("username")
+.description(
+"The username used for authentication.")
+.required(false)
+.defaultValue("Guest")
+.addValidator(NON_EMPTY_VALIDATOR)
+.build();
+public static final PropertyDescriptor PASSWORD = new
PropertyDescriptor.Builder()
+.displayName("Password")
+.name("password")
+.description("The password used for authentication.")
+.required(false)
+.addValidator(NON_EMPTY_VALIDATOR)
+.sensitive(true)
+.build();
+public static final PropertyDescriptor PORT = new
PropertyDescriptor.Builder()
+.displayName("Port")
+.name("port")
+.description("Port to use for connection.")
+.required(true)
+.addValidator(PORT_VALIDATOR)
+.defaultValue("445")
+.build();
+public static final PropertyDescriptor TIMEOUT = new
PropertyDescriptor.Builder()
+.displayName("Timeout")
+.name("timeout")
+.description("Timeout in seconds for read and write operations.")
+.required(true)
+
[GitHub] [nifi] exceptionfactory commented on a diff in pull request #6192: NIFI-10212 added ListSmb processor and SmbConnectionPoolService
exceptionfactory commented on code in PR #6192:
URL: https://github.com/apache/nifi/pull/6192#discussion_r922291708
##
nifi-nar-bundles/nifi-smb-bundle/nifi-smb-connection-pool/src/main/java/org/apache/nifi/services/smb/SmbjSessionProviderService.java:
##
@@ -0,0 +1,175 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.services.smb;
+
+import static java.util.Arrays.asList;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static
org.apache.nifi.expression.ExpressionLanguageScope.FLOWFILE_ATTRIBUTES;
+import static
org.apache.nifi.processor.util.StandardValidators.INTEGER_VALIDATOR;
+import static
org.apache.nifi.processor.util.StandardValidators.NON_BLANK_VALIDATOR;
+import static
org.apache.nifi.processor.util.StandardValidators.NON_EMPTY_VALIDATOR;
+import static org.apache.nifi.processor.util.StandardValidators.PORT_VALIDATOR;
+
+import com.hierynomus.smbj.SMBClient;
+import com.hierynomus.smbj.SmbConfig;
+import com.hierynomus.smbj.auth.AuthenticationContext;
+import com.hierynomus.smbj.session.Session;
+import com.hierynomus.smbj.transport.tcp.async.AsyncDirectTcpTransportFactory;
+import java.io.IOException;
+import java.io.UncheckedIOException;
+import java.net.URI;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.List;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnDisabled;
+import org.apache.nifi.annotation.lifecycle.OnEnabled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.controller.AbstractControllerService;
+import org.apache.nifi.controller.ConfigurationContext;
+
+@Tags({"microsoft", "samba"})
+@CapabilityDescription("Provides connection pool for ListSmb processor. ")
+public class SmbjSessionProviderService extends AbstractControllerService
implements SmbSessionProviderService {
Review Comment:
It would be great if we could isolate the SMBJ specifics to the Controller
Service implementation, and avoid having interface-level dependency on SMBJ.
Thanks for working these adjustments!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (NIFI-10213) Shopify processor
[ https://issues.apache.org/jira/browse/NIFI-10213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lehel Boér updated NIFI-10213: -- Status: Patch Available (was: In Progress) > Shopify processor > - > > Key: NIFI-10213 > URL: https://issues.apache.org/jira/browse/NIFI-10213 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Lehel Boér >Assignee: Lehel Boér >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] Lehel44 opened a new pull request, #6213: NIFI-10213: Created GetShopify processor
Lehel44 opened a new pull request, #6213: URL: https://github.com/apache/nifi/pull/6213 # Summary [NIFI-10213](https://issues.apache.org/jira/browse/NIFI-10213) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI-10213) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 8 - [ ] JDK 11 - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1369: MINIFICPP-1874 docker-verify should only run tests that are enabled
lordgamez commented on code in PR #1369:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1369#discussion_r922262730
##
extensions/sftp/tests/PutSFTPTests.cpp:
##
@@ -506,7 +507,7 @@ TEST_CASE_METHOD(PutSFTPTestsFixture, "PutSFTP set mtime",
"[PutSFTP]") {
testFile("nifi_test/tstFile1.ext", "content 1");
using namespace std::chrono; // NOLINT(build/namespaces)
- time_point modification_time = sys_days(January / 24 / 2065) +
5h + 20min;
+ system_clock::time_point modification_time = date::sys_days(date::January /
24 / 2065) + 5h + 20min;
Review Comment:
Thanks for the info!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] martinzink commented on a diff in pull request #1369: MINIFICPP-1874 docker-verify should only run tests that are enabled
martinzink commented on code in PR #1369: URL: https://github.com/apache/nifi-minifi-cpp/pull/1369#discussion_r922256892 ## docker/Dockerfile: ## @@ -28,47 +28,7 @@ ARG GID=1000 # PDH and WEL extensions and not listed as they are Windows specific # SYSTEMD extension is turned OFF explicitly as it has no use in an alpine container Review Comment: nice catch, removed this in https://github.com/apache/nifi-minifi-cpp/pull/1369/commits/69bbe6d590f5608d809a28fff7aa062bfd80800b -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] martinzink commented on a diff in pull request #1369: MINIFICPP-1874 docker-verify should only run tests that are enabled
martinzink commented on code in PR #1369: URL: https://github.com/apache/nifi-minifi-cpp/pull/1369#discussion_r922256688 ## docker/Dockerfile: ## @@ -28,47 +28,7 @@ ARG GID=1000 # PDH and WEL extensions and not listed as they are Windows specific # SYSTEMD extension is turned OFF explicitly as it has no use in an alpine container -ARG ENABLE_ALL=OFF -ARG ENABLE_PYTHON=OFF -ARG ENABLE_OPS=ON -ARG ENABLE_JNI=OFF -ARG ENABLE_OPENCV=OFF -ARG ENABLE_OPC=OFF -ARG ENABLE_GPS=OFF -ARG ENABLE_COAP=OFF -ARG ENABLE_SQL=OFF -ARG ENABLE_MQTT=OFF -ARG ENABLE_PCAP=OFF -ARG ENABLE_LIBRDKAFKA=OFF -ARG ENABLE_SENSORS=OFF -ARG ENABLE_USB_CAMERA=OFF -ARG ENABLE_TENSORFLOW=OFF -ARG ENABLE_AWS=OFF -ARG ENABLE_BUSTACHE=OFF -ARG ENABLE_SFTP=OFF -ARG ENABLE_OPENWSMAN=OFF -ARG ENABLE_AZURE=OFF -ARG ENABLE_ENCRYPT_CONFIG=ON -ARG ENABLE_NANOFI=OFF -ARG ENABLE_SPLUNK=OFF -ARG ENABLE_GCP=OFF -ARG ENABLE_ELASTICSEARCH=OFF -ARG ENABLE_TEST_PROCESSORS=OFF -ARG DISABLE_CURL=OFF -ARG DISABLE_JEMALLOC=ON -ARG DISABLE_CIVET=OFF -ARG DISABLE_EXPRESSION_LANGUAGE=OFF -ARG DISABLE_ROCKSDB=OFF -ARG DISABLE_LIBARCHIVE=OFF -ARG DISABLE_LZMA=OFF -ARG DISABLE_BZIP2=OFF -ARG ENABLE_SCRIPTING=OFF -ARG DISABLE_PYTHON_SCRIPTING= -ARG ENABLE_LUA_SCRIPTING= -ARG ENABLE_KUBERNETES=OFF -ARG ENABLE_PROCFS=OFF -ARG ENABLE_PROMETHEUS=OFF -ARG DISABLE_CONTROLLER=OFF +ARG MINIFI_OPTIONS="" Review Comment: good idea, I've added comments in https://github.com/apache/nifi-minifi-cpp/pull/1369/commits/69bbe6d590f5608d809a28fff7aa062bfd80800b -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] kulikg commented on a diff in pull request #6192: NIFI-10212 added ListSmb processor and SmbConnectionPoolService
kulikg commented on code in PR #6192:
URL: https://github.com/apache/nifi/pull/6192#discussion_r922251851
##
nifi-nar-bundles/nifi-smb-bundle/nifi-smb-connection-pool/src/main/java/org/apache/nifi/services/smb/SmbjSessionProviderService.java:
##
@@ -0,0 +1,175 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.services.smb;
+
+import static java.util.Arrays.asList;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static
org.apache.nifi.expression.ExpressionLanguageScope.FLOWFILE_ATTRIBUTES;
+import static
org.apache.nifi.processor.util.StandardValidators.INTEGER_VALIDATOR;
+import static
org.apache.nifi.processor.util.StandardValidators.NON_BLANK_VALIDATOR;
+import static
org.apache.nifi.processor.util.StandardValidators.NON_EMPTY_VALIDATOR;
+import static org.apache.nifi.processor.util.StandardValidators.PORT_VALIDATOR;
+
+import com.hierynomus.smbj.SMBClient;
+import com.hierynomus.smbj.SmbConfig;
+import com.hierynomus.smbj.auth.AuthenticationContext;
+import com.hierynomus.smbj.session.Session;
+import com.hierynomus.smbj.transport.tcp.async.AsyncDirectTcpTransportFactory;
+import java.io.IOException;
+import java.io.UncheckedIOException;
+import java.net.URI;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.List;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnDisabled;
+import org.apache.nifi.annotation.lifecycle.OnEnabled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.controller.AbstractControllerService;
+import org.apache.nifi.controller.ConfigurationContext;
+
+@Tags({"microsoft", "samba"})
+@CapabilityDescription("Provides connection pool for ListSmb processor. ")
+public class SmbjSessionProviderService extends AbstractControllerService
implements SmbSessionProviderService {
Review Comment:
Good point. I think the api shouldn't depend on smbj and this should be
fixed. As you suggested to extract the nifi smb client interface I think the
best would be to lift that interface right into the api and move the
implementation into the controller service module.
This way both the api and the processor would be independent from smbj so
that other java driver could be easily added, also I could get rid of the
NiFiSmbClientFactory and move that functionality into the controller service
and mock it nicely using mock framework in the tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] (NIFI-10236) ByteArrayContentRepository in core framework
[ https://issues.apache.org/jira/browse/NIFI-10236 ] Matthieu RÉ deleted comment on NIFI-10236: was (Author: mathemare): Oh, it was not your point, I'm sorry! About the possibility to use the stateless engine, our main blocking point among others was that we need in our current use-case to operate our Flows directly through NiFi UI (investigating transformation, potential bugs in production without having the need to commit and apply through NiFi Registry), from my understanding it is not the spirit of the Stateless implementation. > ByteArrayContentRepository in core framework > > > Key: NIFI-10236 > URL: https://issues.apache.org/jira/browse/NIFI-10236 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework >Affects Versions: 1.17.0 >Reporter: Matthieu RÉ >Assignee: Matthieu RÉ >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > As discussed in NIFI-8760 and by mail, the use case of using an in-memory > based implementation of the ContentRepository is not completed after 1.17.0 > and the removal of the old VolatileContentRepository (NIFI-9823) when the > ExecuteStateless is not a solution. > As proposed by mail, the ByteArrayContentRepository could be duplicated from > the nifi-stateless framework to the core one, but maybe with more > functionalities, for instance the follow-up of the space taken in memory, and > the limitation of its total usage. > Don't hesitate to comment, add or discuss requirements or the implementation > I will propose. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-10236) ByteArrayContentRepository in core framework
[ https://issues.apache.org/jira/browse/NIFI-10236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17567272#comment-17567272 ] Matthieu RÉ commented on NIFI-10236: Oh, it was not your point, I'm sorry! About the possibility to use the stateless engine, our main blocking point among others was that we need in our current use-case to operate our Flows directly through NiFi UI (investigating transformation, potential bugs in production without having the need to commit and apply through NiFi Registry), from my understanding it is not the spirit of the Stateless implementation. > ByteArrayContentRepository in core framework > > > Key: NIFI-10236 > URL: https://issues.apache.org/jira/browse/NIFI-10236 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework >Affects Versions: 1.17.0 >Reporter: Matthieu RÉ >Assignee: Matthieu RÉ >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > As discussed in NIFI-8760 and by mail, the use case of using an in-memory > based implementation of the ContentRepository is not completed after 1.17.0 > and the removal of the old VolatileContentRepository (NIFI-9823) when the > ExecuteStateless is not a solution. > As proposed by mail, the ByteArrayContentRepository could be duplicated from > the nifi-stateless framework to the core one, but maybe with more > functionalities, for instance the follow-up of the space taken in memory, and > the limitation of its total usage. > Don't hesitate to comment, add or discuss requirements or the implementation > I will propose. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi-minifi-cpp] martinzink commented on a diff in pull request #1369: MINIFICPP-1874 docker-verify should only run tests that are enabled
martinzink commented on code in PR #1369:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1369#discussion_r922243196
##
docker/DockerVerify.sh:
##
@@ -67,11 +67,11 @@ TEST_DIRECTORY="${docker_dir}/test/integration"
export TEST_DIRECTORY
# Add --no-logcapture to see logs interleaved with the test output
-BEHAVE_OPTS=(-f pretty --logging-level INFO --logging-clear-handlers --tags
~@no-ci)
Review Comment:
The problem with the tag was that it influenced local runs as well.
https://github.com/apache/nifi-minifi-cpp/pull/1349#discussion_r907500456
Due to this reason Id rather readd it later if we need it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (NIFI-10236) ByteArrayContentRepository in core framework
[ https://issues.apache.org/jira/browse/NIFI-10236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17567269#comment-17567269 ] David Handermann commented on NIFI-10236: - Thanks for the background [~mathemare], that is helpful. [~markap14] may have some additional input on the general approach, and potential for abstracting common features into a shared class. Thanks for submitting the pull request for review. > ByteArrayContentRepository in core framework > > > Key: NIFI-10236 > URL: https://issues.apache.org/jira/browse/NIFI-10236 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework >Affects Versions: 1.17.0 >Reporter: Matthieu RÉ >Assignee: Matthieu RÉ >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > As discussed in NIFI-8760 and by mail, the use case of using an in-memory > based implementation of the ContentRepository is not completed after 1.17.0 > and the removal of the old VolatileContentRepository (NIFI-9823) when the > ExecuteStateless is not a solution. > As proposed by mail, the ByteArrayContentRepository could be duplicated from > the nifi-stateless framework to the core one, but maybe with more > functionalities, for instance the follow-up of the space taken in memory, and > the limitation of its total usage. > Don't hesitate to comment, add or discuss requirements or the implementation > I will propose. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi-minifi-cpp] martinzink commented on a diff in pull request #1369: MINIFICPP-1874 docker-verify should only run tests that are enabled
martinzink commented on code in PR #1369:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1369#discussion_r922238458
##
extensions/sftp/tests/PutSFTPTests.cpp:
##
@@ -506,7 +507,7 @@ TEST_CASE_METHOD(PutSFTPTestsFixture, "PutSFTP set mtime",
"[PutSFTP]") {
testFile("nifi_test/tstFile1.ext", "content 1");
using namespace std::chrono; // NOLINT(build/namespaces)
- time_point modification_time = sys_days(January / 24 / 2065) +
5h + 20min;
+ system_clock::time_point modification_time = date::sys_days(date::January /
24 / 2065) + 5h + 20min;
Review Comment:
I found this during development, because it didnt compile on centos (usually
this is not a problem because -DSKIP_TESTS is hardcoded to be ON with the
centos target)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] martinzink commented on a diff in pull request #1369: MINIFICPP-1874 docker-verify should only run tests that are enabled
martinzink commented on code in PR #1369:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1369#discussion_r922238458
##
extensions/sftp/tests/PutSFTPTests.cpp:
##
@@ -506,7 +507,7 @@ TEST_CASE_METHOD(PutSFTPTestsFixture, "PutSFTP set mtime",
"[PutSFTP]") {
testFile("nifi_test/tstFile1.ext", "content 1");
using namespace std::chrono; // NOLINT(build/namespaces)
- time_point modification_time = sys_days(January / 24 / 2065) +
5h + 20min;
+ system_clock::time_point modification_time = date::sys_days(date::January /
24 / 2065) + 5h + 20min;
Review Comment:
I found this during development, because it didnt compile on centos (usually
this is not a problem because -DSKIP_TESTS is hardcoded to be ON)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (NIFI-10236) ByteArrayContentRepository in core framework
[ https://issues.apache.org/jira/browse/NIFI-10236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17567267#comment-17567267 ] Matthieu RÉ commented on NIFI-10236: Hello David, Sorry for that! Yes I first tried to use the ByteArrayContentRepository from the nifi-stateless framework, but there were two limits (in my non-expert opinion) for it to be directly set in the core framework : * the current implementation does not set limits to the max size it should take in memory, and does not provide the ability to get the size the contents occupy * the current StandardProcessSession tries in its current implementation to use a StandardContentClaim to be stored in a Cache ([StandardContentClaimWriteCache|https://github.com/apache/nifi/blob/main/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/repository/claim/StandardContentClaimWriteCache.java#L87]), and the StandardContentClaim is not implemented by the current ByteArrayContentRepository The proposed implementation tries to solve these two requirements... but if I can have misunderstood something about the two points or their need > ByteArrayContentRepository in core framework > > > Key: NIFI-10236 > URL: https://issues.apache.org/jira/browse/NIFI-10236 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework >Affects Versions: 1.17.0 >Reporter: Matthieu RÉ >Assignee: Matthieu RÉ >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > As discussed in NIFI-8760 and by mail, the use case of using an in-memory > based implementation of the ContentRepository is not completed after 1.17.0 > and the removal of the old VolatileContentRepository (NIFI-9823) when the > ExecuteStateless is not a solution. > As proposed by mail, the ByteArrayContentRepository could be duplicated from > the nifi-stateless framework to the core one, but maybe with more > functionalities, for instance the follow-up of the space taken in memory, and > the limitation of its total usage. > Don't hesitate to comment, add or discuss requirements or the implementation > I will propose. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] nandorsoma commented on a diff in pull request #6158: NIFI-10152 Storage client caching in Azure ADLS processors
nandorsoma commented on code in PR #6158:
URL: https://github.com/apache/nifi/pull/6158#discussion_r90997
##
nifi-nar-bundles/nifi-azure-bundle/nifi-azure-processors/src/main/java/org/apache/nifi/processors/azure/storage/utils/StorageClientFactory.java:
##
@@ -0,0 +1,117 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.azure.storage.utils;
+
+import com.azure.core.credential.AccessToken;
+import com.azure.core.credential.TokenCredential;
+import com.azure.core.http.HttpClient;
+import com.azure.core.http.ProxyOptions;
+import com.azure.core.http.netty.NettyAsyncHttpClientBuilder;
+import com.azure.identity.ClientSecretCredential;
+import com.azure.identity.ClientSecretCredentialBuilder;
+import com.azure.identity.ManagedIdentityCredential;
+import com.azure.identity.ManagedIdentityCredentialBuilder;
+import com.azure.storage.common.StorageSharedKeyCredential;
+import com.azure.storage.file.datalake.DataLakeServiceClient;
+import com.azure.storage.file.datalake.DataLakeServiceClientBuilder;
+import com.github.benmanes.caffeine.cache.Cache;
+import com.github.benmanes.caffeine.cache.Caffeine;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.services.azure.storage.ADLSCredentialsDetails;
+import reactor.core.publisher.Mono;
+
+public class StorageClientFactory {
+
+private static final long STORAGE_CLIENT_CACHE_SIZE = 10;
+
+private final Cache storageClientCache;
+
+public StorageClientFactory() {
+this.storageClientCache = createCache();
+}
+
+private Cache createCache() {
+return Caffeine.newBuilder()
+.maximumSize(STORAGE_CLIENT_CACHE_SIZE)
+.build();
+}
+
+public Cache getCache() {
+return storageClientCache;
+}
+
+/**
+ * Retrieves a {@link DataLakeServiceClient}
+ *
+ * @param proxyOptions not used for caching, because proxy parameters are
set in the ProxyConfiguration service
+ * @param credentialsDetails used for caching because it can contain
properties that are results of an expression
+ * @return DataLakeServiceClient
+ */
+public DataLakeServiceClient getStorageClient(ProxyOptions proxyOptions,
ADLSCredentialsDetails credentialsDetails) {
Review Comment:
I've checked the related code in `azure-storage-file-datalake-12.7.4.jar`
and I didn't find anything that would make me think that the library was thread
unsafe.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] nandorsoma commented on a diff in pull request #6158: NIFI-10152 Storage client caching in Azure ADLS processors
nandorsoma commented on code in PR #6158:
URL: https://github.com/apache/nifi/pull/6158#discussion_r90997
##
nifi-nar-bundles/nifi-azure-bundle/nifi-azure-processors/src/main/java/org/apache/nifi/processors/azure/storage/utils/StorageClientFactory.java:
##
@@ -0,0 +1,117 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.azure.storage.utils;
+
+import com.azure.core.credential.AccessToken;
+import com.azure.core.credential.TokenCredential;
+import com.azure.core.http.HttpClient;
+import com.azure.core.http.ProxyOptions;
+import com.azure.core.http.netty.NettyAsyncHttpClientBuilder;
+import com.azure.identity.ClientSecretCredential;
+import com.azure.identity.ClientSecretCredentialBuilder;
+import com.azure.identity.ManagedIdentityCredential;
+import com.azure.identity.ManagedIdentityCredentialBuilder;
+import com.azure.storage.common.StorageSharedKeyCredential;
+import com.azure.storage.file.datalake.DataLakeServiceClient;
+import com.azure.storage.file.datalake.DataLakeServiceClientBuilder;
+import com.github.benmanes.caffeine.cache.Cache;
+import com.github.benmanes.caffeine.cache.Caffeine;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.services.azure.storage.ADLSCredentialsDetails;
+import reactor.core.publisher.Mono;
+
+public class StorageClientFactory {
+
+private static final long STORAGE_CLIENT_CACHE_SIZE = 10;
+
+private final Cache storageClientCache;
+
+public StorageClientFactory() {
+this.storageClientCache = createCache();
+}
+
+private Cache createCache() {
+return Caffeine.newBuilder()
+.maximumSize(STORAGE_CLIENT_CACHE_SIZE)
+.build();
+}
+
+public Cache getCache() {
+return storageClientCache;
+}
+
+/**
+ * Retrieves a {@link DataLakeServiceClient}
+ *
+ * @param proxyOptions not used for caching, because proxy parameters are
set in the ProxyConfiguration service
+ * @param credentialsDetails used for caching because it can contain
properties that are results of an expression
+ * @return DataLakeServiceClient
+ */
+public DataLakeServiceClient getStorageClient(ProxyOptions proxyOptions,
ADLSCredentialsDetails credentialsDetails) {
Review Comment:
I've checked the related code in `azure-storage-file-datalake-12.7.4.jar`
and I didn't find anything that would made me think that the library is thread
unsafe.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (MINIFICPP-1875) HTTPClient should be reusable
[
https://issues.apache.org/jira/browse/MINIFICPP-1875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martin Zink updated MINIFICPP-1875:
---
Fix Version/s: 0.13.0
> HTTPClient should be reusable
> -
>
> Key: MINIFICPP-1875
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1875
> Project: Apache NiFi MiNiFi C++
> Issue Type: Improvement
>Reporter: Martin Zink
>Assignee: Martin Zink
>Priority: Major
> Fix For: 0.13.0
>
>
> The current form of HTTPCLient is not safe to reuse.
> (e.g. the response headers are appended to the same vector on each submit,
> the request header if not empty gets a {{"Expect:"}} appended on each submit)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (MINIFICPP-1735) Add Google Cloud Storage related processors
[ https://issues.apache.org/jira/browse/MINIFICPP-1735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martin Zink updated MINIFICPP-1735: --- Fix Version/s: 0.13.0 > Add Google Cloud Storage related processors > --- > > Key: MINIFICPP-1735 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1735 > Project: Apache NiFi MiNiFi C++ > Issue Type: Epic >Reporter: Martin Zink >Assignee: Martin Zink >Priority: Major > Fix For: 0.13.0 > > > Minifi already support AWS and Azure, but we currently lack the support for > Google Cloud Storage. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-1761) PutGCSObject, FetchGCSObject, DeleteGCSObject multithreading
[ https://issues.apache.org/jira/browse/MINIFICPP-1761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martin Zink updated MINIFICPP-1761: --- Fix Version/s: 0.13.0 > PutGCSObject, FetchGCSObject, DeleteGCSObject multithreading > > > Key: MINIFICPP-1761 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1761 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Martin Zink >Assignee: Martin Zink >Priority: Minor > Fix For: 0.13.0 > > Time Spent: 10m > Remaining Estimate: 0h > > On first glance there is no reason why the PutGCSObject processor should be > single-threaded only. > But we need some proper testing before enabling it. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-1862) Change leftover boost filesystem usages from MINIFICPP-1636
[ https://issues.apache.org/jira/browse/MINIFICPP-1862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martin Zink updated MINIFICPP-1862: --- Fix Version/s: 0.13.0 > Change leftover boost filesystem usages from MINIFICPP-1636 > --- > > Key: MINIFICPP-1862 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1862 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Martin Zink >Assignee: Martin Zink >Priority: Minor > Labels: MiNiFi-CPP-Hygiene > Fix For: 0.13.0 > > > There are a few leftover boost::filesystem usages which could be removed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-1882) centos build default CA doesnt work on ubuntu
[
https://issues.apache.org/jira/browse/MINIFICPP-1882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martin Zink updated MINIFICPP-1882:
---
Fix Version/s: 0.13.0
> centos build default CA doesnt work on ubuntu
> -
>
> Key: MINIFICPP-1882
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1882
> Project: Apache NiFi MiNiFi C++
> Issue Type: Bug
>Reporter: Martin Zink
>Priority: Major
> Fix For: 0.13.0
>
>
> the default CA path is different on centos and ubuntu, and due to this if we
> use the centos build on ubuntu any libcurl processor trying to verify ssl
> without explicit SSLContextService will fail with
> {code:java}
> curl_easy_perform() failed Problem with the SSL CA cert (path? access
> rights?) on XYZ, error code 77 {code}
> There are workarounds
> * with a symlink to the corrent path
> * explicit SSLContextService with CA set
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (MINIFICPP-1882) centos build default CA doesnt work on ubuntu
Martin Zink created MINIFICPP-1882:
--
Summary: centos build default CA doesnt work on ubuntu
Key: MINIFICPP-1882
URL: https://issues.apache.org/jira/browse/MINIFICPP-1882
Project: Apache NiFi MiNiFi C++
Issue Type: Bug
Reporter: Martin Zink
the default CA path is different on centos and ubuntu, and due to this if we
use the centos build on ubuntu any libcurl processor trying to verify ssl
without explicit SSLContextService will fail with
{code:java}
curl_easy_perform() failed Problem with the SSL CA cert (path? access rights?)
on XYZ, error code 77 {code}
There are workarounds
* with a symlink to the corrent path
* explicit SSLContextService with CA set
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-10236) ByteArrayContentRepository in core framework
[
https://issues.apache.org/jira/browse/NIFI-10236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17567259#comment-17567259
]
David Handermann commented on NIFI-10236:
-
Thanks for proposing this framework repository [~mathemare]. The Fix Version
field should remain blank until the issue is resolved.
One general note on the proposal, did you consider refactoring the existing
ByteArrayContentRepository in {{nifi-stateless-engine}} as opposed to creating
a new version?
> ByteArrayContentRepository in core framework
>
>
> Key: NIFI-10236
> URL: https://issues.apache.org/jira/browse/NIFI-10236
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Core Framework
>Affects Versions: 1.17.0
>Reporter: Matthieu RÉ
>Assignee: Matthieu RÉ
>Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> As discussed in NIFI-8760 and by mail, the use case of using an in-memory
> based implementation of the ContentRepository is not completed after 1.17.0
> and the removal of the old VolatileContentRepository (NIFI-9823) when the
> ExecuteStateless is not a solution.
> As proposed by mail, the ByteArrayContentRepository could be duplicated from
> the nifi-stateless framework to the core one, but maybe with more
> functionalities, for instance the follow-up of the space taken in memory, and
> the limitation of its total usage.
> Don't hesitate to comment, add or discuss requirements or the implementation
> I will propose.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-10236) ByteArrayContentRepository in core framework
[ https://issues.apache.org/jira/browse/NIFI-10236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-10236: Fix Version/s: (was: 1.17.0) > ByteArrayContentRepository in core framework > > > Key: NIFI-10236 > URL: https://issues.apache.org/jira/browse/NIFI-10236 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework >Affects Versions: 1.17.0 >Reporter: Matthieu RÉ >Assignee: Matthieu RÉ >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > As discussed in NIFI-8760 and by mail, the use case of using an in-memory > based implementation of the ContentRepository is not completed after 1.17.0 > and the removal of the old VolatileContentRepository (NIFI-9823) when the > ExecuteStateless is not a solution. > As proposed by mail, the ByteArrayContentRepository could be duplicated from > the nifi-stateless framework to the core one, but maybe with more > functionalities, for instance the follow-up of the space taken in memory, and > the limitation of its total usage. > Don't hesitate to comment, add or discuss requirements or the implementation > I will propose. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-10234) implement a processor of Apache IoTDB
[ https://issues.apache.org/jira/browse/NIFI-10234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17567258#comment-17567258 ] David Handermann commented on NIFI-10234: - Thanks for offering to contribute a new processor [~xuanronaldo]! Please join the [NiFi Developers mailing list|https://nifi.apache.org/mailing_lists.html] and send a request for contributor access to the NiFi project, then you will be able to assign the issue to yourself. [~luke.miao] created an initial pull request for NIFI-10210, but I noted some initial design issues. It would probably be helpful to outline a potential design in the Jira issue before submitting a new pull request. Outlining features such as processor properties and relationships would be a helpful starting point. > implement a processor of Apache IoTDB > - > > Key: NIFI-10234 > URL: https://issues.apache.org/jira/browse/NIFI-10234 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Xuan Ronaldo >Priority: Major > > Hi, folks > > I'm a contributer of Apache IoTDB. Recently, I have implemented a processor > which can write data to IoTDB. I'd like to submit it to the NiFi as a > build-in processor. Besides, there are more processors or controller services > will be developed. > > Regards, > Xuan Ronaldo -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-10236) ByteArrayContentRepository in core framework
[ https://issues.apache.org/jira/browse/NIFI-10236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthieu RÉ updated NIFI-10236: --- Fix Version/s: 1.17.0 > ByteArrayContentRepository in core framework > > > Key: NIFI-10236 > URL: https://issues.apache.org/jira/browse/NIFI-10236 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework >Affects Versions: 1.17.0 >Reporter: Matthieu RÉ >Assignee: Matthieu RÉ >Priority: Major > Fix For: 1.17.0 > > Time Spent: 20m > Remaining Estimate: 0h > > As discussed in NIFI-8760 and by mail, the use case of using an in-memory > based implementation of the ContentRepository is not completed after 1.17.0 > and the removal of the old VolatileContentRepository (NIFI-9823) when the > ExecuteStateless is not a solution. > As proposed by mail, the ByteArrayContentRepository could be duplicated from > the nifi-stateless framework to the core one, but maybe with more > functionalities, for instance the follow-up of the space taken in memory, and > the limitation of its total usage. > Don't hesitate to comment, add or discuss requirements or the implementation > I will propose. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] mathemare commented on pull request #6211: NIFI-10236: ByteArrayContentRepository in core framework
mathemare commented on PR #6211: URL: https://github.com/apache/nifi/pull/6211#issuecomment-1185570613 Hello, I found a bug when restarting NiFi, flowfile counts in queues are not cleared while flowfiles are removed from ContentRepo, I'll investigate where the removal is not done Any example of how to build unit tests around this is really welcome! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1364: MINIFICPP-1878 Enable Ninja build in Windows builds
lordgamez commented on code in PR #1364:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1364#discussion_r92289
##
cmake/CivetWeb.cmake:
##
@@ -0,0 +1,46 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+include(FetchContent)
+
+set(CIVETWEB_ENABLE_SSL_DYNAMIC_LOADING "OFF" CACHE STRING "" FORCE)
+set(CIVETWEB_BUILD_TESTING "OFF" CACHE STRING "" FORCE)
+set(CIVETWEB_ENABLE_DUKTAPE "OFF" CACHE STRING "" FORCE)
+set(CIVETWEB_ENABLE_LUA "OFF" CACHE STRING "" FORCE)
+set(CIVETWEB_ENABLE_CXX "ON" CACHE STRING "" FORCE)
+set(CIVETWEB_ALLOW_WARNINGS "ON" CACHE STRING "" FORCE)
+set(CIVETWEB_ENABLE_ASAN "OFF" CACHE STRING "" FORCE)
+
+FetchContent_Declare(civetweb
+GIT_REPOSITORY "https://github.com/civetweb/civetweb.git;
+GIT_TAG "4447b6501d5c568b4c6c0940eac801ec690b2250" # commit containing fix
for MSVC issue https://github.com/civetweb/civetweb/issues/1024
+)
+
+FetchContent_MakeAvailable(civetweb)
+
+add_dependencies(civetweb-c-library OpenSSL::Crypto OpenSSL::SSL)
+add_dependencies(civetweb-cpp OpenSSL::Crypto OpenSSL::SSL)
+
+set(CIVETWEB_C_FLAGS "${CMAKE_CFLAGS} -Wno-error" CACHE STRING "" FORCE)
+set(CIVETWEB_CXX_LAGS "${CMAKE_CXX_FLAGS} -Wno-error" CACHE STRING "" FORCE)
+string(REPLACE " " ";" REPLACED_C_FLAGS ${CIVETWEB_C_FLAGS})
+string(REPLACE " " ";" REPLACED_CXX_FLAGS ${CIVETWEB_CXX_LAGS})
+target_compile_options(civetweb-c-library INTERFACE ${REPLACED_C_FLAGS})
+target_compile_options(civetweb-cpp INTERFACE ${REPLACED_CXX_FLAGS})
Review Comment:
Good point, I think you are right, fixed in
e5b0aad87214253154820ee11f858fafa622e12d
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1364: MINIFICPP-1878 Enable Ninja build in Windows builds
lordgamez commented on code in PR #1364:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1364#discussion_r92289
##
cmake/CivetWeb.cmake:
##
@@ -0,0 +1,46 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+include(FetchContent)
+
+set(CIVETWEB_ENABLE_SSL_DYNAMIC_LOADING "OFF" CACHE STRING "" FORCE)
+set(CIVETWEB_BUILD_TESTING "OFF" CACHE STRING "" FORCE)
+set(CIVETWEB_ENABLE_DUKTAPE "OFF" CACHE STRING "" FORCE)
+set(CIVETWEB_ENABLE_LUA "OFF" CACHE STRING "" FORCE)
+set(CIVETWEB_ENABLE_CXX "ON" CACHE STRING "" FORCE)
+set(CIVETWEB_ALLOW_WARNINGS "ON" CACHE STRING "" FORCE)
+set(CIVETWEB_ENABLE_ASAN "OFF" CACHE STRING "" FORCE)
+
+FetchContent_Declare(civetweb
+GIT_REPOSITORY "https://github.com/civetweb/civetweb.git;
+GIT_TAG "4447b6501d5c568b4c6c0940eac801ec690b2250" # commit containing fix
for MSVC issue https://github.com/civetweb/civetweb/issues/1024
+)
+
+FetchContent_MakeAvailable(civetweb)
+
+add_dependencies(civetweb-c-library OpenSSL::Crypto OpenSSL::SSL)
+add_dependencies(civetweb-cpp OpenSSL::Crypto OpenSSL::SSL)
+
+set(CIVETWEB_C_FLAGS "${CMAKE_CFLAGS} -Wno-error" CACHE STRING "" FORCE)
+set(CIVETWEB_CXX_LAGS "${CMAKE_CXX_FLAGS} -Wno-error" CACHE STRING "" FORCE)
+string(REPLACE " " ";" REPLACED_C_FLAGS ${CIVETWEB_C_FLAGS})
+string(REPLACE " " ";" REPLACED_CXX_FLAGS ${CIVETWEB_CXX_LAGS})
+target_compile_options(civetweb-c-library INTERFACE ${REPLACED_C_FLAGS})
+target_compile_options(civetweb-cpp INTERFACE ${REPLACED_CXX_FLAGS})
Review Comment:
Good point, fixed in e5b0aad87214253154820ee11f858fafa622e12d
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm commented on a diff in pull request #1364: MINIFICPP-1878 Enable Ninja build in Windows builds
szaszm commented on code in PR #1364:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1364#discussion_r922107019
##
cmake/CivetWeb.cmake:
##
@@ -0,0 +1,46 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+include(FetchContent)
+
+set(CIVETWEB_ENABLE_SSL_DYNAMIC_LOADING "OFF" CACHE STRING "" FORCE)
+set(CIVETWEB_BUILD_TESTING "OFF" CACHE STRING "" FORCE)
+set(CIVETWEB_ENABLE_DUKTAPE "OFF" CACHE STRING "" FORCE)
+set(CIVETWEB_ENABLE_LUA "OFF" CACHE STRING "" FORCE)
+set(CIVETWEB_ENABLE_CXX "ON" CACHE STRING "" FORCE)
+set(CIVETWEB_ALLOW_WARNINGS "ON" CACHE STRING "" FORCE)
+set(CIVETWEB_ENABLE_ASAN "OFF" CACHE STRING "" FORCE)
+
+FetchContent_Declare(civetweb
+GIT_REPOSITORY "https://github.com/civetweb/civetweb.git;
+GIT_TAG "4447b6501d5c568b4c6c0940eac801ec690b2250" # commit containing fix
for MSVC issue https://github.com/civetweb/civetweb/issues/1024
+)
+
+FetchContent_MakeAvailable(civetweb)
+
+add_dependencies(civetweb-c-library OpenSSL::Crypto OpenSSL::SSL)
+add_dependencies(civetweb-cpp OpenSSL::Crypto OpenSSL::SSL)
+
+set(CIVETWEB_C_FLAGS "${CMAKE_CFLAGS} -Wno-error" CACHE STRING "" FORCE)
+set(CIVETWEB_CXX_LAGS "${CMAKE_CXX_FLAGS} -Wno-error" CACHE STRING "" FORCE)
+string(REPLACE " " ";" REPLACED_C_FLAGS ${CIVETWEB_C_FLAGS})
+string(REPLACE " " ";" REPLACED_CXX_FLAGS ${CIVETWEB_CXX_LAGS})
+target_compile_options(civetweb-c-library INTERFACE ${REPLACED_C_FLAGS})
+target_compile_options(civetweb-cpp INTERFACE ${REPLACED_CXX_FLAGS})
Review Comment:
This takes all of the global flags, appends `-Wno-error` and appends them to
anything linking to civet. Two problems:
1. It's a bit excessive to append all the flags from C/CXXFLAGS again to
civet. Anything before `-Wno-error` will be duplicated on the command line.
2. The flags should apply to civet itself, but not the dependencies. Use
`PRIVATE` instead of `INTERFACE`.
I'm not 100% sure, but the solution should be something like this:
```suggestion
target_compile_options(civetweb-c-library PRIVATE -Wno-error)
target_compile_options(civetweb-cpp PRIVATE -Wno-error)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm commented on a diff in pull request #1364: MINIFICPP-1878 Enable Ninja build in Windows builds
szaszm commented on code in PR #1364:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1364#discussion_r922107019
##
cmake/CivetWeb.cmake:
##
@@ -0,0 +1,46 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+include(FetchContent)
+
+set(CIVETWEB_ENABLE_SSL_DYNAMIC_LOADING "OFF" CACHE STRING "" FORCE)
+set(CIVETWEB_BUILD_TESTING "OFF" CACHE STRING "" FORCE)
+set(CIVETWEB_ENABLE_DUKTAPE "OFF" CACHE STRING "" FORCE)
+set(CIVETWEB_ENABLE_LUA "OFF" CACHE STRING "" FORCE)
+set(CIVETWEB_ENABLE_CXX "ON" CACHE STRING "" FORCE)
+set(CIVETWEB_ALLOW_WARNINGS "ON" CACHE STRING "" FORCE)
+set(CIVETWEB_ENABLE_ASAN "OFF" CACHE STRING "" FORCE)
+
+FetchContent_Declare(civetweb
+GIT_REPOSITORY "https://github.com/civetweb/civetweb.git;
+GIT_TAG "4447b6501d5c568b4c6c0940eac801ec690b2250" # commit containing fix
for MSVC issue https://github.com/civetweb/civetweb/issues/1024
+)
+
+FetchContent_MakeAvailable(civetweb)
+
+add_dependencies(civetweb-c-library OpenSSL::Crypto OpenSSL::SSL)
+add_dependencies(civetweb-cpp OpenSSL::Crypto OpenSSL::SSL)
+
+set(CIVETWEB_C_FLAGS "${CMAKE_CFLAGS} -Wno-error" CACHE STRING "" FORCE)
+set(CIVETWEB_CXX_LAGS "${CMAKE_CXX_FLAGS} -Wno-error" CACHE STRING "" FORCE)
+string(REPLACE " " ";" REPLACED_C_FLAGS ${CIVETWEB_C_FLAGS})
+string(REPLACE " " ";" REPLACED_CXX_FLAGS ${CIVETWEB_CXX_LAGS})
+target_compile_options(civetweb-c-library INTERFACE ${REPLACED_C_FLAGS})
+target_compile_options(civetweb-cpp INTERFACE ${REPLACED_CXX_FLAGS})
Review Comment:
This takes all of the global flags, appends `-Wno-error` and appends them to
anything linking to civet. Two problems:
1. It's a bit excessive to append all the flags from C/CXXFLAGS again to
civet. Anything before `-Wno-error` will be duplicated on the command line.
2. The flags should apply to civet itself, but not the dependencies. Use
`PRIVATE` instead of `INTERFACE`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Comment Edited] (NIFI-10235) Provenance replay fails when repository encryption is enabled
[
https://issues.apache.org/jira/browse/NIFI-10235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17567177#comment-17567177
]
Peter Kimberley edited comment on NIFI-10235 at 7/15/22 10:16 AM:
--
As an update, I've done a completely fresh, traditional install on a pristine
Rocky Linux 8.5 VM. No containers / Kubernetes.
Java version: OpenJDK 1.8.0.332
Same issue - once repository encryption is enabled, attempting to replay fails,
with a similar log message recorded. See attached: [^error-base-install.log]
NiFi flow is attached: [^NiFi_Flow.json]
was (Author: JIRAUSER292873):
As an update, I've done a completely fresh, traditional install on a pristine
Rocky Linux 8.5 VM. No containers / Kubernetes.
Java version: OpenJDK 1.8.0.332
Same issue - once repository encryption is enabled, attempting to replay fails,
with a similar log message recorded. See attached {*}error-base-install.log{*}.
NiFi flow is attached[^NiFi_Flow.json]
> Provenance replay fails when repository encryption is enabled
> -
>
> Key: NIFI-10235
> URL: https://issues.apache.org/jira/browse/NIFI-10235
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework, Security
>Affects Versions: 1.16.3
> Environment: RHEL 8.5
>Reporter: Peter Kimberley
>Priority: Major
> Labels: encryption, provenance, replay
> Attachments: NiFi_Flow.json, error-base-install.log, error.log
>
>
> h3. Problem summary
> When repository encryption is enabled, replaying a DROP provenance record
> fails, with the following error appearing in the logs:
> {quote}org.apache.nifi.processor.exception.FlowFileAccessException: Failed to
> export
> StandardFlowFileRecord[uuid=df985fc5-23da-4094-8783-2e0186bcb92d,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1657864218374-23, container=default,
> section=23], offset=379,
> length=1048576],offset=0,name=b29633c4-324e-42fe-b3e8-1ea455fc3650,size=1048576]
> to /opt/nifi/nifi-current/data/store/.b29633c4-324e-42fe-b3e8-1ea455fc3650
> due to java.io.EOFException: *Attempted to copy {color:#ff8b00}1048576{color}
> bytes but only {color:#ff8b00}1048197{color} bytes were available*
> {quote}
>
> I've observed that the difference between the sizes mentioned in the log is
> {+}*always 379 bytes*{+}, regardless of the length of the input file.
>
> With repository encryption disabled, provenance replay works as expected.
> h3. Configuration
> # NiFi v1.16.3 running as a three-node cluster in Kubernetes.
> # Each node has up to 8GB memory and 4 CPUs available to it.
> # Testing has included both NFS and ephemeral (emptyDir) storage.
> # The encryption key was generated by the following command, using the same
> JDK version:
> ## keytool -genseckey -alias key-1 -keyalg AES -keysize 256 -keystore
> repository.p12 -storetype PKCS12
> h4. nifi.properties
> {quote}nifi.repository.encryption.protocol.version=1
> nifi.repository.encryption.key.id=key-1
> nifi.repository.encryption.key.provider=KEYSTORE
> nifi.repository.encryption.key.provider.keystore.location=conf/repository.p12
> nifi.repository.encryption.key.provider.keystore.password=
> {quote}
> h3. Processor group
> GenerateFlowFile processor generating 1MB random files every second to a
> PutFile processor. Have also tested with InvokeHTTP.
> h3. Other comments
> With repository encryption enabled, I am able to download files via the
> provenance UI (suggesting that encryption/decryption works). The processor
> group also performs all other actions as expected.
> Not having the ability to replay provenance records is a blocker for our
> deployment, which requires data to be encrypted at rest and in transit.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-10235) Provenance replay fails when repository encryption is enabled
[
https://issues.apache.org/jira/browse/NIFI-10235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17567177#comment-17567177
]
Peter Kimberley commented on NIFI-10235:
As an update, I've done a completely fresh, traditional install on a pristine
Rocky Linux 8.5 VM. No containers / Kubernetes.
Java version: OpenJDK 1.8.0.332
Same issue - once repository encryption is enabled, attempting to replay fails,
with a similar log message recorded. See attached {*}error-base-install.log{*}.
NiFi flow is attached[^NiFi_Flow.json]
> Provenance replay fails when repository encryption is enabled
> -
>
> Key: NIFI-10235
> URL: https://issues.apache.org/jira/browse/NIFI-10235
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework, Security
>Affects Versions: 1.16.3
> Environment: RHEL 8.5
>Reporter: Peter Kimberley
>Priority: Major
> Labels: encryption, provenance, replay
> Attachments: NiFi_Flow.json, error-base-install.log, error.log
>
>
> h3. Problem summary
> When repository encryption is enabled, replaying a DROP provenance record
> fails, with the following error appearing in the logs:
> {quote}org.apache.nifi.processor.exception.FlowFileAccessException: Failed to
> export
> StandardFlowFileRecord[uuid=df985fc5-23da-4094-8783-2e0186bcb92d,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1657864218374-23, container=default,
> section=23], offset=379,
> length=1048576],offset=0,name=b29633c4-324e-42fe-b3e8-1ea455fc3650,size=1048576]
> to /opt/nifi/nifi-current/data/store/.b29633c4-324e-42fe-b3e8-1ea455fc3650
> due to java.io.EOFException: *Attempted to copy {color:#ff8b00}1048576{color}
> bytes but only {color:#ff8b00}1048197{color} bytes were available*
> {quote}
>
> I've observed that the difference between the sizes mentioned in the log is
> {+}*always 379 bytes*{+}, regardless of the length of the input file.
>
> With repository encryption disabled, provenance replay works as expected.
> h3. Configuration
> # NiFi v1.16.3 running as a three-node cluster in Kubernetes.
> # Each node has up to 8GB memory and 4 CPUs available to it.
> # Testing has included both NFS and ephemeral (emptyDir) storage.
> # The encryption key was generated by the following command, using the same
> JDK version:
> ## keytool -genseckey -alias key-1 -keyalg AES -keysize 256 -keystore
> repository.p12 -storetype PKCS12
> h4. nifi.properties
> {quote}nifi.repository.encryption.protocol.version=1
> nifi.repository.encryption.key.id=key-1
> nifi.repository.encryption.key.provider=KEYSTORE
> nifi.repository.encryption.key.provider.keystore.location=conf/repository.p12
> nifi.repository.encryption.key.provider.keystore.password=
> {quote}
> h3. Processor group
> GenerateFlowFile processor generating 1MB random files every second to a
> PutFile processor. Have also tested with InvokeHTTP.
> h3. Other comments
> With repository encryption enabled, I am able to download files via the
> provenance UI (suggesting that encryption/decryption works). The processor
> group also performs all other actions as expected.
> Not having the ability to replay provenance records is a blocker for our
> deployment, which requires data to be encrypted at rest and in transit.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-10235) Provenance replay fails when repository encryption is enabled
[
https://issues.apache.org/jira/browse/NIFI-10235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Kimberley updated NIFI-10235:
---
Attachment: NiFi_Flow.json
> Provenance replay fails when repository encryption is enabled
> -
>
> Key: NIFI-10235
> URL: https://issues.apache.org/jira/browse/NIFI-10235
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework, Security
>Affects Versions: 1.16.3
> Environment: RHEL 8.5
>Reporter: Peter Kimberley
>Priority: Major
> Labels: encryption, provenance, replay
> Attachments: NiFi_Flow.json, error-base-install.log, error.log
>
>
> h3. Problem summary
> When repository encryption is enabled, replaying a DROP provenance record
> fails, with the following error appearing in the logs:
> {quote}org.apache.nifi.processor.exception.FlowFileAccessException: Failed to
> export
> StandardFlowFileRecord[uuid=df985fc5-23da-4094-8783-2e0186bcb92d,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1657864218374-23, container=default,
> section=23], offset=379,
> length=1048576],offset=0,name=b29633c4-324e-42fe-b3e8-1ea455fc3650,size=1048576]
> to /opt/nifi/nifi-current/data/store/.b29633c4-324e-42fe-b3e8-1ea455fc3650
> due to java.io.EOFException: *Attempted to copy {color:#ff8b00}1048576{color}
> bytes but only {color:#ff8b00}1048197{color} bytes were available*
> {quote}
>
> I've observed that the difference between the sizes mentioned in the log is
> {+}*always 379 bytes*{+}, regardless of the length of the input file.
>
> With repository encryption disabled, provenance replay works as expected.
> h3. Configuration
> # NiFi v1.16.3 running as a three-node cluster in Kubernetes.
> # Each node has up to 8GB memory and 4 CPUs available to it.
> # Testing has included both NFS and ephemeral (emptyDir) storage.
> # The encryption key was generated by the following command, using the same
> JDK version:
> ## keytool -genseckey -alias key-1 -keyalg AES -keysize 256 -keystore
> repository.p12 -storetype PKCS12
> h4. nifi.properties
> {quote}nifi.repository.encryption.protocol.version=1
> nifi.repository.encryption.key.id=key-1
> nifi.repository.encryption.key.provider=KEYSTORE
> nifi.repository.encryption.key.provider.keystore.location=conf/repository.p12
> nifi.repository.encryption.key.provider.keystore.password=
> {quote}
> h3. Processor group
> GenerateFlowFile processor generating 1MB random files every second to a
> PutFile processor. Have also tested with InvokeHTTP.
> h3. Other comments
> With repository encryption enabled, I am able to download files via the
> provenance UI (suggesting that encryption/decryption works). The processor
> group also performs all other actions as expected.
> Not having the ability to replay provenance records is a blocker for our
> deployment, which requires data to be encrypted at rest and in transit.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-10235) Provenance replay fails when repository encryption is enabled
[
https://issues.apache.org/jira/browse/NIFI-10235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Kimberley updated NIFI-10235:
---
Environment: RHEL 8.5 (was: RHEL 8.5 / Kubernetes)
> Provenance replay fails when repository encryption is enabled
> -
>
> Key: NIFI-10235
> URL: https://issues.apache.org/jira/browse/NIFI-10235
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework, Security
>Affects Versions: 1.16.3
> Environment: RHEL 8.5
>Reporter: Peter Kimberley
>Priority: Major
> Labels: encryption, provenance, replay
> Attachments: error-base-install.log, error.log
>
>
> h3. Problem summary
> When repository encryption is enabled, replaying a DROP provenance record
> fails, with the following error appearing in the logs:
> {quote}org.apache.nifi.processor.exception.FlowFileAccessException: Failed to
> export
> StandardFlowFileRecord[uuid=df985fc5-23da-4094-8783-2e0186bcb92d,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1657864218374-23, container=default,
> section=23], offset=379,
> length=1048576],offset=0,name=b29633c4-324e-42fe-b3e8-1ea455fc3650,size=1048576]
> to /opt/nifi/nifi-current/data/store/.b29633c4-324e-42fe-b3e8-1ea455fc3650
> due to java.io.EOFException: *Attempted to copy {color:#ff8b00}1048576{color}
> bytes but only {color:#ff8b00}1048197{color} bytes were available*
> {quote}
>
> I've observed that the difference between the sizes mentioned in the log is
> {+}*always 379 bytes*{+}, regardless of the length of the input file.
>
> With repository encryption disabled, provenance replay works as expected.
> h3. Configuration
> # NiFi v1.16.3 running as a three-node cluster in Kubernetes.
> # Each node has up to 8GB memory and 4 CPUs available to it.
> # Testing has included both NFS and ephemeral (emptyDir) storage.
> # The encryption key was generated by the following command, using the same
> JDK version:
> ## keytool -genseckey -alias key-1 -keyalg AES -keysize 256 -keystore
> repository.p12 -storetype PKCS12
> h4. nifi.properties
> {quote}nifi.repository.encryption.protocol.version=1
> nifi.repository.encryption.key.id=key-1
> nifi.repository.encryption.key.provider=KEYSTORE
> nifi.repository.encryption.key.provider.keystore.location=conf/repository.p12
> nifi.repository.encryption.key.provider.keystore.password=
> {quote}
> h3. Processor group
> GenerateFlowFile processor generating 1MB random files every second to a
> PutFile processor. Have also tested with InvokeHTTP.
> h3. Other comments
> With repository encryption enabled, I am able to download files via the
> provenance UI (suggesting that encryption/decryption works). The processor
> group also performs all other actions as expected.
> Not having the ability to replay provenance records is a blocker for our
> deployment, which requires data to be encrypted at rest and in transit.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-10235) Provenance replay fails when repository encryption is enabled
[
https://issues.apache.org/jira/browse/NIFI-10235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Kimberley updated NIFI-10235:
---
Attachment: error-base-install.log
> Provenance replay fails when repository encryption is enabled
> -
>
> Key: NIFI-10235
> URL: https://issues.apache.org/jira/browse/NIFI-10235
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework, Security
>Affects Versions: 1.16.3
> Environment: RHEL 8.5
>Reporter: Peter Kimberley
>Priority: Major
> Labels: encryption, provenance, replay
> Attachments: error-base-install.log, error.log
>
>
> h3. Problem summary
> When repository encryption is enabled, replaying a DROP provenance record
> fails, with the following error appearing in the logs:
> {quote}org.apache.nifi.processor.exception.FlowFileAccessException: Failed to
> export
> StandardFlowFileRecord[uuid=df985fc5-23da-4094-8783-2e0186bcb92d,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1657864218374-23, container=default,
> section=23], offset=379,
> length=1048576],offset=0,name=b29633c4-324e-42fe-b3e8-1ea455fc3650,size=1048576]
> to /opt/nifi/nifi-current/data/store/.b29633c4-324e-42fe-b3e8-1ea455fc3650
> due to java.io.EOFException: *Attempted to copy {color:#ff8b00}1048576{color}
> bytes but only {color:#ff8b00}1048197{color} bytes were available*
> {quote}
>
> I've observed that the difference between the sizes mentioned in the log is
> {+}*always 379 bytes*{+}, regardless of the length of the input file.
>
> With repository encryption disabled, provenance replay works as expected.
> h3. Configuration
> # NiFi v1.16.3 running as a three-node cluster in Kubernetes.
> # Each node has up to 8GB memory and 4 CPUs available to it.
> # Testing has included both NFS and ephemeral (emptyDir) storage.
> # The encryption key was generated by the following command, using the same
> JDK version:
> ## keytool -genseckey -alias key-1 -keyalg AES -keysize 256 -keystore
> repository.p12 -storetype PKCS12
> h4. nifi.properties
> {quote}nifi.repository.encryption.protocol.version=1
> nifi.repository.encryption.key.id=key-1
> nifi.repository.encryption.key.provider=KEYSTORE
> nifi.repository.encryption.key.provider.keystore.location=conf/repository.p12
> nifi.repository.encryption.key.provider.keystore.password=
> {quote}
> h3. Processor group
> GenerateFlowFile processor generating 1MB random files every second to a
> PutFile processor. Have also tested with InvokeHTTP.
> h3. Other comments
> With repository encryption enabled, I am able to download files via the
> provenance UI (suggesting that encryption/decryption works). The processor
> group also performs all other actions as expected.
> Not having the ability to replay provenance records is a blocker for our
> deployment, which requires data to be encrypted at rest and in transit.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi-minifi-cpp] lordgamez opened a new pull request, #1370: MINIFICPP-1866 Secure connection for ListenSyslog, ListenTCP
lordgamez opened a new pull request, #1370: URL: https://github.com/apache/nifi-minifi-cpp/pull/1370 https://issues.apache.org/jira/browse/MINIFICPP-1866 Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFICPP- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically main)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check GitHub Actions CI results for build issues and submit an update to your PR as soon as possible. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1364: MINIFICPP-1878 Enable Ninja build in Windows builds
lordgamez commented on code in PR #1364:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1364#discussion_r922011417
##
cmake/CivetWeb.cmake:
##
@@ -0,0 +1,42 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+include(FetchContent)
+
+if (NOT WIN32)
+ set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wno-error" CACHE STRING "" FORCE)
+ set(CMAKE_CFLAGS "${CMAKE_CFLAGS} -Wno-error" CACHE STRING "" FORCE)
+endif()
Review Comment:
I'm not sure if they could affect other modules, but it could be possible
that the targets defined after civetweb would be affected. To make sure it
doesn't happen I updated to only set target specific flags in civetweb in
4e37ffca0fc66548cf89b4738a8f73a7f9a5645e
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (NIFI-10238) Priority queues may return incorrect status when flowfiles have just been migrated from swap to active
[ https://issues.apache.org/jira/browse/NIFI-10238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hsin-Ying Lee updated NIFI-10238: - Status: Patch Available (was: In Progress) > Priority queues may return incorrect status when flowfiles have just been > migrated from swap to active > -- > > Key: NIFI-10238 > URL: https://issues.apache.org/jira/browse/NIFI-10238 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.16.3, 1.16.2, 1.16.1, 1.16.0 >Reporter: Hsin-Ying Lee >Assignee: Hsin-Ying Lee >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > The call of `getFlowFileAvailability` returns incorrect status when the > active queue is empty and there are flowfiles swapped out -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] s9514171 opened a new pull request, #6212: NIFI-10238 Priority queues may return incorrect status when flowfiles…
s9514171 opened a new pull request, #6212: URL: https://github.com/apache/nifi/pull/6212 … have just been migrated from swap to active # Summary [NIFI-10238](https://issues.apache.org/jira/browse/NIFI-10238) The call of `getFlowFileAvailability` returns incorrect status when the active queue is empty and there are flowfiles swapped out # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [X] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 8 - [ ] JDK 11 - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] mathemare opened a new pull request, #6211: NIFI-10236: ByteArrayContentRepository in core framework
mathemare opened a new pull request, #6211: URL: https://github.com/apache/nifi/pull/6211 New ByteArrayContentRepository for the nifi-framework-core - added implementation of the ByteArray ContentClaim / ResourceClaim - added an (useful ?) interface for the StandardContentClaimWriteCache not to only take into account StandardContentClaim but CachableContentClaim that is implemented by the ByteArrayContentClaim - added tests from the VolatileContentRepository Signed-off-by: mathemare # Summary [NIFI-0](https://issues.apache.org/jira/browse/NIFI-0) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 8 - [ ] JDK 11 - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (NIFI-10236) ByteArrayContentRepository in core framework
[ https://issues.apache.org/jira/browse/NIFI-10236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthieu RÉ reassigned NIFI-10236: -- Assignee: Matthieu RÉ > ByteArrayContentRepository in core framework > > > Key: NIFI-10236 > URL: https://issues.apache.org/jira/browse/NIFI-10236 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework >Affects Versions: 1.17.0 >Reporter: Matthieu RÉ >Assignee: Matthieu RÉ >Priority: Major > > As discussed in NIFI-8760 and by mail, the use case of using an in-memory > based implementation of the ContentRepository is not completed after 1.17.0 > and the removal of the old VolatileContentRepository (NIFI-9823) when the > ExecuteStateless is not a solution. > As proposed by mail, the ByteArrayContentRepository could be duplicated from > the nifi-stateless framework to the core one, but maybe with more > functionalities, for instance the follow-up of the space taken in memory, and > the limitation of its total usage. > Don't hesitate to comment, add or discuss requirements or the implementation > I will propose. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi-minifi-cpp] szaszm commented on a diff in pull request #1364: MINIFICPP-1878 Enable Ninja build in Windows builds
szaszm commented on code in PR #1364:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1364#discussion_r921981470
##
cmake/CivetWeb.cmake:
##
@@ -0,0 +1,42 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+include(FetchContent)
+
+if (NOT WIN32)
+ set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wno-error" CACHE STRING "" FORCE)
+ set(CMAKE_CFLAGS "${CMAKE_CFLAGS} -Wno-error" CACHE STRING "" FORCE)
+endif()
Review Comment:
Is this leaking outside of the flags of this module? I wouldn't want to
accidentally disable Werror globally if we have `FAIL_ON_WARNINGS=ON`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1364: MINIFICPP-1878 Enable Ninja build in Windows builds
lordgamez commented on code in PR #1364: URL: https://github.com/apache/nifi-minifi-cpp/pull/1364#discussion_r921955068 ## win_build_vs.bat: ## @@ -73,12 +73,18 @@ for %%x in (%*) do ( if [%%~x] EQU [/NONFREEUCRT] set "redist=-DMSI_REDISTRIBUTE_UCRT_NONASL=ON" if [%%~x] EQU [/L] set build_linter=ON if [%%~x] EQU [/RO] set real_odbc=ON +if [%%~x] EQU [/NJ] set generator="Ninja" Review Comment: I think it's better to be more explicit, updated in 5aa6d9a5942b642f49b0903ee80d1f74fc0ec82c -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1364: MINIFICPP-1878 Enable Ninja build in Windows builds
lordgamez commented on code in PR #1364: URL: https://github.com/apache/nifi-minifi-cpp/pull/1364#discussion_r921954555 ## win_build_vs.bat: ## @@ -73,12 +73,18 @@ for %%x in (%*) do ( if [%%~x] EQU [/NONFREEUCRT] set "redist=-DMSI_REDISTRIBUTE_UCRT_NONASL=ON" if [%%~x] EQU [/L] set build_linter=ON if [%%~x] EQU [/RO] set real_odbc=ON +if [%%~x] EQU [/NJ] set generator="Ninja" ) mkdir %builddir% pushd %builddir%\ -cmake -G %generator% -A %build_platform% -DINSTALLER_MERGE_MODULES=%installer_merge_modules% -DTEST_CUSTOM_WEL_PROVIDER=%test_custom_wel_provider% -DENABLE_SQL=%build_SQL% -DUSE_REAL_ODBC_TEST_DRIVER=%real_odbc% -DCMAKE_BUILD_TYPE_INIT=%cmake_build_type% -DCMAKE_BUILD_TYPE=%cmake_build_type% -DWIN32=WIN32 -DENABLE_LIBRDKAFKA=%build_kafka% -DENABLE_JNI=%build_jni% -DOPENSSL_OFF=OFF -DENABLE_COAP=%build_coap% -DENABLE_AWS=%build_AWS% -DENABLE_PDH=%build_PDH% -DENABLE_AZURE=%build_azure% -DENABLE_SFTP=%build_SFTP% -DENABLE_SPLUNK=%build_SPLUNK% -DENABLE_GCP=%build_GCP% -DENABLE_NANOFI=%build_nanofi% -DENABLE_OPENCV=%build_opencv% -DENABLE_PROMETHEUS=%build_prometheus% -DENABLE_ELASTICSEARCH=%build_ELASTIC% -DUSE_SHARED_LIBS=OFF -DDISABLE_CONTROLLER=ON -DBUILD_ROCKSDB=ON -DFORCE_WINDOWS=ON -DUSE_SYSTEM_UUID=OFF -DDISABLE_LIBARCHIVE=OFF -DENABLE_SCRIPTING=OFF -DEXCLUDE_BOOST=ON -DENABLE_WEL=ON -DFAIL_ON_WARNINGS=OFF -DSKIP_TESTS=%skiptests% %strict_gsl_checks% %redist% -DENABLE_LINTER=% build_linter% "%scriptdir%" && msbuild /m nifi-minifi-cpp.sln /property:Configuration=%cmake_build_type% /property:Platform=%build_platform% && copy bin\%cmake_build_type%\minifi.exe main\ +if [%generator%] EQU ["Ninja"] ( +set "buildcmd=ninja" +) else ( +set "buildcmd=msbuild /m nifi-minifi-cpp.sln /property:Configuration=%cmake_build_type% /property:Platform=%build_platform% && copy bin\%cmake_build_type%\minifi.exe main\" Review Comment: Good catch, it should be copied, updated in 5aa6d9a5942b642f49b0903ee80d1f74fc0ec82c -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-10239) Fix Curator dependencies in Hive3 module
[
https://issues.apache.org/jira/browse/NIFI-10239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Turcsanyi updated NIFI-10239:
---
Status: Patch Available (was: Open)
> Fix Curator dependencies in Hive3 module
>
>
> Key: NIFI-10239
> URL: https://issues.apache.org/jira/browse/NIFI-10239
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Peter Turcsanyi
>Assignee: Peter Turcsanyi
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Error:
> {code:java}
> 2022-07-14 23:15:15,741 WARN [Timer-Driven Process Thread-6]
> o.a.n.controller.tasks.ConnectableTask Processing halted: uncaught exception
> in Component [SelectHive3QL[id=e22cea71-f161-381e-c2c8-f241c6f363f7]]
> java.lang.NoSuchMethodError:
> org.apache.curator.shaded.com.google.common.util.concurrent.MoreExecutors.sameThreadExecutor()Lorg/apache/curator/shaded/com/google/common/util/concurrent/ListeningExecutorService;
> at
> org.apache.curator.framework.listen.ListenerContainer.addListener(ListenerContainer.java:41)
> at
> org.apache.curator.framework.imps.CuratorFrameworkImpl.start(CuratorFrameworkImpl.java:257)
> at
> org.apache.hive.jdbc.ZooKeeperHiveClientHelper.getZkClient(ZooKeeperHiveClientHelper.java:92)
> at
> org.apache.hive.jdbc.ZooKeeperHiveClientHelper.configureConnParams(ZooKeeperHiveClientHelper.java:141)
> at
> org.apache.hive.jdbc.Utils.configureConnParamsFromZooKeeper(Utils.java:511)
> at org.apache.hive.jdbc.Utils.parseURL(Utils.java:334)
> at org.apache.hive.jdbc.HiveConnection.(HiveConnection.java:168)
> at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:107)
> {code}
> RCA:
> NIFI-10145 upgraded to Hadoop 3.3.3.
> In Hive3 module, it brings curator-client:4.2.0 in but hive-common:3.1.3
> still depends on curator-framework 2.12.0 which would need a compatible
> version of curator-client.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi] turcsanyip opened a new pull request, #6210: NIFI-10239: Fixed Curator dependencies in Hive3 module
turcsanyip opened a new pull request, #6210: URL: https://github.com/apache/nifi/pull/6210 # Summary [NIFI-10239](https://issues.apache.org/jira/browse/NIFI-10239) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 8 - [ ] JDK 11 - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-10239) Fix Curator dependencies in Hive3 module
Peter Turcsanyi created NIFI-10239:
--
Summary: Fix Curator dependencies in Hive3 module
Key: NIFI-10239
URL: https://issues.apache.org/jira/browse/NIFI-10239
Project: Apache NiFi
Issue Type: Bug
Reporter: Peter Turcsanyi
Assignee: Peter Turcsanyi
Error:
{code:java}
2022-07-14 23:15:15,741 WARN [Timer-Driven Process Thread-6]
o.a.n.controller.tasks.ConnectableTask Processing halted: uncaught exception in
Component [SelectHive3QL[id=e22cea71-f161-381e-c2c8-f241c6f363f7]]
java.lang.NoSuchMethodError:
org.apache.curator.shaded.com.google.common.util.concurrent.MoreExecutors.sameThreadExecutor()Lorg/apache/curator/shaded/com/google/common/util/concurrent/ListeningExecutorService;
at
org.apache.curator.framework.listen.ListenerContainer.addListener(ListenerContainer.java:41)
at
org.apache.curator.framework.imps.CuratorFrameworkImpl.start(CuratorFrameworkImpl.java:257)
at
org.apache.hive.jdbc.ZooKeeperHiveClientHelper.getZkClient(ZooKeeperHiveClientHelper.java:92)
at
org.apache.hive.jdbc.ZooKeeperHiveClientHelper.configureConnParams(ZooKeeperHiveClientHelper.java:141)
at
org.apache.hive.jdbc.Utils.configureConnParamsFromZooKeeper(Utils.java:511)
at org.apache.hive.jdbc.Utils.parseURL(Utils.java:334)
at org.apache.hive.jdbc.HiveConnection.(HiveConnection.java:168)
at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:107)
{code}
RCA:
NIFI-10145 upgraded to Hadoop 3.3.3.
In Hive3 module, it brings curator-client:4.2.0 in but hive-common:3.1.3 still
depends on curator-framework 2.12.0 which would need a compatible version of
curator-client.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (NIFI-10238) Priority queues may return incorrect status when flowfiles have just been migrated from swap to active
Hsin-Ying Lee created NIFI-10238: Summary: Priority queues may return incorrect status when flowfiles have just been migrated from swap to active Key: NIFI-10238 URL: https://issues.apache.org/jira/browse/NIFI-10238 Project: Apache NiFi Issue Type: Bug Components: Core Framework Affects Versions: 1.16.3, 1.16.2, 1.16.1, 1.16.0 Reporter: Hsin-Ying Lee Assignee: Hsin-Ying Lee The call of `getFlowFileAvailability` returns incorrect status when the active queue is empty and there are flowfiles swapped out -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-10237) ByteArrayContentRepository in core framework
[ https://issues.apache.org/jira/browse/NIFI-10237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthieu RÉ resolved NIFI-10237. Resolution: Duplicate > ByteArrayContentRepository in core framework > > > Key: NIFI-10237 > URL: https://issues.apache.org/jira/browse/NIFI-10237 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework >Affects Versions: 1.17.0 >Reporter: Matthieu RÉ >Priority: Major > > As discussed in NIFI-8760 and by mail, the use case of using an in-memory > based implementation of the ContentRepository is not completed after 1.17.0 > and the removal of the old VolatileContentRepository (NIFI-9823) when the > ExecuteStateless is not a solution. > As proposed by mail, the ByteArrayContentRepository could be duplicated from > the nifi-stateless framework to the core one, but maybe with more > functionalities, for instance the follow-up of the space taken in memory, and > the limitation of its total usage. > Don't hesitate to comment, add or discuss requirements or the implementation > I will propose. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-10237) ByteArrayContentRepository in core framework
Matthieu RÉ created NIFI-10237: -- Summary: ByteArrayContentRepository in core framework Key: NIFI-10237 URL: https://issues.apache.org/jira/browse/NIFI-10237 Project: Apache NiFi Issue Type: New Feature Components: Core Framework Affects Versions: 1.17.0 Reporter: Matthieu RÉ As discussed in NIFI-8760 and by mail, the use case of using an in-memory based implementation of the ContentRepository is not completed after 1.17.0 and the removal of the old VolatileContentRepository (NIFI-9823) when the ExecuteStateless is not a solution. As proposed by mail, the ByteArrayContentRepository could be duplicated from the nifi-stateless framework to the core one, but maybe with more functionalities, for instance the follow-up of the space taken in memory, and the limitation of its total usage. Don't hesitate to comment, add or discuss requirements or the implementation I will propose. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-10236) ByteArrayContentRepository in core framework
Matthieu RÉ created NIFI-10236: -- Summary: ByteArrayContentRepository in core framework Key: NIFI-10236 URL: https://issues.apache.org/jira/browse/NIFI-10236 Project: Apache NiFi Issue Type: New Feature Components: Core Framework Affects Versions: 1.17.0 Reporter: Matthieu RÉ As discussed in NIFI-8760 and by mail, the use case of using an in-memory based implementation of the ContentRepository is not completed after 1.17.0 and the removal of the old VolatileContentRepository (NIFI-9823) when the ExecuteStateless is not a solution. As proposed by mail, the ByteArrayContentRepository could be duplicated from the nifi-stateless framework to the core one, but maybe with more functionalities, for instance the follow-up of the space taken in memory, and the limitation of its total usage. Don't hesitate to comment, add or discuss requirements or the implementation I will propose. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-10235) Provenance replay fails when repository encryption is enabled
[
https://issues.apache.org/jira/browse/NIFI-10235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Kimberley updated NIFI-10235:
---
Description:
h3. Problem summary
When repository encryption is enabled, replaying a DROP provenance record
fails, with the following error appearing in the logs:
{quote}org.apache.nifi.processor.exception.FlowFileAccessException: Failed to
export
StandardFlowFileRecord[uuid=df985fc5-23da-4094-8783-2e0186bcb92d,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=1657864218374-23, container=default,
section=23], offset=379,
length=1048576],offset=0,name=b29633c4-324e-42fe-b3e8-1ea455fc3650,size=1048576]
to /opt/nifi/nifi-current/data/store/.b29633c4-324e-42fe-b3e8-1ea455fc3650 due
to java.io.EOFException: *Attempted to copy {color:#ff8b00}1048576{color} bytes
but only {color:#ff8b00}1048197{color} bytes were available*
{quote}
I've observed that the difference between the sizes mentioned in the log is
{+}*always 379 bytes*{+}, regardless of the length of the input file.
With repository encryption disabled, provenance replay works as expected.
h3. Configuration
# NiFi v1.16.3 running as a three-node cluster in Kubernetes.
# Each node has up to 8GB memory and 4 CPUs available to it.
# Testing has included both NFS and ephemeral (emptyDir) storage.
# The encryption key was generated by the following command, using the same
JDK version:
## keytool -genseckey -alias key-1 -keyalg AES -keysize 256 -keystore
repository.p12 -storetype PKCS12
h4. nifi.properties
{quote}nifi.repository.encryption.protocol.version=1
nifi.repository.encryption.key.id=key-1
nifi.repository.encryption.key.provider=KEYSTORE
nifi.repository.encryption.key.provider.keystore.location=conf/repository.p12
nifi.repository.encryption.key.provider.keystore.password=
{quote}
h3. Processor group
GenerateFlowFile processor generating 1MB random files every second to a
PutFile processor. Have also tested with InvokeHTTP.
h3. Other comments
With repository encryption enabled, I am able to download files via the
provenance UI (suggesting that encryption/decryption works). The processor
group also performs all other actions as expected.
Not having the ability to replay provenance records is a blocker for our
deployment, which requires data to be encrypted at rest and in transit.
was:
h3. Problem summary
When repository encryption is enabled, replaying a DROP provenance record
fails, with the following error appearing in the logs:
{quote}org.apache.nifi.processor.exception.FlowFileAccessException: Failed to
export
StandardFlowFileRecord[uuid=df985fc5-23da-4094-8783-2e0186bcb92d,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=1657864218374-23, container=default,
section=23], offset=379,
length=1048576],offset=0,name=b29633c4-324e-42fe-b3e8-1ea455fc3650,size=1048576]
to /opt/nifi/nifi-current/data/store/.b29633c4-324e-42fe-b3e8-1ea455fc3650 due
to java.io.EOFException: *Attempted to copy 1048576 bytes but only 1048197
bytes were available*{quote}
The difference between the two size bytes is {+}*always 379*{+}, regardless of
the length of the input file.
With repository encryption disabled, provenance replay works as expected.
h3. Configuration
# NiFi v1.16.3 running as a three-node cluster in Kubernetes.
# Each node has up to 8GB memory and 4 CPUs available to it.
# Testing has included both NFS and ephemeral (emptyDir) storage.
# The encryption key was generated by the following command, using the same
JDK version:
## keytool -genseckey -alias key-1 -keyalg AES -keysize 256 -keystore
repository.p12 -storetype PKCS12
h4. nifi.properties
{quote}nifi.repository.encryption.protocol.version=1
nifi.repository.encryption.key.id=key-1
nifi.repository.encryption.key.provider=KEYSTORE
nifi.repository.encryption.key.provider.keystore.location=conf/repository.p12
nifi.repository.encryption.key.provider.keystore.password={quote}
h3. Processor group
GenerateFlowFile processor generating 1MB random files every second to a
PutFile processor. Have also tested with InvokeHTTP.
h3. Other comments
With repository encryption enabled, I am able to download files via the
provenance UI (suggesting that encryption/decryption works). The processor
group also performs all other actions as expected.
Not having the ability to replay provenance records is a blocker for our
deployment, which requires data to be encrypted at rest and in transit.
> Provenance replay fails when repository encryption is enabled
> -
>
> Key: NIFI-10235
> URL: https://issues.apache.org/jira/browse/NIFI-10235
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework, Security
>Affects Versions: 1.16.3
> Environment: RHEL 8.5 / Kubernetes
>Reporter: Peter
[jira] [Resolved] (MINIFICPP-1881) Add SSL support to ListenSyslog and ListenTCP processors
[ https://issues.apache.org/jira/browse/MINIFICPP-1881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gábor Gyimesi resolved MINIFICPP-1881. -- Resolution: Duplicate > Add SSL support to ListenSyslog and ListenTCP processors > > > Key: MINIFICPP-1881 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1881 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Gábor Gyimesi >Assignee: Gábor Gyimesi >Priority: Minor > > Add SSLContextService property to ListenTCP and ListenSyslog processors to > retrieve SSL information and extend them to support SSL connection. Asio > provides SSL context for this purpose. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-10235) Provenance replay fails when repository encryption is enabled
Peter Kimberley created NIFI-10235:
--
Summary: Provenance replay fails when repository encryption is
enabled
Key: NIFI-10235
URL: https://issues.apache.org/jira/browse/NIFI-10235
Project: Apache NiFi
Issue Type: Bug
Components: Core Framework, Security
Affects Versions: 1.16.3
Environment: RHEL 8.5 / Kubernetes
Reporter: Peter Kimberley
Attachments: error.log
h3. Problem summary
When repository encryption is enabled, replaying a DROP provenance record
fails, with the following error appearing in the logs:
{quote}org.apache.nifi.processor.exception.FlowFileAccessException: Failed to
export
StandardFlowFileRecord[uuid=df985fc5-23da-4094-8783-2e0186bcb92d,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=1657864218374-23, container=default,
section=23], offset=379,
length=1048576],offset=0,name=b29633c4-324e-42fe-b3e8-1ea455fc3650,size=1048576]
to /opt/nifi/nifi-current/data/store/.b29633c4-324e-42fe-b3e8-1ea455fc3650 due
to java.io.EOFException: *Attempted to copy 1048576 bytes but only 1048197
bytes were available*{quote}
The difference between the two size bytes is {+}*always 379*{+}, regardless of
the length of the input file.
With repository encryption disabled, provenance replay works as expected.
h3. Configuration
# NiFi v1.16.3 running as a three-node cluster in Kubernetes.
# Each node has up to 8GB memory and 4 CPUs available to it.
# Testing has included both NFS and ephemeral (emptyDir) storage.
# The encryption key was generated by the following command, using the same
JDK version:
## keytool -genseckey -alias key-1 -keyalg AES -keysize 256 -keystore
repository.p12 -storetype PKCS12
h4. nifi.properties
{quote}nifi.repository.encryption.protocol.version=1
nifi.repository.encryption.key.id=key-1
nifi.repository.encryption.key.provider=KEYSTORE
nifi.repository.encryption.key.provider.keystore.location=conf/repository.p12
nifi.repository.encryption.key.provider.keystore.password={quote}
h3. Processor group
GenerateFlowFile processor generating 1MB random files every second to a
PutFile processor. Have also tested with InvokeHTTP.
h3. Other comments
With repository encryption enabled, I am able to download files via the
provenance UI (suggesting that encryption/decryption works). The processor
group also performs all other actions as expected.
Not having the ability to replay provenance records is a blocker for our
deployment, which requires data to be encrypted at rest and in transit.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi-minifi-cpp] szaszm commented on a diff in pull request #1369: MINIFICPP-1874 docker-verify should only run tests that are enabled
szaszm commented on code in PR #1369:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1369#discussion_r921862093
##
docker/DockerVerify.sh:
##
@@ -67,11 +67,11 @@ TEST_DIRECTORY="${docker_dir}/test/integration"
export TEST_DIRECTORY
# Add --no-logcapture to see logs interleaved with the test output
-BEHAVE_OPTS=(-f pretty --logging-level INFO --logging-clear-handlers --tags
~@no-ci)
Review Comment:
My point is that there would be no harm in keeping the tag, even if unused.
The added complexity is minimal, and there may be cases in the future where we
need to disable individual (e.g. long running) tests, not the whole extension.
It can also be readded, so it's fine if you prefer to still remove it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] kulikg commented on a diff in pull request #6192: NIFI-10212 added ListSmb processor and SmbConnectionPoolService
kulikg commented on code in PR #6192:
URL: https://github.com/apache/nifi/pull/6192#discussion_r921856706
##
nifi-nar-bundles/nifi-smb-bundle/nifi-smb-connection-pool/src/main/java/org/apache/nifi/services/smb/SmbjSessionProviderService.java:
##
@@ -0,0 +1,175 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.services.smb;
+
+import static java.util.Arrays.asList;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static
org.apache.nifi.expression.ExpressionLanguageScope.FLOWFILE_ATTRIBUTES;
+import static
org.apache.nifi.processor.util.StandardValidators.INTEGER_VALIDATOR;
+import static
org.apache.nifi.processor.util.StandardValidators.NON_BLANK_VALIDATOR;
+import static
org.apache.nifi.processor.util.StandardValidators.NON_EMPTY_VALIDATOR;
+import static org.apache.nifi.processor.util.StandardValidators.PORT_VALIDATOR;
+
+import com.hierynomus.smbj.SMBClient;
+import com.hierynomus.smbj.SmbConfig;
+import com.hierynomus.smbj.auth.AuthenticationContext;
+import com.hierynomus.smbj.session.Session;
+import com.hierynomus.smbj.transport.tcp.async.AsyncDirectTcpTransportFactory;
+import java.io.IOException;
+import java.io.UncheckedIOException;
+import java.net.URI;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.List;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnDisabled;
+import org.apache.nifi.annotation.lifecycle.OnEnabled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.controller.AbstractControllerService;
+import org.apache.nifi.controller.ConfigurationContext;
+
+@Tags({"microsoft", "samba"})
+@CapabilityDescription("Provides connection pool for ListSmb processor. ")
+public class SmbjSessionProviderService extends AbstractControllerService
implements SmbSessionProviderService {
+
+public static final PropertyDescriptor HOSTNAME = new
PropertyDescriptor.Builder()
+.displayName("Hostname")
+.name("hostname")
+.description("The network host of the SMB file server.")
+.required(false)
+.expressionLanguageSupported(FLOWFILE_ATTRIBUTES)
+.addValidator(NON_BLANK_VALIDATOR)
+.build();
+public static final PropertyDescriptor DOMAIN = new
PropertyDescriptor.Builder()
+.displayName("Domain")
+.name("domain")
+.description(
+"The domain used for authentication. Optional, in most
cases username and password is sufficient.")
+.required(false)
+.addValidator(NON_EMPTY_VALIDATOR)
+.build();
+public static final PropertyDescriptor USERNAME = new
PropertyDescriptor.Builder()
+.displayName("Username")
+.name("username")
+.description(
+"The username used for authentication.")
+.required(false)
+.defaultValue("Guest")
+.addValidator(NON_EMPTY_VALIDATOR)
+.build();
+public static final PropertyDescriptor PASSWORD = new
PropertyDescriptor.Builder()
+.displayName("Password")
+.name("password")
+.description("The password used for authentication.")
+.required(false)
+.addValidator(NON_EMPTY_VALIDATOR)
+.sensitive(true)
+.build();
+public static final PropertyDescriptor PORT = new
PropertyDescriptor.Builder()
+.displayName("Port")
+.name("port")
+.description("Port to use for connection.")
+.required(true)
+.addValidator(PORT_VALIDATOR)
+.defaultValue("445")
+.build();
+public static final PropertyDescriptor TIMEOUT = new
PropertyDescriptor.Builder()
+.displayName("Timeout")
+.name("timeout")
+.description("Timeout in seconds for read and write operations.")
+.required(true)
+
[GitHub] [nifi] kulikg commented on pull request #6192: NIFI-10212 added ListSmb processor and SmbConnectionPoolService
kulikg commented on PR #6192: URL: https://github.com/apache/nifi/pull/6192#issuecomment-1185218734 > @kulikg - Did you have the intention of refactoring the existing GetSmb and PutSmb processors to utilise the client? Or do you think this is a body of work for later? It would be really nice to refactor them but I'm not planning to do so in this PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] kulikg commented on a diff in pull request #6192: NIFI-10212 added ListSmb processor and SmbConnectionPoolService
kulikg commented on code in PR #6192:
URL: https://github.com/apache/nifi/pull/6192#discussion_r921842384
##
nifi-nar-bundles/nifi-smb-bundle/nifi-smb-processors/src/main/java/org/apache/nifi/processors/smb/NiFiSmbClient.java:
##
@@ -0,0 +1,156 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.smb;
+
+import static java.util.Arrays.asList;
+import static java.util.stream.StreamSupport.stream;
+
+import com.hierynomus.msdtyp.AccessMask;
+import com.hierynomus.msfscc.FileAttributes;
+import com.hierynomus.msfscc.fileinformation.FileIdBothDirectoryInformation;
+import com.hierynomus.mssmb2.SMB2CreateDisposition;
+import com.hierynomus.mssmb2.SMB2CreateOptions;
+import com.hierynomus.mssmb2.SMB2ShareAccess;
+import com.hierynomus.mssmb2.SMBApiException;
+import com.hierynomus.smbj.session.Session;
+import com.hierynomus.smbj.share.Directory;
+import com.hierynomus.smbj.share.DiskShare;
+import com.hierynomus.smbj.share.File;
+import java.io.IOException;
+import java.io.OutputStream;
+import java.io.UncheckedIOException;
+import java.util.EnumSet;
+import java.util.List;
+import java.util.stream.Stream;
+
+public class NiFiSmbClient {
+
+private static final List SPECIAL_DIRECTORIES = asList(".", "..");
Review Comment:
. and .. are special inode mappings on linux/mac. This is why I gave this
name to this variable. Any folder could be hidden. So I'm not sure if
HIDDEN_DIRECTORIES is a better name. Using that name we would read in the code:
filter(this::hiddenDirectories) which would make me think we filter every
hidden directory which is not the case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] kulikg commented on a diff in pull request #6192: NIFI-10212 added ListSmb processor and SmbConnectionPoolService
kulikg commented on code in PR #6192:
URL: https://github.com/apache/nifi/pull/6192#discussion_r921839944
##
nifi-nar-bundles/nifi-smb-bundle/nifi-smb-connection-pool/src/main/java/org/apache/nifi/services/smb/SmbjSessionProviderService.java:
##
@@ -0,0 +1,175 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.services.smb;
+
+import static java.util.Arrays.asList;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static
org.apache.nifi.expression.ExpressionLanguageScope.FLOWFILE_ATTRIBUTES;
+import static
org.apache.nifi.processor.util.StandardValidators.INTEGER_VALIDATOR;
+import static
org.apache.nifi.processor.util.StandardValidators.NON_BLANK_VALIDATOR;
+import static
org.apache.nifi.processor.util.StandardValidators.NON_EMPTY_VALIDATOR;
+import static org.apache.nifi.processor.util.StandardValidators.PORT_VALIDATOR;
+
+import com.hierynomus.smbj.SMBClient;
+import com.hierynomus.smbj.SmbConfig;
+import com.hierynomus.smbj.auth.AuthenticationContext;
+import com.hierynomus.smbj.session.Session;
+import com.hierynomus.smbj.transport.tcp.async.AsyncDirectTcpTransportFactory;
+import java.io.IOException;
+import java.io.UncheckedIOException;
+import java.net.URI;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.List;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnDisabled;
+import org.apache.nifi.annotation.lifecycle.OnEnabled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.controller.AbstractControllerService;
+import org.apache.nifi.controller.ConfigurationContext;
+
+@Tags({"microsoft", "samba"})
+@CapabilityDescription("Provides connection pool for ListSmb processor. ")
+public class SmbjSessionProviderService extends AbstractControllerService
implements SmbSessionProviderService {
+
+public static final PropertyDescriptor HOSTNAME = new
PropertyDescriptor.Builder()
+.displayName("Hostname")
+.name("hostname")
+.description("The network host of the SMB file server.")
+.required(false)
+.expressionLanguageSupported(FLOWFILE_ATTRIBUTES)
+.addValidator(NON_BLANK_VALIDATOR)
+.build();
+public static final PropertyDescriptor DOMAIN = new
PropertyDescriptor.Builder()
+.displayName("Domain")
+.name("domain")
+.description(
+"The domain used for authentication. Optional, in most
cases username and password is sufficient.")
+.required(false)
+.addValidator(NON_EMPTY_VALIDATOR)
+.build();
+public static final PropertyDescriptor USERNAME = new
PropertyDescriptor.Builder()
+.displayName("Username")
+.name("username")
+.description(
+"The username used for authentication.")
+.required(false)
+.defaultValue("Guest")
Review Comment:
in the auth context you have 2 options to authenticate without any
credentials:
AuthenticationContext.anonymous();
AuthenticationContext.guest();
guest() will authenticate with the "Guest" user and empty password. This is
one way to keep this convention. I can remove it if you'd like.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
