This is an automated email from the ASF dual-hosted git repository.
lordgamez pushed a change to branch MINIFICPP-2852
in repository https://gitbox.apache.org/repos/asf/nifi-minifi-cpp.git
omit c04e43fbd MINIFICPP-2852 Implement LMDB based flow file repository
omit 3653c3b22 Update LMDB and add conan recipe
omit f0f3414be Review update
omit e0c2d8ee6 Review update
omit 3195845b2 Fix LMDB build on Windows with WIN32_LEAN_AND_MEAN defined
omit 46cce2068 Fix lmdb lifetime issues
omit 524611258 MINIFICPP-2849 Implement LMDB based content repository
add d89fea9eb MINIFICPP-2881 Set MINIFI_API_VERSION to 2 (#2244)
add 894ba4d1e MINIFICPP-2861 Add Conan support for docker and clang CI
builds (#2216)
add a6d147bbb MINIFICPP-2877 Clean up memcheck (#2243)
add dfba4edfb MINIFICPP-2866 Add Conan support for SOCI and iODBC
libraries (#2218)
add 48eedde6d MINIFICPP-2878 PRs from apache should not double run CI
(#2248)
add a7419e286 MINIFICPP-2870 Lint/Check python code with ruff (#2237)
add 8be29c104 MINIFICPP-2885 Allow explictily unsetting properties with
default value (#2241)
add 148097ab3 MINIFICPP-2888 Fixing build failure in rocky (#2250)
add 4ee6b4fe8 MINIFICPP-2887 Memory leak in
minifi_process_session_transfer/minifi_process_session_remove (#2249)
add 241aa38b1 MINIFICPP-2869 Improve rust api property validation and
property getters (#2220)
add 776ba835e MINIFICPP-2883 Support host file copies in docker behave
(#2238)
add 89be46fe9 MINIFICPP-2849 Implement LMDB based content repository
(#2201)
add c392cf4f5 MINIFICPP-2871 Create conanfile for openssl-fips target
(#2223)
add 8e3172929 MINIFICPP-2873 Add option to run create release artifacts
job with Conan (#2242)
add 1280948e5 MINIFICPP-2891 Fix rocky9_from_rocky_package docker (#2252)
add 7f19f9f17 MINIFICPP-2893 Only upload conan packages from apache (CI)
(#2253)
add 11f92483b MINIFICPP-2896 Fix cpack for openssl-fips in debug mode
using conan (#2255)
add a06d0d872 MINIFICPP-2897 Fix LMDB patch command on Windows (#2256)
add 36f1a2d3f MINIFICPP-2899 procfs: Fix CpuStat and CPUStatTests
inaccuracy (#2260)
add e5d8f57ee MINIFICPP-2852 Implement LMDB based flow file repository
add a0d3d5c5e Review update
add 04c1a11f1 Mark LmdbFlowFileRepository as experimental in docs
add 2492811cf Review update
add 78a412fd2 Fix after rebase
add d5f711e81 Add LMDB flowfile repository to README and change default db
size to 1GB
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (c04e43fbd)
\
N -- N -- N refs/heads/MINIFICPP-2852 (d5f711e81)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.dockerignore | 1 +
.github/conan/compatibility.py | 5 +-
.../github_scripts/github_actions_cache_cleanup.py | 52 ++-
.../github_actions_cache_cleanup_tests.py | 52 +--
.../ubuntu_22_04_clang_arm_manifest.json | 68 +++-
.github/workflows/ci.yml | 70 +++-
.github/workflows/create-release-artifacts.yml | 38 +-
.github/workflows/memcheck_ci.yml | 20 ++
CMakeLists.txt | 4 +-
CONFIGURE.md | 13 +-
CONTRIBUTING.md | 19 +-
.../minifi_behave/containers/container_linux.py | 194 ++++++----
.../minifi_behave/containers/container_protocol.py | 33 +-
.../minifi_behave/containers/container_windows.py | 104 +++---
.../src/minifi_behave/containers/directory.py | 1 +
.../containers/docker_image_builder.py | 45 ++-
.../src/minifi_behave/containers/file.py | 1 +
.../src/minifi_behave/containers/host_file.py | 1 +
.../containers/http_proxy_container.py | 58 ++-
.../minifi_behave/containers/minifi_controller.py | 33 +-
.../containers/minifi_linux_container.py | 109 ++++--
.../minifi_behave/containers/minifi_protocol.py | 15 +-
.../containers/minifi_win_container.py | 20 +-
.../src/minifi_behave/containers/nifi_container.py | 118 +++---
behave_framework/src/minifi_behave/core/helpers.py | 48 ++-
behave_framework/src/minifi_behave/core/hooks.py | 86 ++++-
.../src/minifi_behave/core/minifi_test_context.py | 27 +-
.../src/minifi_behave/core/ssl_utils.py | 17 +-
.../src/minifi_behave/minifi/controller_service.py | 7 +-
.../src/minifi_behave/minifi/flow_definition.py | 19 +-
.../src/minifi_behave/minifi/funnel.py | 5 +-
.../src/minifi_behave/minifi/input_port.py | 8 +-
.../minifi_behave/minifi/minifi_flow_definition.py | 48 ++-
.../minifi_behave/minifi/nifi_flow_definition.py | 163 ++++-----
.../src/minifi_behave/minifi/output_port.py | 8 +-
.../src/minifi_behave/minifi/parameter.py | 9 +-
.../src/minifi_behave/minifi/parameter_context.py | 6 +-
.../src/minifi_behave/minifi/processor.py | 28 +-
.../src/minifi_behave/minifi/remote_port.py | 9 +-
.../minifi_behave/minifi/remote_process_group.py | 14 +-
.../src/minifi_behave/steps/checking_steps.py | 371 +++++++++++++------
.../src/minifi_behave/steps/configuration_steps.py | 13 +-
.../src/minifi_behave/steps/core_steps.py | 103 ++++--
.../src/minifi_behave/steps/flow_building_steps.py | 398 ++++++++++++++++-----
bootstrap/cli.py | 107 ++++--
bootstrap/cmake_parser.py | 26 +-
bootstrap/main.py | 78 ++--
bootstrap/minifi_option.py | 79 +++-
bootstrap/package_manager.py | 216 ++++++-----
bootstrap/system_dependency.py | 27 +-
cmake/BundledIodbc.cmake | 56 +++
cmake/DockerConfig.cmake | 30 +-
cmake/GenericPython.cmake | 2 +-
cmake/{GetAsio.cmake => GetIODBC.cmake} | 12 +-
cmake/GetOpenSSL.cmake | 53 ++-
cmake/{GetSpdlog.cmake => GetSOCI.cmake} | 24 +-
cmake/LMDB.cmake | 2 +-
cmake/MiNiFiOptions.cmake | 13 +-
cmake/PahoMqttC.cmake | 2 +-
cmake/Soci.cmake | 47 +++
cmake/VerifyPackageWithDocker.cmake | 12 +-
cmake/valgrind.supp | 121 +++++++
conanfile.py | 199 +++++++++--
.../include/core/ConfigurableComponentImpl.h | 1 +
.../src/core/ConfigurableComponentImpl.cpp | 10 +
docker/DockerBuild.sh | 18 +-
docker/Dockerfile | 28 +-
docker/rockylinux/Dockerfile | 27 +-
.../config.yml => docker/settings_user.yml | 6 +-
.../scripts/python/reverse_flow_file_content.py | 6 +-
.../containers/kinesis_server_container.py | 21 +-
.../features/containers/s3_server_container.py | 75 +++-
extensions/aws/tests/features/environment.py | 6 +-
extensions/aws/tests/features/steps/steps.py | 57 +--
.../features/containers/azure_server_container.py | 108 ++++--
extensions/azure/tests/features/environment.py | 6 +-
extensions/azure/tests/features/steps/steps.py | 31 +-
extensions/civetweb/tests/features/environment.py | 3 +-
extensions/civetweb/tests/features/steps/steps.py | 10 +-
.../containers/couchbase_server_container.py | 145 ++++++--
extensions/couchbase/tests/features/environment.py | 13 +-
extensions/couchbase/tests/features/steps/steps.py | 33 +-
.../features/containers/elastic_base_container.py | 39 +-
.../features/containers/elasticsearch_container.py | 96 +++--
.../features/containers/opensearch_container.py | 55 ++-
.../elasticsearch/tests/features/environment.py | 6 +-
.../elasticsearch/tests/features/steps/steps.py | 62 ++--
.../containers/fake_gcs_server_container.py | 20 +-
extensions/gcp/tests/features/environment.py | 6 +-
extensions/gcp/tests/features/steps/steps.py | 17 +-
.../features/containers/grafana_loki_container.py | 59 ++-
.../features/containers/reverse_proxy_container.py | 11 +-
.../grafana-loki/tests/features/environment.py | 13 +-
.../resources/check_log_lines_on_grafana.py | 14 +-
.../grafana-loki/tests/features/steps/steps.py | 49 ++-
.../features/containers/kafka_server_container.py | 127 +++++--
extensions/kafka/tests/features/environment.py | 11 +-
extensions/kafka/tests/features/steps/steps.py | 73 ++--
.../kubernetes/tests/features/environment.py | 11 +-
.../tests/features/steps/kubernetes_proxy.py | 120 ++++---
.../steps/minifi_as_pod_in_kubernetes_cluster.py | 4 +-
.../kubernetes/tests/features/steps/steps.py | 62 +++-
extensions/llamacpp/tests/CMakeLists.txt | 1 +
extensions/llamacpp/tests/features/environment.py | 18 +-
extensions/llamacpp/tests/features/steps/steps.py | 10 +-
extensions/lmdb/CMakeLists.txt | 2 +-
extensions/lmdb/LmdbContentRepository.cpp | 8 +-
extensions/lmdb/LmdbFlowFileRepository.cpp | 53 +--
extensions/lmdb/LmdbFlowFileRepository.h | 2 +-
extensions/lmdb/LmdbStream.cpp | 11 +-
extensions/lmdb/LmdbStream.h | 2 +-
extensions/lmdb/LmdbWrapper.cpp | 4 +-
.../lmdb/tests/LmdbContentRepositoryTests.cpp | 7 +-
.../lmdb/tests/LmdbFlowFileRepositoryTests.cpp | 4 +-
extensions/lmdb/tests/LmdbStreamTests.cpp | 16 +
extensions/lua/tests/features/environment.py | 5 +-
extensions/lua/tests/features/steps/steps.py | 10 +-
.../features/containers/mqtt_broker_container.py | 49 ++-
extensions/mqtt/tests/features/environment.py | 11 +-
.../features/resources/publish_mqtt_message.py | 3 +-
extensions/mqtt/tests/features/steps/steps.py | 48 +--
extensions/opc/include/opc.h | 2 +-
extensions/opc/src/opc.cpp | 39 +-
extensions/opc/tests/PutOpcProcessorTests.cpp | 5 +
.../features/containers/opc_ua_server_container.py | 14 +-
extensions/opc/tests/features/environment.py | 9 +-
extensions/opc/tests/features/steps/steps.py | 64 ++--
extensions/procfs/CpuStat.h | 2 +-
extensions/procfs/tests/CPUStatTests.cpp | 22 +-
.../features/containers/prometheus_container.py | 114 ++++--
.../prometheus/tests/features/environment.py | 11 +-
.../tests/features/resources/prometheus_checker.py | 220 +++++++++---
.../prometheus/tests/features/steps/steps.py | 74 ++--
extensions/python/CMakeLists.txt | 10 +-
extensions/python/PYTHON.md | 24 +-
.../pythonprocessor-examples/AddPythonAttribute.py | 1 -
.../pythonprocessor-examples/CountingProcessor.py | 3 +-
.../GaussianDistributionWithNumpy.py | 2 +-
.../pythonprocessor-examples/MoveContentToJson.py | 5 +-
.../pythonprocessor-examples/RemoveFlowFile.py | 1 -
.../pythonprocessor-examples/SentimentAnalysis.py | 12 +-
.../google/SentimentAnalyzer.py | 28 +-
.../pythonprocessor-examples/h2o/ConvertDsToCsv.py | 51 ++-
.../h2o/h2o3/mojo/ExecuteH2oMojoScoring.py | 158 ++++----
.../utils/dependency_installer.py | 32 +-
.../pythonprocessors/nifiapi/componentstate.py | 10 +-
.../python/pythonprocessors/nifiapi/decorators.py | 1 +
.../pythonprocessors/nifiapi/documentation.py | 60 +++-
.../pythonprocessors/nifiapi/flowfilesource.py | 8 +-
.../pythonprocessors/nifiapi/flowfiletransform.py | 12 +-
.../pythonprocessors/nifiapi/processorbase.py | 49 ++-
.../python/pythonprocessors/nifiapi/properties.py | 106 ++++--
.../pythonprocessors/nifiapi/recordtransform.py | 62 +++-
.../pythonprocessors/nifiapi/relationship.py | 1 +
extensions/python/tests/features/environment.py | 101 +++---
.../tests/features/resources/CreateFlowFile.py | 30 +-
.../tests/features/resources/CreateNothing.py | 13 +-
.../resources/EvaluateExpressionLanguageChecker.py | 21 +-
.../ExpressionLanguagePropertyWithValidator.py | 25 +-
.../features/resources/LogDynamicProperties.py | 9 +-
.../resources/NifiStyleLogDynamicProperties.py | 27 +-
.../resources/ProcessContextInterfaceChecker.py | 30 +-
.../resources/RelativeImporterProcessor.py | 5 +-
.../tests/features/resources/RotatingForwarder.py | 1 +
.../tests/features/resources/SetRecordField.py | 27 +-
.../resources/SingleThreadedSleepForever.py | 12 +-
.../tests/features/resources/SleepForever.py | 10 +-
.../resources/SpecialPropertyTypeChecker.py | 15 +-
.../tests/features/resources/TestStateManager.py | 22 +-
.../tests/features/resources/multiplierutils.py | 1 +
.../tests/features/resources/subtractutils.py | 1 +
extensions/python/tests/features/steps/steps.py | 121 ++++---
.../tests/test_python_scripts/foo_bar_processor.py | 2 +-
.../non_transferring_processor.py | 4 +-
...passthrough_processor_transfering_to_failure.py | 4 +-
...passthrough_processor_transfering_to_success.py | 4 +-
.../test_python_scripts/stateful_processor.py | 9 +-
.../tests/features/containers/splunk_container.py | 149 +++++---
extensions/splunk/tests/features/environment.py | 6 +-
extensions/splunk/tests/features/steps/steps.py | 28 +-
extensions/sql/CMakeLists.txt | 152 +-------
.../containers/postgress_server_container.py | 27 +-
extensions/sql/tests/features/environment.py | 36 +-
extensions/sql/tests/features/steps/steps.py | 27 +-
.../stable-api-testing/ExtensionInitializer.cpp | 2 +
extensions/stable-api-testing/PropertyTester.h | 97 +++++
extensions/stable-api-testing/tests/CMakeLists.txt | 1 +
.../tests/PropertyTesterTests.cpp | 221 ++++++++++++
.../features/containers/diag_slave_container.py | 17 +-
.../tests/features/containers/syslog_container.py | 2 +-
.../features/containers/tcp_client_container.py | 9 +-
.../tests/features/environment.py | 15 +-
.../features/steps/minifi_c2_server_container.py | 98 +++--
.../features/steps/minifi_controller_steps.py | 43 ++-
.../tests/features/steps/steps.py | 82 +++--
.../src/core/flow/StructuredConfiguration.cpp | 21 +-
libminifi/src/minifi-api.cpp | 8 +-
.../resources/TestC2DescribeCoreComponentState.yml | 4 +
libminifi/test/unit/ProcessSessionTests.cpp | 30 ++
minifi-api/include/minifi-api.h | 2 +-
.../minifi-cpp/core/ConfigurableComponent.h | 1 +
minifi-api/include/minifi-cpp/core/Property.h | 1 +
.../extensions/minifi_rs_playground/Cargo.toml | 1 +
.../features/controller_apis.feature | 2 +-
.../minifi_rs_playground/features/environment.py | 61 +---
.../minifi_rs_playground/features/steps/steps.py | 21 +-
.../minifi_rs_playground/minifi_rs_playground.md | 30 +-
.../controller_services/dog_controller_service.rs | 46 +--
.../controller_services/duck_controller_service.rs | 4 +-
.../dummy_controller_service.rs | 4 +-
.../lorem_ipsum_controller_service.rs | 23 +-
.../asciify_german/processor_definition.rs | 5 +-
.../src/processors/count_actual_logging.rs | 6 +-
.../src/processors/duplicate_text.rs | 4 +-
.../src/processors/generate_flow_file.rs | 41 ++-
.../generate_flow_file/processor_definition.rs | 7 +-
.../processors/generate_flow_file/properties.rs | 82 ++---
.../src/processors/generate_flow_file/tests.rs | 30 +-
.../src/processors/get_file.rs | 35 +-
.../processors/get_file/processor_definition.rs | 5 +-
.../src/processors/get_file/properties.rs | 158 +++-----
.../src/processors/get_file/tests.rs | 18 +-
.../src/processors/kamikaze_processor.rs | 27 +-
.../kamikaze_processor/processor_definition.rs | 7 +-
.../processors/kamikaze_processor/properties.rs | 55 ++-
.../src/processors/kamikaze_processor/tests.rs | 18 +-
.../src/processors/log_attribute.rs | 57 ++-
.../log_attribute/processor_definition.rs | 5 +-
.../src/processors/log_attribute/properties.rs | 127 +++----
.../src/processors/lorem_ipsum_cs_user.rs | 27 +-
.../lorem_ipsum_cs_user/processor_definition.rs | 6 +-
.../processors/lorem_ipsum_cs_user/properties.rs | 39 +-
.../src/processors/lorem_ipsum_cs_user/tests.rs | 23 +-
.../src/processors/put_file.rs | 88 +----
.../processors/put_file/processor_definition.rs | 38 +-
.../src/processors/put_file/properties.rs | 67 ++--
.../processors/put_file/unix_only_properties.rs | 33 +-
.../src/processors/put_file/unix_permissions.rs | 79 ++++
.../src/processors/zoo_processor.rs | 42 +--
minifi_rust/minifi_native/Cargo.toml | 3 +
minifi_rust/minifi_native/src/api.rs | 4 +-
minifi_rust/minifi_native/src/api/attribute.rs | 4 +
.../src/api/component_definition_traits.rs | 6 +-
minifi_rust/minifi_native/src/api/errors.rs | 27 +-
minifi_rust/minifi_native/src/api/logger.rs | 18 +-
.../minifi_native/src/api/process_context.rs | 127 ++-----
.../api/processor_wrappers/flow_file_transform.rs | 54 +++
.../utils/context_session_flowfile_bundle.rs | 23 +-
minifi_rust/minifi_native/src/api/property.rs | 305 +++++++++++++---
minifi_rust/minifi_native/src/api/relationship.rs | 8 +
.../src/c_ffi/c_ffi_controller_service_context.rs | 14 +-
.../c_ffi/c_ffi_controller_service_definition.rs | 6 +-
.../src/c_ffi/c_ffi_process_context.rs | 54 +--
.../src/c_ffi/c_ffi_processor_definition.rs | 6 +-
.../minifi_native/src/c_ffi/c_ffi_property.rs | 41 ++-
minifi_rust/minifi_native/src/lib.rs | 11 +-
.../src/mock/mock_controller_service_context.rs | 6 +-
.../minifi_native/src/mock/mock_process_context.rs | 46 ++-
minifi_rust/minifi_native_macros/src/lib.rs | 37 ++
minifi_rust/minifi_native_sys/build.rs | 3 +-
minifi_rust/minifi_rs_behave/Dockerfile.alpine | 2 +-
minifi_rust/minifi_rs_behave/Dockerfile.debian | 2 +-
minifi_rust/minifi_rs_behave/src/main.rs | 1 +
ruff.toml | 17 +
run_flake8.sh | 22 --
thirdparty/google-styleguide/cpplint.py | 3 +
.../{google-cloud-cpp => iodbc}/all/conandata.yml | 12 +-
thirdparty/{ossp-uuid => iodbc}/all/conanfile.py | 61 ++--
.../GCC-15-needs-typedef-SQLRETURN-HPROC.patch | 0
thirdparty/{magic-enum => iodbc}/config.yml | 2 +-
.../{jolt-tests => openssl-fips}/all/conandata.yml | 6 +-
thirdparty/openssl-fips/all/conanfile.py | 75 ++++
thirdparty/{magic-enum => openssl-fips}/config.yml | 2 +-
thirdparty/paho-mqtt/all/conandata.yml | 7 +
thirdparty/paho-mqtt/all/conanfile.py | 171 +++++++++
.../{ => all/patches}/cmake-openssl.patch | 0
thirdparty/{grpc => paho-mqtt}/config.yml | 2 +-
thirdparty/rocksdb/all/conanfile.py | 5 +-
.../{google-cloud-cpp => soci}/all/conandata.yml | 15 +-
thirdparty/soci/all/conanfile.py | 180 ++++++++++
.../all/patches/disable-sqlwchar-support.patch | 31 ++
.../odbc-get-parameter-name-bounds-safe.patch | 17 +
thirdparty/{magic-enum => soci}/config.yml | 2 +-
thirdparty/soci/sqlite3-path.patch | 36 --
284 files changed, 7246 insertions(+), 3624 deletions(-)
create mode 100644 cmake/BundledIodbc.cmake
copy cmake/{GetAsio.cmake => GetIODBC.cmake} (76%)
copy cmake/{GetSpdlog.cmake => GetSOCI.cmake} (66%)
create mode 100644 cmake/Soci.cmake
create mode 100644 cmake/valgrind.supp
copy thirdparty/couchbase/config.yml => docker/settings_user.yml (94%)
create mode 100644 extensions/stable-api-testing/PropertyTester.h
create mode 100644 extensions/stable-api-testing/tests/PropertyTesterTests.cpp
create mode 100644
minifi_rust/extensions/minifi_rs_playground/src/processors/put_file/unix_permissions.rs
create mode 100644 ruff.toml
delete mode 100755 run_flake8.sh
copy thirdparty/{google-cloud-cpp => iodbc}/all/conandata.yml (72%)
copy thirdparty/{ossp-uuid => iodbc}/all/conanfile.py (56%)
rename thirdparty/{iODBC =>
iodbc/all/patches}/GCC-15-needs-typedef-SQLRETURN-HPROC.patch (100%)
copy thirdparty/{magic-enum => iodbc}/config.yml (98%)
copy thirdparty/{jolt-tests => openssl-fips}/all/conandata.yml (80%)
create mode 100644 thirdparty/openssl-fips/all/conanfile.py
copy thirdparty/{magic-enum => openssl-fips}/config.yml (98%)
create mode 100644 thirdparty/paho-mqtt/all/conandata.yml
create mode 100644 thirdparty/paho-mqtt/all/conanfile.py
rename thirdparty/paho-mqtt/{ => all/patches}/cmake-openssl.patch (100%)
copy thirdparty/{grpc => paho-mqtt}/config.yml (98%)
copy thirdparty/{google-cloud-cpp => soci}/all/conandata.yml (60%)
create mode 100644 thirdparty/soci/all/conanfile.py
create mode 100644 thirdparty/soci/all/patches/disable-sqlwchar-support.patch
create mode 100644
thirdparty/soci/all/patches/odbc-get-parameter-name-bounds-safe.patch
copy thirdparty/{magic-enum => soci}/config.yml (98%)
delete mode 100644 thirdparty/soci/sqlite3-path.patch
