lordgamez commented on code in PR #1712: URL: https://github.com/apache/nifi-minifi-cpp/pull/1712#discussion_r1500708936
########## extensions/python/ExecutePythonProcessor.cpp: ########## @@ -146,11 +151,38 @@ std::unique_ptr<PythonScriptEngine> ExecutePythonProcessor::createScriptEngine() auto engine = std::make_unique<PythonScriptEngine>(); python_logger_ = core::logging::LoggerFactory<ExecutePythonProcessor>::getAliasedLogger(getName()); - engine->initialize(Success, Failure, python_logger_); + engine->initialize(Success, Failure, Original, python_logger_); return engine; } +core::Property* ExecutePythonProcessor::findProperty(const std::string& name) const { + if (auto prop_ptr = core::ConfigurableComponent::findProperty(name)) { + return prop_ptr; + } + + auto it = ranges::find_if(python_properties_, [&name](const auto& item){ + return item.getName() == name; + }); + if (it != python_properties_.end()) { + return const_cast<core::Property*>(&*it); + } + + return nullptr; +} + +std::map<std::string, core::Property> ExecutePythonProcessor::getProperties() const { + auto result = ConfigurableComponent::getProperties(); + + std::lock_guard<std::mutex> lock(configuration_mutex_); Review Comment: You are right, it should be a separate mutex for the `python_properties_`, it didn't really make sense this way. Updated in 6593d9b63514dd2a3b2060b747680f37691abecd ########## extensions/python/PYTHON.md: ########## @@ -106,20 +127,39 @@ class VaderSentiment(object): To enable python Processor capabilities, the following options need to be provided in minifi.properties. The directory specified can contain processors. Note that the processor name will be the reference in your flow. Directories are treated like package names. Therefore if the nifi.python.processor.dir is /tmp/ and you have a subdirectory named packagedir with the file name file.py, it will -produce a processor with the name org.apache.nifi.minifi.processors.packagedir.file. Note that each subdirectory will append a package -to the reference class name. +produce a processor with the name org.apache.nifi.minifi.processors.packagedir.file. Note that each subdirectory will append a package +to the reference class name. in minifi.properties #directory where processors exist nifi.python.processor.dir=XXXX - - + + ## Processors The python directory (extensions/pythonprocessors) contains implementations that will be available for flows if the required dependencies exist. - -## Sentiment Analysis + +### Sentiment Analysis The SentimentAnalysis processor will perform a Vader Sentiment Analysis. This requires that you install nltk and VaderSentiment pip install nltk pip install VaderSentiment + +## Using NiFi Python Processors + +MiNiFi C++ supports the use of NiFi Python processors, that are inherited from the FlowFileTransform base class. To use these processors, you must copy the Python processor module under the nifi_python_processors directory located under the python directory (by default it can be found at ${minifi_root}/minifi-python/nifi_python_processors). To see how to write NiFi Python processors, please refer to the Python Developer Guide under the [Apache NiFi documentation](https://nifi.apache.org/documentation/v2/). + +In the flow configuration these Python processors can be referenced by their fully qualified class name, which looks like this: org.apache.nifi.minifi.processors.nifi_python_processors.<package_name>.<processor_name>. For example, the fully qualified class name of the PromptChatGPT processor implemented in the file nifi_python_processors/PromptChatGPT.py is org.apache.nifi.minifi.processors.nifi_python_processors.PromptChatGPT. If a processor is copied under a subdirectory, because it is part of a python submodule, the submodule name will be appended to the fully qualified class name. For example, if the QueryPinecone processor is implemented in the QueryPinecone.py file that is copied to nifi_python_processors/vectorstores/QueryPinecone.py, the fully qualified class name will be org.apache.nifi.minifi.processors.nifi_python_processors.vectorstores.QueryPinecone in the configuration file. + +**NOTE:** The name of the NiFi Python processor file should match the class name in the file, otherwise the processor will not be found. + +Due to some differences between the NiFi and MiNiFi C++ processors and implementation, there are some limitations using the NiFi Python processors: +- Record based processors are not yet supported in MiNiFi C++, so the NiFi Python processors inherited from RecordTransform are not supported. +- Virtualenv support is not yet available in MiNiFi C++, so all required packaged must be installed on the system. Review Comment: Updated in 6593d9b63514dd2a3b2060b747680f37691abecd ########## extensions/python/PYTHON.md: ########## @@ -106,20 +127,39 @@ class VaderSentiment(object): To enable python Processor capabilities, the following options need to be provided in minifi.properties. The directory specified can contain processors. Note that the processor name will be the reference in your flow. Directories are treated like package names. Therefore if the nifi.python.processor.dir is /tmp/ and you have a subdirectory named packagedir with the file name file.py, it will -produce a processor with the name org.apache.nifi.minifi.processors.packagedir.file. Note that each subdirectory will append a package -to the reference class name. +produce a processor with the name org.apache.nifi.minifi.processors.packagedir.file. Note that each subdirectory will append a package +to the reference class name. in minifi.properties #directory where processors exist nifi.python.processor.dir=XXXX - - + + ## Processors The python directory (extensions/pythonprocessors) contains implementations that will be available for flows if the required dependencies exist. - -## Sentiment Analysis + +### Sentiment Analysis The SentimentAnalysis processor will perform a Vader Sentiment Analysis. This requires that you install nltk and VaderSentiment pip install nltk pip install VaderSentiment + +## Using NiFi Python Processors + +MiNiFi C++ supports the use of NiFi Python processors, that are inherited from the FlowFileTransform base class. To use these processors, you must copy the Python processor module under the nifi_python_processors directory located under the python directory (by default it can be found at ${minifi_root}/minifi-python/nifi_python_processors). To see how to write NiFi Python processors, please refer to the Python Developer Guide under the [Apache NiFi documentation](https://nifi.apache.org/documentation/v2/). Review Comment: Updated in 6593d9b63514dd2a3b2060b747680f37691abecd -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org