[ https://issues.apache.org/jira/browse/NIFI-11240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17695866#comment-17695866 ]
Otto Fowler commented on NIFI-11240: ------------------------------------ We should include a cookie cutter template creating projects. > Introduce Python API for building Processors > -------------------------------------------- > > Key: NIFI-11240 > URL: https://issues.apache.org/jira/browse/NIFI-11240 > Project: Apache NiFi > Issue Type: Epic > Components: Core Framework, Documentation & Website, Extensions > Reporter: Mark Payne > Assignee: Mark Payne > Priority: Major > Fix For: 2.0.0 > > > The scripting processors are very common for data transformation in NiFi. In > particular, the Jython based scripts are quite heavily used. However, Jython > is run on the JVM and does not support CPython libraries. As a result, it's > syntax compatible but doesn't make use of the wealth of Python libraries. And > the wealth of Python libraries are what make Python popular to begin with. > Additionally, use of many script-based processors hurts the UX. They are > cumbersome to configure, with script files and/or script bodies. They result > in a dataflow that's difficult to understand because instead of nicely named > processors like CompressContent the type and default name are > "ExecuteScript." They're also difficult to share. > I have been playing with Py4J for introduce a true Python-based API for > developing Processors. This will introduce new APIs, new framework changes, > and documentation. And this will likely take a while to stabilize. However, > the sooner that we are able to land it into the hands of users, the better. > Therefore, I pose that we introduce it in multiple milestones. We can create > sub-tickets for different milestones, but in general it should follow: > Milestone 1: Initial implementation. Provides the capability and an API for > building processors. Includes sample code and some documentation. Includes > tests to ensure proper operation. Should not be used in production. API will > not be stable and may change frequently. Performance may be subpar. Get into > the hands of developers to begin exploring and providing feedback / > submitting PRs. > Milestone 2: Bug fixes. API refinement. Improve performance. > Milestone 3: Additional bug fixes and API refinement. API should become more > stable. > Milestone 4: Additional bug fixes. API becomes stable. Documentation is clear > and sufficient. Recommend production use. > > -- This message was sent by Atlassian Jira (v8.20.10#820010)