Team, As we get closer to a full release of Apache NiFi 2.0.0, we have an important opportunity to set the direction for future development of Python-based Processors.
The introduction of native Python support presents a number of new integration opportunities, and it also raises questions about maintenance and versioning. As the journey to NiFi 2.0.0 has shown, it requires significant effort to coordinate maintenance and modernization across hundreds of project modules. Although the internal project structure has maintained helpful separation of API and implementation, the current release strategy highlights the challenges of verifying multiple layers of changes. Introducing a new programming language provides greater possibilities, but also makes it more difficult to maintain a single repository with a single versioning strategy. I propose creating a new Git repository named nifi-python-extensions, which would have its own versioning and release process. This would contain the extensions now under the module of the same name in the NiFi repository. Having a separate repository and release process for Python-based extensions has the following advantages: 1. Clean separation between NiFi APIs for Python and Python-based Processors 2. Independent release cycles for Python-based Processors 3. Focused release verification and testing on Python-based modules These advantages can also enable more rapid iteration on Python-based Processors, without impacting the NiFi Framework or requiring new releases at that level. Although this would require a separate installation process for Python-based components, this could follow an approach similar to what is already required for optional Java-based components. Thanks in advance for your consideration. Regards, David Handermann Apache NiFi PMC Member
