Hi David, While I agree with your summary, I have a concern here which is about user awareness of this feature. We've seen in the past: as soon as we don't include NARs in the convenience binary, we see that users have no clue about those NARs (and some are super powerful/useful). I agree that python is a bit different because it requires a user action to enable it in the first place but I still think that including the components in the convenience binary of Apache NiFi would drive user awareness, adoption, etc.
If we have a separated repo with its own release cycle can we imagine a process where, when releasing Apache NiFi, it'd include whatever is the latest version of the Python repo? Or something along those lines? Pierre Le ven. 26 janv. 2024 à 08:01, David Handermann <[email protected]> a écrit : > Team, > > As we get closer to a full release of Apache NiFi 2.0.0, we have an > important opportunity to set the direction for future development of > Python-based Processors. > > The introduction of native Python support presents a number of new > integration opportunities, and it also raises questions about > maintenance and versioning. As the journey to NiFi 2.0.0 has shown, it > requires significant effort to coordinate maintenance and > modernization across hundreds of project modules. Although the > internal project structure has maintained helpful separation of API > and implementation, the current release strategy highlights the > challenges of verifying multiple layers of changes. Introducing a new > programming language provides greater possibilities, but also makes it > more difficult to maintain a single repository with a single > versioning strategy. > > I propose creating a new Git repository named nifi-python-extensions, > which would have its own versioning and release process. This would > contain the extensions now under the module of the same name in the > NiFi repository. Having a separate repository and release process for > Python-based extensions has the following advantages: > > 1. Clean separation between NiFi APIs for Python and Python-based > Processors > 2. Independent release cycles for Python-based Processors > 3. Focused release verification and testing on Python-based modules > > These advantages can also enable more rapid iteration on Python-based > Processors, without impacting the NiFi Framework or requiring new > releases at that level. Although this would require a separate > installation process for Python-based components, this could follow an > approach similar to what is already required for optional Java-based > components. > > Thanks in advance for your consideration. > > Regards, > David Handermann > Apache NiFi PMC Member >
