Hi Team, As a user I think re-implementing certain scripting languages in Java is a bad idea compared to having a robust API that any language current or future can interface with. I make a lot of use of ExecuteProcess and anecdotally haven't experienced a situation where externally running the script vs the internal ExecuteScript made a poomteenth of difference in load or timing. It may have been the case in the 1980's that forking a process was time consuming; we've moved on.
I think we have (almost) enough options now to expedite data transfer with an external system so it wouldn't even be necessary to bloat out data sizes 4x with encoded JSON via the REST API, just load flowfiles in via Kafka or whatever to and from the script. It is no longer your concern; NiFi exposed the ability to in a generic way and you're done. Stop worrying about things out of your control like the only available java version of ruby has a billion and 1 CVE's and is unmaintained, or Jython has a working 3.x version but it's been labelled "beta" for 15 years. You shouldn't have to care what a user uses to interface with the NiFi API, and certainly don't bloat the NiFi installer by including each and every option! Let someone else write a native API library in scripting language of choice to expose eg "from NiFi import FlowFileQueue" so an even more expansive library is available than what was exposed via ExecuteScript. This is what friends on GitHub are for 😁 Concentrate on making the NiFi API robust, complete and performant. (I know the REST API exists, to me the missing piece is being able to register a script in a generic fashion so that it integrates with the flow and hence understands which IDs it requires to call esp FlowFile Queues via the rest API without having to always trawl through eg /flow and dynamically generate it that way, which would be expensive imo on each schedule) Kind regards, On Thu, 16 Nov 2023, 08:07 Joe Witt, <joe.w...@gmail.com> wrote: > Team, > > Coming back to this as I've been reviewing all the great output of OWASP > that we're getting daily now (thanks David) as well as some findings from a > nice vendor tool by JFrog. And what is popping up left and right are > issues in terms of vulnerabilities tied to Jython-standalone. It is very > clear this component needs to go and is not actively maintained. It is > also as noted tied to Python 2 which is eol. I'll file a JIRA to deprecate > in 1.x and remove in 2.x. > > The discussion about Javascript/Ruby, etc.. I think the best course going > forward is a specific component appropriate to each desired mechanism is > introduced in 2.x as interest/effort dictates. The native Python support > will eliminate the need for Jython obviously so not worried about that > coming back. > > Thanks > > On Tue, Nov 7, 2023 at 5:17 AM Mike Thomsen <mikerthom...@gmail.com> > wrote: > > > Matt, > > > > I don't use JRuby, but the reason it raised my eyebrow a little was that > > AFAIK JRuby is the complete opposite of Jython in terms of enterprise > > readiness and parity with its C-based counterpart. > > > > Edward, > > > > The problem with Lua here is that the Java versions of Lua aren't being > > maintained. I think the most recent release on any branch of them was 3 > > years ago. They're basically dead in the water as projects. > > > > On Mon, Nov 6, 2023 at 2:57 PM Edward Armes <edward.ar...@gmail.com> > > wrote: > > > > > While I do like the ExecuteScript processors as they are great at > digging > > > you out of a hole. The performance of them isn't that great. > > > > > > That being said I would ve careful about dropping Lua support as there > > is a > > > growing list of software that supports end user/administrator > extensions > > > via Lua for those that don't want to have to re-compile software > > > themselves. On the other hand given that Jython doesn't yet have a > > Python 3 > > > implementation it could be argued dropping Jython support is a must > given > > > that the Python 2.x line is basically end of life. > > > > > > Now I wonder if its worth re-factoring the ExecuteScript processors to > be > > > per language implementations inheriting from a common base like a few > > > processors do already. > > > > > > Edward > > > > > > On Mon, 6 Nov 2023, 16:24 Matt Burgess, <mattyb...@apache.org> wrote: > > > > > > > I believe it is because in both ExecuteScript and ExecuteGroovyScript > > > > you can do "regular" groovy, but EGS has extra built-ins such as easy > > > > access to controller services, Groovy SQL stuff, etc, and we could > > > > keep building it out. But IMO we'd have to port the rest of the > > > > scripted components (ScriptedReader/Writer, etc.) over to the Groovy > > > > bundle and make sure there's a drop-in replacement in the Python > stuff > > > > before we'd want to deprecate the scripted bundle. > > > > > > > > On the JRuby front, is that something you use actively? This question > > > > is for you and the entire community of course. > > > > > > > > Regards, > > > > Matt > > > > > > > > On Mon, Nov 6, 2023 at 7:12 AM Mike Thomsen <mikerthom...@gmail.com> > > > > wrote: > > > > > > > > > > If we deprecate ExecuteScript, I think we need to have groovyx be > > ready > > > > to > > > > > function as a drop-in replacement if it's not there already. > > > > > > > > > > On Sun, Nov 5, 2023 at 9:21 PM Matt Burgess <mattyb...@apache.org> > > > > wrote: > > > > > > > > > > > IIRC the removal of these engines was mostly due to lack of use > or > > at > > > > > > least the perception thereof. If JRuby is being used by the > > community > > > > > > actively, I'm happy to revisit that discussion. Luaj's JSR-223 > > > > > > interface left something to be desired, but JRuby just needed a > > > system > > > > > > variable set or something like that. > > > > > > > > > > > > For the groovyx bundle, because it is Groovy-specific I tend to > > think > > > > > > we could make better use of that than ExecuteScript, especially > if > > we > > > > > > do get rid of all the engines. We have a Groovy-specific > > processor, a > > > > > > "real" Python SDK, and no more Nashorn. Perhaps we move all the > > > > > > scripted components to the Groovy bundle, although I believe some > > > > > > folks still make use of Jython for these. Of course if we > reinstate > > > > > > JRuby for ExecuteScript it's probably best to keep things the way > > > they > > > > > > are, or create a jruby bundle. The original scripting bundle was > > > > > > aiming to support several engines, but if it turns out only one > or > > > two > > > > > > will be useful, it may not be worth shoehorning all that JSR-223 > > > logic > > > > > > when engine-specific components could be simpler, more easily > > > > > > maintained, and allow for the idioms of the language to be better > > > used > > > > > > (as is done in the groovyx bundle). > > > > > > > > > > > > Just my two cents, looking forward to everyone's thoughts! > > > > > > > > > > > > - Matt > > > > > > > > > > > > On Sun, Nov 5, 2023 at 8:31 PM Mike Thomsen < > > mikerthom...@gmail.com> > > > > > > wrote: > > > > > > > > > > > > > > https://issues.apache.org/jira/browse/NIFI-11646 > > > > > > > > > > > > > > I get the removal of Lua, but not the removal of JRuby. It's a > > > clean > > > > > > > reimplementation of Ruby native to the JVM and AFAICT is pound > > for > > > > pound > > > > > > as > > > > > > > actively maintained as Groovy. > > > > > > > > > > > > > > Also, at this point, does it make sense to even keep the > groovyx > > > > bundle > > > > > > > rather than deprecate it for 2.X? > > > > > > > > > > > > > > > >
