Hi, it would be very useful to have such a processor.
I like the ideas for 1 & 2 and I think we can do it this way. Actually I think that the definition of the output scheme could be quite tricky. Because users can do anything with the event. How about we provide an option property? Users can select whether they want to define the whole event or just append properties. For the configurations of the properties we could do something similar then we have in the PLC4xS7Adapter. A collection property containing the fields of the output properties. Therefore, we have to decide what information we have to provide. I think if we would support all options it becomes very cluttered for the user. What do you think? Philipp > On 14. May 2020, at 06:55, Grainier Perera <[email protected]> wrote: > > Hi Dominik, > > I agree. We have to do some enhancements make this processor user friendly. > Like you said it's nice to have; > 1. A new static property that supports code highlighting & syntax > checking. We can simply do such client-side validations using highlight.js > (BSD) [1], jshint (MIT) [2], etc... > 2. A mechanism to show the user with available fields they can use to > write code (similar to what you've done with usable templates in > EmailPublisher using HtmlInputParameter). > 3. A mechanism to map output schema. > > I think 1 & 2 go together and for the 3rd requirement given that we are > targetting a more technical user group for this processor, we can let them > define the output. Otherwise, it would be quite difficult to derive return > type from the JS. Moreover, I'm planning to use Java's builtin ScriptEngine > to eval. Of course, there'll be limitations. But, we can always improve > them :) > > [1] https://highlightjs.org/ > [2] https://jshint.com/ > > Regards, > Grainier. > > On Thu, 14 May 2020 at 00:53, Dominik Riemer <[email protected]> wrote: > >> Hi Grainier, >> >> in my opinion, this would definitely be a very interesting addition! We >> already briefly discussed such a feature in the past, before we went to the >> ASF. I’d say that a JS evaluator would be definitely useful to a slightly >> more technical target user group, but would provide great flexibility for, >> e.g., data harmonization tasks. So let’s discuss what is needed to >> implement this! I’d guess that we (maybe) need to add some enhancements to >> the core to make this processor good from a usability perspective. >> >> From a conceptual point of view, I guess the JS processor would have no >> specific input requirements and a single static property that allows users >> to enter the code (maybe it would be cool to add a new static property here >> that supports code highlighting or even syntax checking). The only open >> issue I see is the output schema – we would somehow need to extract the >> output from the JS function. This could probably partially be done using >> the CustomTransformOutput (see >> http://streampipes.apache.org/docs/docs/dev-guide-output-strategies/), >> but we would somehow need to derive type information from the JS function >> or introduce a feature to let users define/refine the output manually >> (e.g., to add semantic metadata). >> >> What do you think would be best? And would you evaluate the JS code in >> Java or somehow else? >> >> Also, as I’ve just started to improve the SDK (e.g., supporting optional >> static properties and an easier way to define and extract model >> parameters), we can collect your requirements for the JS processor and >> extend the SDK if needed, just say what you would like to have 😉 >> >> Dominik >> >> >> From: Grainier Perera <[email protected]> >> Sent: Wednesday, May 13, 2020 2:41 PM >> To: [email protected] >> Subject: Data processor to evaluate JavaScript >> >> Hi all, >> >> I'm planning to implement a JavaScript eval data processor. As the name >> suggests, users will be able to write some JavaScript code which takes in >> an event (as a map), do some processing on the event, and return a new map >> which then gets converted to an sp-event. >> >> example js: >> function process(event) { >> // do processing here. >> // return processed event. >> return {id: http://event.id, tempInCelsius: (event.tempInKelvin - >> 273.15)}; >> }; >> >> Will this be a good addition to pipeline-elements? What do you think? >> >> Regards, >> Grainier. >> >>
