Hi Dominik, unfortunately this was not the problem, but this could become a problem in the future ;) The problem was the java version. So far we used Java 8 in the docker images, after updating it to Java 9 it worked.
So far I just changed the docker versions of the images using the JS processor and it should work now. I saw that we use java 11 for the arm images. Which version should we use for x86? I think it makes sense to harmonize the base image for all containers. So the question is, which base image should we use? Any suggestions? Philipp > On 15. Jun 2020, at 23:11, Dominik Riemer <[email protected]> wrote: > > Hi Philipp, > > coming back to this (and with later I really didn't mean ten days, sorry for > that) ;-) > I could also reproduce this now - it seems to be only an issue when running > in Docker. > > When looking at the container logs when running the element in Docker, I get > a NullPointerException in line 45 of JsEval (which could indicate that either > engine is null or code is null). > > So could this be an issue with the Java version we're using in the Docker > images? The currently used version is AdoptOpenJDK 11. > > I found a similar issue on so: > https://stackoverflow.com/questions/60916424/getenginebynamejavascript-returning-null-on-java-11 > > Could this be the cause? > > Dominik > > On 2020/06/05 09:02:10, "Dominik Riemer" <[email protected]> wrote: >> Hi Philipp, >> >> I can try to reproduce it later. >> >> Dominik >> >> -----Original Message----- >> From: Philipp Zehnder <[email protected]> >> Sent: Friday, June 5, 2020 11:01 AM >> To: [email protected] >> Subject: Re: Data processor to evaluate JavaScript >> >> Hi, >> >> I just tried it again and removed all docker images from the VM did a fresh >> pull. >> Unfortunately, with the same result. I tried the processor in both >> containers (pipeline-elements-all-jvm / processors-enricher-jvm). >> >> Does anyone else have an idea what the problem could be? >> >> Philipp >> >>> On 4. Jun 2020, at 12:00, Grainier Perera <[email protected]> wrote: >>> >>> Hmm, That's odd. It doesn't use any 3rd party libraries for the PE (we >>> use javax.script.*). However, I came across a similar situation with >>> this RDFMappingException[1]. The reason for that is JSEval use newly >>> introduced .requiredCodeblock() controller prop, and for some reason, >>> there's a cached Docker image without it. But when I cleaned that and >>> re-build, it worked. Could this be the reason? >>> >>> [1] io.fogsy.empire.pinto.RDFMappingException: Could not create an >>> instance of class org.apache.streampipes.model.output.OutputStrategy, >>> it does not have a default constructor >>> >>> Grainier Perera. >>> >>> >>> On Thu, 4 Jun 2020 at 15:17, Philipp Zehnder <[email protected]> wrote: >>> >>>> Hi all, >>>> >>>> I tried to use the Processor JavaScript Eval in a deployment of >>>> StreamPipes (Lite), running in Docker. >>>> Unfortunately, I get an error in the UI when starting the pipeline. >>>> There is no specific error message, just an error that the JavaScript >>>> Eval did not start. >>>> I also did not see anything in the logs. Did anyone else have a >>>> similar issue? >>>> >>>> The processor runs in the container pipeline-elements-all-jvm using >>>> the CLI. >>>> >>>> Running the same pipeline locally in my IDE works well, so I guess >>>> there is a problem when building the project. >>>> Is it possible that some libraries are not bundled correctly? >>>> >>>> Philipp >>>> >>>>> On 22. May 2020, at 16:57, Grainier Perera >>>>> <[email protected]> >>>> wrote: >>>>> >>>>> Hi Dominik, >>>>> >>>>> My bad. There were some cached images & that was the issue. Also >>>>> fixed >>>> the >>>>> PR as per your comments. Please review and merge [1]. >>>>> >>>>> [1] >>>>> https://github.com/apache/incubator-streampipes-extensions/pull/15 >>>>> >>>>> Thanks & Regards, >>>>> Grainier >>>>> >>>>> On Fri, 22 May 2020 at 16:25, Dominik Riemer <[email protected]> wrote: >>>>> >>>>>> Hi Grainier, >>>>>> >>>>>> we have a Github Actions workflow [1] for core+extensions that >>>>>> usually builds and pushes the latest development containers for dev >>>>>> on each >>>> push. >>>>>> You are using the latest CLI from dev and the tmpl_env or .env >>>>>> version >>>> is >>>>>> set to 0.67.0-SNAPSHOT, right? >>>>>> >>>>>> I just tested the JsEval processor (with CLI 0.67.0-SNAPSHOT, local >>>>>> backend with the latest changes from dev and your PR) and it works >>>> great 😊 >>>>>> The only thing I had to change was a typo in line 59 (use outEvent >>>> instead >>>>>> of event). >>>>>> >>>>>> In the controller, the supportedProtocols and supportedFormats are >>>>>> no longer needed, as they can now be declared for a whole module in >>>>>> the >>>> Init >>>>>> class (registerFormats/Protocols). I need to update this in the >>>>>> documentation. >>>>>> >>>>>> Dominik >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>> https://github.com/apache/incubator-streampipes/blob/dev/.github/work >>>> flows/build.yml >>>>>> >>>>>> -----Original Message----- >>>>>> From: Grainier Perera <[email protected]> >>>>>> Sent: Friday, May 22, 2020 11:58 AM >>>>>> To: [email protected] >>>>>> Subject: Re: Data processor to evaluate JavaScript >>>>>> >>>>>> Hi Dominik, >>>>>> >>>>>> I've implemented the following[1]. However, when I try to install >>>>>> the processor it gives me the following error. Any clue? I've used >>>>>> CLI to >>>> start >>>>>> sp. So, maybe the snapshot docker image doesn't have the recent >>>> changes? If >>>>>> so what are the steps to build the images with changes in-place? >>>>>> >>>>>> [1] >>>>>> https://github.com/apache/incubator-streampipes-extensions/pull/15 >>>>>> >>>>>> ERROR: >>>>>> https://streampipes.org/vocabulary/v1/UserDefinedOutputStrategy >>>>>> io.fogsy.empire.pinto.RDFMappingException: Could not create an >>>>>> instance >>>> of >>>>>> class org.apache.streampipes.model.output.OutputStrategy, it does >>>>>> not >>>> have >>>>>> a default constructor at >>>>>> io.fogsy.empire.pinto.RDFMapper.newInstance(RDFMapper.java:178) >>>>>> at io.fogsy.empire.pinto.RDFMapper.readValue(RDFMapper.java:261) >>>>>> at >>>>>> io.fogsy.empire.pinto.RDFMapper.valueToObject(RDFMapper.java:695) >>>>>> at >>>>>> io.fogsy.empire.pinto.RDFMapper.lambda$toObject$3(RDFMapper.java:38 >>>>>> 8) >>>>>> at >>>>>> >>>> io.fogsy.empire.pinto.RDFMapper$$Lambda$802/0000000000096040.apply(Un >>>> known >>>>>> Source) >>>>>> at >>>>>> >>>> io.fogsy.empire.pinto.RDFMapper$$Lambda$803/0000000000096580.apply(Un >>>> known >>>>>> Source) >>>>>> at java.util.stream.ReferencePipeline$3$1.accept(Unknown Source) at >>>>>> java.util.ArrayList$ArrayListSpliterator.forEachRemaining(Unknown >>>> Source) >>>>>> at java.util.stream.AbstractPipeline.copyInto(Unknown Source) at >>>>>> java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown Source) >>>>>> at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(Unknown >>>> Source) at >>>>>> java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(Unkn >>>>>> own >>>>>> Source) >>>>>> at java.util.stream.AbstractPipeline.evaluate(Unknown Source) at >>>>>> java.util.stream.ReferencePipeline.forEach(Unknown Source) at >>>>>> io.fogsy.empire.pinto.RDFMapper.readValue(RDFMapper.java:299) >>>>>> at >>>>>> >>>>>> >>>> org.apache.streampipes.serializers.jsonld.JsonLdTransformer.fromJsonL >>>> d(JsonLdTransformer.java:100) >>>>>> at >>>>>> >>>>>> >>>> org.apache.streampipes.manager.verification.ElementVerifier.transform >>>> (ElementVerifier.java:169) >>>>>> at >>>>>> >>>>>> >>>> org.apache.streampipes.manager.verification.ElementVerifier.verifyAnd >>>> Add(ElementVerifier.java:84) >>>>>> at >>>>>> >>>>>> >>>> org.apache.streampipes.manager.operations.Operations.verifyAndAddElem >>>> ent(Operations.java:96) >>>>>> at >>>>>> >>>>>> >>>> org.apache.streampipes.manager.endpoint.EndpointItemParser.parseAndAd >>>> dEndpointItem(EndpointItemParser.java:37) >>>>>> at >>>>>> >>>>>> >>>> org.apache.streampipes.rest.impl.PipelineElementImport.verifyAndAddEl >>>> ement(PipelineElementImport.java:93) >>>>>> at >>>>>> >>>>>> >>>> org.apache.streampipes.rest.impl.PipelineElementImport.addElement(Pip >>>> elineElementImport.java:89) >>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at >>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) >>>>>> >>>>>> Thanks, >>>>>> Grainier. >>>>>> >>>>>> On Thu, 21 May 2020 at 03:39, Dominik Riemer <[email protected]> wrote: >>>>>> >>>>>>> Hi Grainier, >>>>>>> >>>>>>> sounds good! >>>>>>> I've just pushed a commit that provides an initial version of the >>>>>>> user-defined output strategy. It lets you select the input schema >>>>>>> from the incoming stream and add/remove event properties. There is >>>>>>> one limitation, the component currently only works for flat event >>>>>>> structures, but it shouldn't be a problem to have a more advanced >>>>>>> schema editor for this output strategy soon (we already have this >>>>>>> in the StreamPipes connect code, but unfortunately, the pipeline >>>>>>> editor has yet to be migrated from AngularJS to Angular 2+ to >>>>>>> reuse these components). This is something we plan to do within the >>>>>>> next weeks. >>>>>>> >>>>>>> I added an example how to use the strategy to the examples repo: >>>>>>> >>>>>>> https://github.com/apache/incubator-streampipes-examples/blob/dev/ >>>>>>> stre >>>>>>> ampipes-pipeline-elements-examples-processors-jvm/src/main/java/or >>>>>>> g/ap >>>>>>> ache/streampipes/pe/examples/jvm/staticproperty/CodeInputExampleCo >>>>>>> ntro >>>>>>> ller.java >>>>>>> >>>>>>> Let me know what you think and if you need anything more/else! >>>>>>> >>>>>>> Dominik >>>>>>> >>>>>>> On 2020/05/19 17:14:21, Grainier Perera >>>>>>> <[email protected]> >>>>>>> wrote: >>>>>>>> Hi Dominik, >>>>>>>> >>>>>>>> Awesome. This is really cool & exactly what I was looking for. >>>>>>>> I'll use this with my implementation, and keep this thread posted >>>>>>>> with the >>>>>>> updates. >>>>>>>> ps: let us know when you add output strategy as well. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Grainier. >>>>>>>> >>>>>>>> On Tue, 19 May 2020 at 03:05, Dominik Riemer <[email protected]> >>>>>> wrote: >>>>>>>> >>>>>>>>> Hi Grainier, >>>>>>>>> >>>>>>>>> I just pushed an initial version of the static property to add >>>>>>> javascript >>>>>>>>> code. >>>>>>>>> It's a simple codeblock with linting enabled and a (currently >>>>>>>>> very >>>>>>> simple) >>>>>>>>> autocomplete feature that suggests existing properties from the >>>>>>>>> input >>>>>>> event >>>>>>>>> (currently, only if you press ctrl-space directly after typing >>>>>> "event" >>>>>>>>> followed by a dot, I'll fix that soon ;-) >>>>>>>>> >>>>>>>>> You can find an example how to use it here: >>>>>>>>> >>>>>>>>> >>>>>>> https://github.com/apache/incubator-streampipes-examples/blob/dev/ >>>>>>> stre >>>>>>> ampipes-pipeline-elements-examples-processors-jvm/src/main/java/or >>>>>>> g/ap >>>>>>> ache/streampipes/pe/examples/jvm/staticproperty/CodeInputExampleCo >>>>>>> ntro >>>>>>> ller.java >>>>>>>>> >>>>>>>>> Feedback and ideas for improvement are welcome! >>>>>>>>> >>>>>>>>> Dominik >>>>>>>>> >>>>>>>>> >>>>>>>>> On 2020/05/17 12:36:05, Grainier Perera >>>>>>>>> <[email protected]> >>>>>>>>> wrote: >>>>>>>>>> Hi Dominik, >>>>>>>>>> >>>>>>>>>> Thank you. I'll work on `Implement JS evaluator` [1]. >>>>>>>>>> >>>>>>>>>> [1] https://issues.apache.org/jira/browse/STREAMPIPES-135 >>>>>>>>>> >>>>>>>>>> Grainier. >>>>>>>>>> >>>>>>>>>> On Sun, 17 May 2020 at 18:02, Dominik Riemer >>>>>>>>>> <[email protected]> >>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Hi Grainier, >>>>>>>>>>> >>>>>>>>>>> great! I created an umbrella issue for that: >>>>>>>>>>> https://issues.apache.org/jira/browse/STREAMPIPES-132 >>>>>>>>>>> >>>>>>>>>>> I'll now start to implement the new static property for >>>>>>>>>>> entering >>>>>>> code. >>>>>>>>>>> >>>>>>>>>>> Dominik >>>>>>>>>>> >>>>>>>>>>> On 2020/05/15 01:29:44, Grainier Perera >>>>>>>>>>> <[email protected] >>>>>>>> >>>>>>>>>>> wrote: >>>>>>>>>>>> Hi, >>>>>>>>>>>> >>>>>>>>>>>> I'm +1 to the idea of `UserDefinedOutputStrategy`. As you >>>>>>>>>>>> said, >>>>>>>>> that'll >>>>>>>>>>> be >>>>>>>>>>>> re-usable in the future processors as well. So let's proceed >>>>>>>>>>>> with >>>>>>>>> that. >>>>>>>>>>>> >>>>>>>>>>>> Regards, >>>>>>>>>>>> Grainier >>>>>>>>>>>> >>>>>>>>>>>> On Fri, 15 May 2020 at 01:37, Dominik Riemer >>>>>>>>>>>> <[email protected]> >>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi, >>>>>>>>>>>>> cool, that sounds very good! >>>>>>>>>>>>> >>>>>>>>>>>>> Concerning the output, I think there are two ways to achieve >>>>>>> this: >>>>>>>>>>>>> - A collection static property in conjunction with a runtime >>>>>>>>> resolvable >>>>>>>>>>>>> output strategy that creates the output schema based on the >>>>>>> input >>>>>>>>> from >>>>>>>>>>> the >>>>>>>>>>>>> collection >>>>>>>>>>>>> - Or we create a new output strategy, e.g., let's call it >>>>>>>>>>>>> UserDefinedOutputStrategy, which would render a UI component >>>>>>> to let >>>>>>>>>>> users >>>>>>>>>>>>> add/remove/change event properties from the input schema. >>>>>>>>>>>>> I >>>>>>> also >>>>>>>>> like >>>>>>>>>>> the >>>>>>>>>>>>> idea of preselecting all input properties. >>>>>>>>>>>>> >>>>>>>>>>>>> I personally tend towards the second option as such a >>>>>>>>>>>>> strategy >>>>>>>>> might be >>>>>>>>>>>>> useful for other pipeline elements as well and it would ease >>>>>>> the >>>>>>>>> model >>>>>>>>>>>>> definition. We could also extend this component in the >>>>>>>>>>>>> future, >>>>>>>>> e.g., by >>>>>>>>>>>>> letting users enrich metadata such as measurement units >>>>>>> directly >>>>>>>>> in a >>>>>>>>>>>>> processor. >>>>>>>>>>>>> >>>>>>>>>>>>> But I'm also open to the first option and we can also just >>>>>>> start >>>>>>>>>>>>> implementing to see what works best. >>>>>>>>>>>>> >>>>>>>>>>>>> I'd also be happy to help with the implementation, e.g., I >>>>>>> could >>>>>>>>> work >>>>>>>>>>> on >>>>>>>>>>>>> the new static property for code input or the output >>>>>> strategy! >>>>>>>>>>>>> >>>>>>>>>>>>> Dominik >>>>>>>>>>>>> >>>>>>>>>>>>> On 2020/05/14 11:59:36, Grainier Perera < >>>>>>> [email protected] >>>>>>>>>> >>>>>>>>>>>>> wrote: >>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>> >>>>>>>>>>>>>> +1 to the idea of having a collection property containing >>>>>>>>>>>>>> +the >>>>>>>>> fields >>>>>>>>>>> of >>>>>>>>>>>>> the >>>>>>>>>>>>>> output properties. Also, If we can show the user with >>>>>>> available >>>>>>>>>>> fields >>>>>>>>>>>>> with >>>>>>>>>>>>>> metadata (of input event) populated in that same collection >>>>>>>>> property, >>>>>>>>>>>>> then >>>>>>>>>>>>>> the user can add/remove/updates fields easily. Let's say >>>>>>>>>>>>>> they >>>>>>>>> just >>>>>>>>>>> have >>>>>>>>>>>>> to >>>>>>>>>>>>>> return a map containing all the fields they defined in >>>>>>>>>>>>>> output >>>>>>>>>>> properties >>>>>>>>>>>>>> (i.e return {"existingId": 1, "existingTempInKelvin": >>>>>>>>>>>>>> 301, >>>>>>>>>>>>>> "newTempInCelsius": 28 } ). >>>>>>>>>>>>>> >>>>>>>>>>>>>> [image: sample.png] >>>>>>>>>>>>>> >>>>>>>>>>>>>> Regards, >>>>>>>>>>>>>> Grainier. >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Thu, 14 May 2020 at 12:33, Patrick Wiener < >>>>>>> [email protected]> >>>>>>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> sounds reasonable. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I share the thoughts of Philipp regarding the output >>>>>>> schema. >>>>>>>>> But >>>>>>>>>>> as you >>>>>>>>>>>>>>> said Grainier, we can also improve :) >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> btw: I really like the idea to also allow the more >>>>>>> technical >>>>>>>>> user >>>>>>>>>>> to >>>>>>>>>>>>>>> easily integrate their function snippets without having to >>>>>>>>>>>>>>> care about the actual execution. This gets a more FaaS >>>>>>> like >>>>>>>>>>> feeling >>>>>>>>>>>>> for >>>>>>>>>>>>>>> streaming. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Patrick >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Am 14.05.2020 um 08:26 schrieb Philipp Zehnder < >>>>>>>>>>> [email protected]>: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> it would be very useful to have such a processor. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I like the ideas for 1 & 2 and I think we can do it this >>>>>>> way. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Actually I think that the definition of the output scheme >>>>>>>>> could >>>>>>>>>>> be >>>>>>>>>>>>> quite >>>>>>>>>>>>>>> tricky. >>>>>>>>>>>>>>>> Because users can do anything with the event. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> How about we provide an option property? >>>>>>>>>>>>>>>> Users can select whether they want to define the whole >>>>>>> event >>>>>>>>> or >>>>>>>>>>> just >>>>>>>>>>>>>>> append properties. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> For the configurations of the properties we could do >>>>>>>>> something >>>>>>>>>>>>> similar >>>>>>>>>>>>>>> then we have in the PLC4xS7Adapter. >>>>>>>>>>>>>>>> A collection property containing the fields of the output >>>>>>>>>>> properties. >>>>>>>>>>>>>>>> Therefore, we have to decide what information we have to >>>>>>>>>>> provide. I >>>>>>>>>>>>>>> think if we would support all options it becomes very >>>>>>> cluttered >>>>>>>>>>> for the >>>>>>>>>>>>>>> user. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> What do you think? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Philipp >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On 14. May 2020, at 06:55, Grainier Perera < >>>>>>>>>>>>> [email protected]> >>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Hi Dominik, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I agree. We have to do some enhancements make this >>>>>>> processor >>>>>>>>>>> user >>>>>>>>>>>>>>> friendly. >>>>>>>>>>>>>>>>> Like you said it's nice to have; 1. A new static >>>>>>>>>>>>>>>>> property that supports code >>>>>>> highlighting & >>>>>>>>>>> syntax >>>>>>>>>>>>>>>>> checking. We can simply do such client-side validations >>>>>>>>> using >>>>>>>>>>>>>>> highlight.js >>>>>>>>>>>>>>>>> (BSD) [1], jshint (MIT) [2], etc... >>>>>>>>>>>>>>>>> 2. A mechanism to show the user with available fields >>>>>>>>> they can >>>>>>>>>>>>> use to >>>>>>>>>>>>>>>>> write code (similar to what you've done with usable >>>>>>>>> templates in >>>>>>>>>>>>>>>>> EmailPublisher using HtmlInputParameter). >>>>>>>>>>>>>>>>> 3. A mechanism to map output schema. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I think 1 & 2 go together and for the 3rd requirement >>>>>>> given >>>>>>>>>>> that we >>>>>>>>>>>>> are >>>>>>>>>>>>>>>>> targetting a more technical user group for this >>>>>>> processor, >>>>>>>>> we >>>>>>>>>>> can >>>>>>>>>>>>> let >>>>>>>>>>>>>>> them >>>>>>>>>>>>>>>>> define the output. Otherwise, it would be quite >>>>>>> difficult to >>>>>>>>>>> derive >>>>>>>>>>>>>>> return >>>>>>>>>>>>>>>>> type from the JS. Moreover, I'm planning to use Java's >>>>>>>>> builtin >>>>>>>>>>>>>>> ScriptEngine >>>>>>>>>>>>>>>>> to eval. Of course, there'll be limitations. But, we can >>>>>>>>> always >>>>>>>>>>>>> improve >>>>>>>>>>>>>>>>> them :) >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> [1] https://highlightjs.org/ [2] https://jshint.com/ >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Regards, >>>>>>>>>>>>>>>>> Grainier. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On Thu, 14 May 2020 at 00:53, Dominik Riemer < >>>>>>>>> [email protected] >>>>>>>>>>>> >>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Hi Grainier, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> in my opinion, this would definitely be a very >>>>>>> interesting >>>>>>>>>>>>> addition! We >>>>>>>>>>>>>>>>>> already briefly discussed such a feature in the past, >>>>>>>>> before we >>>>>>>>>>>>> went >>>>>>>>>>>>>>> to the >>>>>>>>>>>>>>>>>> ASF. I’d say that a JS evaluator would be >>>>>>>>>>>>>>>>>> definitely >>>>>>>>> useful to >>>>>>>>>>> a >>>>>>>>>>>>>>> slightly >>>>>>>>>>>>>>>>>> more technical target user group, but would >>>>>>>>>>>>>>>>>> provide >>>>>>> great >>>>>>>>>>>>> flexibility >>>>>>>>>>>>>>> for, >>>>>>>>>>>>>>>>>> e.g., data harmonization tasks. So let’s discuss >>>>>>>>>>>>>>>>>> what >>>>>>> is >>>>>>>>>>> needed to >>>>>>>>>>>>>>>>>> implement this! I’d guess that we (maybe) need to >>>>>>>>>>>>>>>>>> add >>>>>>> some >>>>>>>>>>>>>>> enhancements to >>>>>>>>>>>>>>>>>> the core to make this processor good from a >>>>>>>>>>>>>>>>>> usability >>>>>>>>>>> perspective. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> From a conceptual point of view, I guess the JS >>>>>>> processor >>>>>>>>> would >>>>>>>>>>>>> have no >>>>>>>>>>>>>>>>>> specific input requirements and a single static >>>>>>> property >>>>>>>>> that >>>>>>>>>>>>> allows >>>>>>>>>>>>>>> users >>>>>>>>>>>>>>>>>> to enter the code (maybe it would be cool to add a >>>>>>>>>>>>>>>>>> new >>>>>>>>> static >>>>>>>>>>>>> property >>>>>>>>>>>>>>> here >>>>>>>>>>>>>>>>>> that supports code highlighting or even syntax >>>>>>> checking). >>>>>>>>> The >>>>>>>>>>> only >>>>>>>>>>>>> open >>>>>>>>>>>>>>>>>> issue I see is the output schema – we would >>>>>>>>>>>>>>>>>> somehow >>>>>>> need to >>>>>>>>>>>>> extract the >>>>>>>>>>>>>>>>>> output from the JS function. This could probably >>>>>>> partially >>>>>>>>> be >>>>>>>>>>> done >>>>>>>>>>>>>>> using >>>>>>>>>>>>>>>>>> the CustomTransformOutput (see >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>> http://streampipes.apache.org/docs/docs/dev-guide-output-strategie >>>>>>>>> s/), >>>>>>>>>>>>>>>>>> but we would somehow need to derive type >>>>>>>>>>>>>>>>>> information >>>>>>> from >>>>>>>>> the >>>>>>>>>>> JS >>>>>>>>>>>>>>> function >>>>>>>>>>>>>>>>>> or introduce a feature to let users define/refine >>>>>>>>>>>>>>>>>> the >>>>>>>>> output >>>>>>>>>>>>> manually >>>>>>>>>>>>>>>>>> (e.g., to add semantic metadata). >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> What do you think would be best? And would you >>>>>>>>>>>>>>>>>> evaluate >>>>>>>>> the JS >>>>>>>>>>>>> code in >>>>>>>>>>>>>>>>>> Java or somehow else? >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Also, as I’ve just started to improve the SDK >>>>>>>>>>>>>>>>>> (e.g., >>>>>>>>> supporting >>>>>>>>>>>>>>> optional >>>>>>>>>>>>>>>>>> static properties and an easier way to define and >>>>>>> extract >>>>>>>>> model >>>>>>>>>>>>>>>>>> parameters), we can collect your requirements for >>>>>>>>>>>>>>>>>> the >>>>>>> JS >>>>>>>>>>> processor >>>>>>>>>>>>> and >>>>>>>>>>>>>>>>>> extend the SDK if needed, just say what you would >>>>>>>>>>>>>>>>>> like >>>>>>> to >>>>>>>>> have >>>>>>>>>>> 😉 >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Dominik >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> From: Grainier Perera <[email protected]> >>>>>>>>>>>>>>>>>> Sent: Wednesday, May 13, 2020 2:41 PM >>>>>>>>>>>>>>>>>> To: [email protected] >>>>>>>>>>>>>>>>>> Subject: Data processor to evaluate JavaScript >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> I'm planning to implement a JavaScript eval data >>>>>>>>> processor. As >>>>>>>>>>> the >>>>>>>>>>>>> name >>>>>>>>>>>>>>>>>> suggests, users will be able to write some >>>>>>>>>>>>>>>>>> JavaScript >>>>>>> code >>>>>>>>>>> which >>>>>>>>>>>>> takes >>>>>>>>>>>>>>> in >>>>>>>>>>>>>>>>>> an event (as a map), do some processing on the >>>>>>>>>>>>>>>>>> event, >>>>>>> and >>>>>>>>>>> return a >>>>>>>>>>>>> new >>>>>>>>>>>>>>> map >>>>>>>>>>>>>>>>>> which then gets converted to an sp-event. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> example js: >>>>>>>>>>>>>>>>>> function process(event) { >>>>>>>>>>>>>>>>>> // do processing here. >>>>>>>>>>>>>>>>>> // return processed event. >>>>>>>>>>>>>>>>>> return {id: http://event.id, tempInCelsius: >>>>>>>>>>> (event.tempInKelvin >>>>>>>>>>>>> - >>>>>>>>>>>>>>>>>> 273.15)}; >>>>>>>>>>>>>>>>>> }; >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Will this be a good addition to pipeline-elements? >>>>>>> What do >>>>>>>>> you >>>>>>>>>>>>> think? >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Regards, >>>>>>>>>>>>>>>>>> Grainier. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>>> >>>> >>>> >>>> >> >> >> >>
