Hi, I have already register the new mapping in the mappings Spark Platform and the mapping will map the new operator with the new Spark Operator.
It could also be that Spark doesn't recognize the implementation or the mapping isn't done correctly from my side, however if it is so then I should got an error no plan to enumerate. However I believe Spark recognize the operator but the planenumerator doesn't know what to do with it? This is my guess. Best regards, Matthew On Fri, Aug 26, 2022, 10:02 Zoi Kaoudi <[email protected]> wrote: > Hi Matthew, > did you register the new mapping? There should be a class or structure > that lists all the mappings available. > Best > -- > Zoi > > Στις Παρασκευή 26 Αυγούστου 2022, 09:20:24 π.μ. CEST, ο χρήστης jorge > Arnulfo Quiané Ruiz <[email protected]> έγραψε: > > I see. It seems the mapping is not being done correctly as wayang is not > being able to find a Spark implementation for your wayang operator. > > I could walk the code with you early next week. In the meantime, perhaps > Bertty or Rodrigo had a similar issue before, @Bertty and @Rodrigo did you > face a similar problem before? > > — > Jorge > > On Thu, 25 Aug 2022 at 9:00 PM MatthewJ Sanyoto <[email protected] > > > wrote: > > > Hi, > > > > So I created a new operator MapHackItOperator in the wayang basic modules > > operator. This functions essentially the same as MapOperator in the same > > module. > > > > Since wayang need a platform to execute this operator and to test, in the > > wayang platform java modules I updated the Mapping and add > MapHackitMapping > > and JavaMapHackItOperator , this is the translation part as well as the > > execution, it works with Java Plugin. > > > > The problem is for only SparkPlugin I updated the wayang platform spark > > modules for the SparkPlugin so I created Mapping, MapHackItMapping and > > SparkMapHackItOperator that extends to MapHackItOperator with the same > code > > as the SparkMap to translate it into SparkOperator, I think the > translation > > part is working but Wayang don't know how to enumerate the plan according > > to the exception or don't know where to output the results? correct me if > > I'm wrong. > > > > I hope it doesn't make it more confusing. > > > > Best regards, > > Matthew > > > > On Thu, Aug 25, 2022, 20:45 jorge Arnulfo Quiané Ruiz < > [email protected] > > > > > wrote: > > > > > Hello Matthew, > > > > > > To better understand the problem, are you trying to copy the > > > JavaMapOperator into a Spark operator? Or you are trying to copy a > > > SparkOperator into a new spark operator such as you did for Java? > > > > > > Best, > > > Jorge > > > > > > On Thu, 25 Aug 2022 at 5:56 PM MatthewJ Sanyoto < > > [email protected] > > > > > > > wrote: > > > > > > > Hi, > > > > > > > > I have a problem regarding implementing a new custom operator using > > > Spark. > > > > So I was trying to map a new operator from Wayang Basic operator to > the > > > > Spark Platform to execute in Spark. > > > > I tried using the Java Platform (only Java Plugin) and just copied > the > > > > code(MapOperator, Mapping, JavaMapOperator and MapMapping) but with > > > > different operators name to test and it worked but not for Spark > > > Platform. > > > > (only Spark Plugin) > > > > > > > > Maybe someone here could please explain to me why I got this error. > > > > Caused by: org.apache.wayang.core.api.exception.WayangException: No > > > > implementations that concatenate out@Alternative[2x > > > > ~Alternative[[MapHackIt[1->1, id=3c947bc5]]], 1e683a3e] with > > > > [in@Alternative[2x > > > > ~Alternative[[LocalCallbackSink[1->0, id=69fb6037]]], 6babf3bf]]. > > > > > > > > Maybe I am missing out on something for Spark? > > > > > > > > Best Regards, > > > > Matthew > > > > > > > > > >
