Hi Jorge, Yes, it is 3pm berlin time (CET) you can found more details in the invitation email [1]
Best regards, Bertty [1] https://lists.apache.org/thread/njdg57srv9bjlyqs3o7og3pclbd88mv4 On Tue, Jan 4, 2022 at 1:52 PM jorge Arnulfo Quiané Ruiz < [email protected]> wrote: > @Bertty 3pm CET time right? :p > > On Tue, Jan 4, 2022 at 1:09 PM kamalesh palanisamy <[email protected]> > wrote: > >> That’s perfect. I am looking forward to it. >> >> On Tue, Jan 4, 2022 at 3:55 AM Bertty Contreras <[email protected]> >> wrote: >> >> > Hi Kamalesh, >> > >> > I sent the invitation on Friday at 3pm :D. >> > >> > If you have any questions let me know. >> > >> > Best regards, >> > Bertty >> > >> > On Tue, Jan 4, 2022 at 1:00 AM kamalesh palanisamy < >> [email protected]> >> > wrote: >> > >> >> Okay that sounds perfect. Thank you! >> >> >> >> On Mon, Jan 3, 2022 at 4:52 AM Bertty Contreras <[email protected]> >> >> wrote: >> >> >> >>> So nice, I will organize my schedule for it and i will come back to >> you >> >>> with options. Mean while I will collecting all the designs and other >> >>> elements that are done and could help you with the implementation of >> the >> >>> new feature ;), >> >>> >> >>> Best regards, >> >>> Bertty >> >>> >> >>> On Mon 3. Jan 2022 at 04:41, kamalesh palanisamy < >> [email protected]> >> >>> wrote: >> >>> >> >>>> Hi, >> >>>> Thank you for the explanation. Yes, I feel it would be better if we >> >>>> could discuss it so that everything is clear. I am free from >> >>>> Wednesday-Saturday anytime after 3 PM Germany time. You can select >> which >> >>>> every day suits your schedule best during this time. >> >>>> >> >>>> Thanks, >> >>>> Kamalesh P >> >>>> >> >>>> >> >>>> On Sun, Jan 2, 2022 at 6:28 PM Bertty Contreras <[email protected] >> > >> >>>> wrote: >> >>>> >> >>>>> The main is idea of wayang is to provide a layer that pick the best >> >>>>> combination of platform to process a query, you can see the details >> on the >> >>>>> paper rheemix[1] >> >>>>> >> >>>>> Then providing a SQL-API will allow to transform a query into >> >>>>> different operators of wayang that will allow optimization with >> platform >> >>>>> that only have SQL like postgres with platforms that don’t SQL >> lenguaje >> >>>>> like giraph. >> >>>>> >> >>>>> The idea to use calcite, is coming from the intermediate >> >>>>> representation that calcite generates that will allows us to create >> the >> >>>>> wayang plan with an “udf” that are translateble again to SQL or >> >>>>> translatable to a executable code that can be executed by flink, as >> an >> >>>>> example. >> >>>>> >> >>>>> Imagen the query that it said something like: >> >>>>> >> >>>>> Select A.a,A.b,A.c from A join A.a = X.a …. >> >>>>> >> >>>>> Then X(10TB) is on HDFS and A(100MB) is on postgres, then the plan >> to >> >>>>> execute will something like: >> >>>>> >> >>>>> Select A.a from A(1MB), this file is small then you can do broadcast >> >>>>> and filter using flink. >> >>>>> >> >>>>> Then the join results are just 2 records, the wayang will perform >> the >> >>>>> query on postgres using the 2 record as condition. >> >>>>> >> >>>>> But also could occurs that the join answer is 1TB, in that case, the >> >>>>> data of postgres will be move to HDFS and the all the rest of the >> process >> >>>>> will be on using flink. >> >>>>> >> >>>>> Currently the optimizer is taking the decision of what platform will >> >>>>> be used depending on the amount of data to process and data >> movement. Then >> >>>>> the SQL-API will provide an way of “freedom” the decisions because >> we will >> >>>>> have all the intermediate representation to performs changes. >> >>>>> >> >>>>> After we have the SQL-API we will be adding platforms that just >> >>>>> support and SQL ;), as you said. >> >>>>> >> >>>>> The idea of using the intermediate representation it maybe sound >> weird >> >>>>> to you, but we can have a meeting to explain you better, then you >> can >> >>>>> understand better the full concept and also give us your feedback, >> let me >> >>>>> if hyou are available and when and I will freedom my schedule for >> it ;). >> >>>>> I’m in Germany just to you figure if we have some timezone >> differences ;). >> >>>>> >> >>>>> Best regards, >> >>>>> Bertty >> >>>>> >> >>>>> [1] >> >>>>> https://wayang.apache.org/assets/pdf/paper/journal_vldb.pdf >> >>>>> >> >>>>> >> >>>>> On Sun 2. Jan 2022 at 17:43, kamalesh palanisamy < >> >>>>> [email protected]> wrote: >> >>>>> >> >>>>>> Hi Bertty, >> >>>>>> Thank you for the information! I would love to work on adding the >> SQL >> >>>>>> API for Wayang. Basically, now I need to add a new platform for the >> >>>>>> wayang-platforms that supports SQL through apache calcite? Am I >> right? >> >>>>>> Please do correct me if I am wrong. >> >>>>>> >> >>>>>> Thanks, >> >>>>>> Kamalesh P >> >>>>>> >> >>>>>> >> >>>>>> On Sun, Jan 2, 2022 at 3:36 AM Bertty Contreras < >> [email protected]> >> >>>>>> wrote: >> >>>>>> >> >>>>>>> Hi Kamalesh, >> >>>>>>> >> >>>>>>> Currently, Apache Wayang(Incubating) has the issues listed in Jira >> >>>>>>> [1]. One feature that the community didn't have time to work on >> is the SQL >> >>>>>>> API for Apache Wayang(Incubating) [2]; the main idea is to use >> Apache >> >>>>>>> Calcite [3] as the parser of the SQL and then do something like >> Spark >> >>>>>>> adapter of calcite [4]. If you want to contribute to this >> feature, it will >> >>>>>>> be so awesome :D. >> >>>>>>> >> >>>>>>> If you found another issue interesting, let me know, or even if >> you >> >>>>>>> have some idea of a feature will be so awesome too :D >> >>>>>>> >> >>>>>>> Best regards, >> >>>>>>> Bertty >> >>>>>>> >> >>>>>>> [1] https://issues.apache.org/jira/projects/WAYANG >> >>>>>>> [2] >> >>>>>>> >> https://issues.apache.org/jira/projects/WAYANG/issues/WAYANG-25?filter=allopenissues >> >>>>>>> [3] https://calcite.apache.org >> >>>>>>> [4] https://github.com/apache/calcite/tree/master/spark >> >>>>>>> >> >>>>>>> On Sun, Jan 2, 2022 at 6:50 AM kamalesh palanisamy < >> >>>>>>> [email protected]> wrote: >> >>>>>>> >> >>>>>>>> Hi, >> >>>>>>>> My name is Kamalesh and I am currently looking to contribute to >> the >> >>>>>>>> project, but I couldn't find any proper issues. Can you help me >> >>>>>>>> with any >> >>>>>>>> features you would like me to contribute to?. Thanks! >> >>>>>>>> Thanks, >> >>>>>>>> Kamalesh P >> >>>>>>>> >> >>>>>>> -- >> >> Thanks, >> >> Kamalesh P >> >> >> > -- >> Thanks, >> Kamalesh P >> >
