Hi Lefteris, It is possible to run the query that you describe using Apache Wayang; one example that could help you understand how to connect different platforms in one task is the Query3Hybrid [1]. Nevertheless, if you could explain more about the shape of the query will look, we can give you more hints on how you could do it.
I mean with a shape something like the following example: The need to run an SQL query in top Postgres, then join the result with data stored in HDFS or S3 and transform it to execute a page rank using Graphchi; after the results are processed, the information needs to be joined with another dataset. Let me know the shape, and we will be able provide more information. Thank you so much for your question; this will help us a lot to know what to write in the documentation in the example section :D Best regards, Bertty [1] https://github.com/apache/incubator-wayang/blob/main/wayang-benchmark/code/main/scala/org/apache/wayang/apps/tpch/queries/Query3Hybrid.scala On Tue, Jan 25, 2022 at 4:08 PM Lefteris Lymperopoulos < [email protected]> wrote: > Hello Dev Team, > Excuse me if this mail isn't appropriate for this mail address. Since I > could not find any documentation for Wayang besides that in the official > website I would like to ask you if you could help me with this issue. I > have 3 VMs that connect to each other. In the first VM I want to run Spark, > in the second I want to run Postgres and in the third GraphChi. I also have > Wayang installed in the first VM and I intend to develop my app in Java. Is > it possible to connect Wayang to Spark, Postgres and GraphChi in order to > run my queries? If yes, could you please show me how to do it? Or do these > platforms have to be in the same VM? Any help would be greatly appreciated. > Best regards, > Lefteris Lymperopoulos >
