Thank you for the feedback so far. It seems like this will be generally helpful :)
I guess next step would be, would anyone be interested in working in this area? We can potentially break this down into starter tasks. On Sat, Jun 1, 2019 at 7:00 PM Ankur Goenka <goe...@google.com> wrote: > +1 for the proposal. > Compatibility Matrix > <https://beam.apache.org/documentation/runners/capability-matrix/> can be > a good place to show case parity between different runners. > +1 > Do you think we should write 2 way examples [Spark, Flink, ..]<=>Beam? > Both ways, would be most useful I believe. > > > > On Sat, Jun 1, 2019 at 4:31 PM Reza Rokni <r...@google.com> wrote: > >> For layer 1, what about working through this link as a starting point : >> https://spark.apache.org/docs/latest/rdd-programming-guide.html#transformations >> ? >> > +1 > >> On Sat, 1 Jun 2019 at 09:21, Ahmet Altay <al...@google.com> wrote: >> >>> Thank you Reza. That separation makes sense to me. >>> >>> On Wed, May 29, 2019 at 6:26 PM Reza Rokni <r...@google.com> wrote: >>> >>>> +1 >>>> >>>> I think there will be at least two layers of this; >>>> >>>> Layer 1 - Using primitives : I do join, GBK, Aggregation... with system >>>> x this way, what is the canonical equivalent in Beam. >>>> Layer 2 - Patterns : I read and join Unbounded and Bounded Data in >>>> system x this way, what is the canonical equivalent in Beam. >>>> >>>> I suspect as a first pass Layer 1 is reasonably well bounded work, >>>> there would need to be agreement on "canonical" version of how to do >>>> something in Beam as this could be seen to be opinionated. As there are >>>> often a multitude of ways of doing x.... >>>> >>> >>> Once we identify a set of layer 1 items, we could crowd source the >>> canonical implementations. I believe we can use our usual code review >>> process to settle on a version that is agreeable. (Examples have the same >>> issue, they are probably opinionated today based on the author but it works >>> out.) >>> >>> >>>> >>>> >>>> On Thu, 30 May 2019 at 08:56, Ahmet Altay <al...@google.com> wrote: >>>> >>>>> Hi all, >>>>> >>>>> Inspired by the user asking about a Spark feature in Beam [1] in the >>>>> release thread, I searched the user@ list and noticed a few instances >>>>> of people asking for question like "I can do X in Spark, how can I do that >>>>> in Beam?" Would it make sense to add documentation to explain how certain >>>>> tasks that can be accomplished in Beam with side by side examples of doing >>>>> the same task in Beam/Spark etc. It could help with on-boarding because it >>>>> will be easier for people to leverage their existing knowledge. It could >>>>> also help other frameworks as well, because it will serve as a Rosetta >>>>> stone with two translations. >>>>> >>>>> Questions I have are: >>>>> - Would such a thing be a helpful? >>>>> - Is it feasible? Would a few pages worth of examples can cover enough >>>>> use cases? >>>>> >>>>> Thank you! >>>>> Ahmet >>>>> >>>>> [1] >>>>> https://lists.apache.org/thread.html/b73a54aa1e6e9933628f177b04a8f907c26cac854745fa081c478eff@%3Cdev.beam.apache.org%3E >>>>> >>>> >>>> >>>> -- >>>> >>>> This email may be confidential and privileged. If you received this >>>> communication by mistake, please don't forward it to anyone else, please >>>> erase all copies and attachments, and please let me know that it has gone >>>> to the wrong person. >>>> >>>> The above terms reflect a potential business arrangement, are provided >>>> solely as a basis for further discussion, and are not intended to be and do >>>> not constitute a legally binding obligation. No legally binding obligations >>>> will be created, implied, or inferred until an agreement in final form is >>>> executed in writing by all parties involved. >>>> >>> >> >> -- >> >> This email may be confidential and privileged. If you received this >> communication by mistake, please don't forward it to anyone else, please >> erase all copies and attachments, and please let me know that it has gone >> to the wrong person. >> >> The above terms reflect a potential business arrangement, are provided >> solely as a basis for further discussion, and are not intended to be and do >> not constitute a legally binding obligation. No legally binding obligations >> will be created, implied, or inferred until an agreement in final form is >> executed in writing by all parties involved. >> >
