For layer 1, what about working through this link as a starting point : https://spark.apache.org/docs/latest/rdd-programming-guide.html#transformations ?
On Sat, 1 Jun 2019 at 09:21, Ahmet Altay <al...@google.com> wrote: > Thank you Reza. That separation makes sense to me. > > On Wed, May 29, 2019 at 6:26 PM Reza Rokni <r...@google.com> wrote: > >> +1 >> >> I think there will be at least two layers of this; >> >> Layer 1 - Using primitives : I do join, GBK, Aggregation... with system x >> this way, what is the canonical equivalent in Beam. >> Layer 2 - Patterns : I read and join Unbounded and Bounded Data in system >> x this way, what is the canonical equivalent in Beam. >> >> I suspect as a first pass Layer 1 is reasonably well bounded work, there >> would need to be agreement on "canonical" version of how to do something in >> Beam as this could be seen to be opinionated. As there are often a >> multitude of ways of doing x.... >> > > Once we identify a set of layer 1 items, we could crowd source the > canonical implementations. I believe we can use our usual code review > process to settle on a version that is agreeable. (Examples have the same > issue, they are probably opinionated today based on the author but it works > out.) > > >> >> >> On Thu, 30 May 2019 at 08:56, Ahmet Altay <al...@google.com> wrote: >> >>> Hi all, >>> >>> Inspired by the user asking about a Spark feature in Beam [1] in the >>> release thread, I searched the user@ list and noticed a few instances >>> of people asking for question like "I can do X in Spark, how can I do that >>> in Beam?" Would it make sense to add documentation to explain how certain >>> tasks that can be accomplished in Beam with side by side examples of doing >>> the same task in Beam/Spark etc. It could help with on-boarding because it >>> will be easier for people to leverage their existing knowledge. It could >>> also help other frameworks as well, because it will serve as a Rosetta >>> stone with two translations. >>> >>> Questions I have are: >>> - Would such a thing be a helpful? >>> - Is it feasible? Would a few pages worth of examples can cover enough >>> use cases? >>> >>> Thank you! >>> Ahmet >>> >>> [1] >>> https://lists.apache.org/thread.html/b73a54aa1e6e9933628f177b04a8f907c26cac854745fa081c478eff@%3Cdev.beam.apache.org%3E >>> >> >> >> -- >> >> This email may be confidential and privileged. If you received this >> communication by mistake, please don't forward it to anyone else, please >> erase all copies and attachments, and please let me know that it has gone >> to the wrong person. >> >> The above terms reflect a potential business arrangement, are provided >> solely as a basis for further discussion, and are not intended to be and do >> not constitute a legally binding obligation. No legally binding obligations >> will be created, implied, or inferred until an agreement in final form is >> executed in writing by all parties involved. >> > -- This email may be confidential and privileged. If you received this communication by mistake, please don't forward it to anyone else, please erase all copies and attachments, and please let me know that it has gone to the wrong person. The above terms reflect a potential business arrangement, are provided solely as a basis for further discussion, and are not intended to be and do not constitute a legally binding obligation. No legally binding obligations will be created, implied, or inferred until an agreement in final form is executed in writing by all parties involved.
