Re: Joining files

2013-11-20 Thread Alex Boisvert
On Nov 20, 2013 8:34 AM, Something Something mailinglist...@gmail.com wrote: Questions: 1) I don't see APIs for LEFT, FULL OUTER Joins. True? The join operations are so documented here: http://spark.incubator.apache.org/docs/latest/api/core/index.html#org.apache.spark.rdd.PairRDDFunctions

Re: Joining files

2013-11-20 Thread Alex Boisvert
On Nov 20, 2013 8:34 AM, Something Something mailinglist...@gmail.com wrote: Questions: 1) I don't see APIs for LEFT, FULL OUTER Joins. True? 2) Apache Pig provides different join types such as 'replicated', 'skewed'. Now 'replicated' may not be a concern in Spark 'cause everything

Re: Joining files

2013-11-18 Thread Something Something
Was my question so dumb? Or, is this not a good use case for Spark? On Sun, Nov 17, 2013 at 11:41 PM, Something Something mailinglist...@gmail.com wrote: I am a newbie to both Spark Scala, but I've been working with Hadoop/Pig for quite some time. We've quite a few ETL processes running