Spark SQL "partition stride"?

2016-01-11 Thread Keith Freeman
The spark docs section for "JDBC to Other Databases" (https://spark.apache.org/docs/latest/sql-programming-guide.html#jdbc-to-other-databases) describes the partitioning as "... Notice that lowerBound and upperBound are just used to decide the partition stride, not for filtering the rows in

python rdd.partionBy(): any examples of a custom partitioner?

2015-12-07 Thread Keith Freeman
I'm not a python expert, so I'm wondering if anybody has a working example of a partitioner for the "partitionFunc" argument (default "portable_hash") to rdd.partitionBy()? - To unsubscribe, e-mail:

Spark 1.4.0 SQL JDBC partition stride?

2015-06-21 Thread Keith Freeman
The spark docs section for JDBC to Other Databases (https://spark.apache.org/docs/latest/sql-programming-guide.html#jdbc-to-other-databases) describes the partitioning as ... Notice that lowerBound and upperBound are just used to decide the partition stride, not for filtering the rows in