The spark docs section for "JDBC to Other Databases"
(https://spark.apache.org/docs/latest/sql-programming-guide.html#jdbc-to-other-databases)
describes the partitioning as "... Notice that lowerBound and upperBound
are just used to decide the partition stride, not for filtering the rows
in table."
What is meant by "partition stride" here, I'm not familiar with the
phrase and googling didn't help.
Also, is the behavior of this partitioning described in detail
somewhere? Looking at my SQL query log I've figured out what it's doing
in my example:
say X = (upperBound - lowerBound) / numPartitions):
query * where partitionColumn < lowerBound
query * where partitionColumn >= lowerBound and partitionColumn <
lowerBound + X
query * where parititionColumn >= lowerBound+X and partitionColumn <
lowerBound+2X
.... until the query gets to upperBound
But it would be nice to know if there's docs on this?
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org