The spark docs section for "JDBC to Other Databases" (https://spark.apache.org/docs/latest/sql-programming-guide.html#jdbc-to-other-databases) describes the partitioning as "... Notice that lowerBound and upperBound are just used to decide the partition stride, not for filtering the rows in table."

What is meant by "partition stride" here, I'm not familiar with the phrase and googling didn't help.

Also, is the behaviour of this partitioning described in detail somewhere? Looking at my SQL query log I've figured out what it's doing in my example (X = (upperBound - lowerBound) / numPartitions):

  query * where partitionColumn < lowerBound
query * where partitionColumn >= lowerBound and partitionColumn < lowerBound + X query * where parititionColumn >= lowerBound+X and partitionColumn < lowerBound+X+X
  .... until the query gets to upperBound

But it would be nice to know if there's docs on this?

Reply via email to