Hi all,
I am using a query to fetch data from MYSQL as follows:
var df = spark.read.
format("jdbc").
option("url", "jdbc:mysql://10.0.0.192:3306/retail_db").
option("driver" ,"com.mysql.jdbc.Driver").
option("user", "retail_dba").
option("password", "cloudera").
option("dbtable", "orders").
option("partitionColumn", "order_id").
option("lowerBound", "1").
option("upperBound", "68883").
option("numPartitions", "4").
load()
Question is, can I use a pseudo column (like ROWNUM in Oracle or
RRN(employeeno) in DB2) in option where I specify the "partitionColumn" ?
If not, can we specify a partition column which is not a primary key ?
Best,
Ravion
