Is it possible to have Spark Data Frame Writer write based on RangePartioning?
For Ex -
I have 10 distinct values for column_a, say 1 to 10.
df.write
.partitionBy("column_a")
Above code by default will create 10 folders .. column_a=1,column_a=2
...column_a=10
I want to see if it is possible to have these partitions based on bucket -
col_a=1to5, col_a=5-10 .. or something like that? Then also have query engine
respect it
Thanks,
Nishit
