Thank you for the clarification. That was my understanding too. However how to provide the upper bound as it changes for every call in real life. For example it is not required for sqoop.
On Mon, 6 Nov 2017 at 8:20 am, Nicolas Paris <nipari...@gmail.com> wrote: > Le 05 nov. 2017 à 22:02, ayan guha écrivait : > > Can you confirm if JDBC DF Reader actually loads all data from source to > driver > > memory and then distributes to the executors? > > apparently yes when not using partition column > > > > And this is true even when a > > partition column is provided? > > No, in this case, each worker send a jdbc call accordingly to > documentation > https://spark.apache.org/docs/latest/sql-programming-guide.html#jdbc-to-other-databases > > -- Best Regards, Ayan Guha
