Re: JDBC Very Slow

2016-09-16 Thread Takeshi Yamamuro
Hi, It'd be better to set `predicates` in jdbc arguments for loading in parallel. See: https://github.com/apache/spark/blob/branch-1.6/sql/core/src/main/scala/org/apache/spark/sql/DataFrameReader.scala#L200 // maropu On Sat, Sep 17, 2016 at 7:46 AM, Benjamin Kim wrote: > I

Re: JDBC Very Slow

2016-09-16 Thread Benjamin Kim
I am testing this in spark-shell. I am following the Spark documentation by simply adding the PostgreSQL driver to the Spark Classpath. SPARK_CLASSPATH=/path/to/postgresql/driver spark-shell Then, I run the code below to connect to the PostgreSQL database to query. This is when I have

Re: JDBC Very Slow

2016-09-16 Thread Nikolay Zhebet
Hi! Can you split init code with current comand? I thing it is main problem in your code. 16 сент. 2016 г. 8:26 PM пользователь "Benjamin Kim" написал: > Has anyone using Spark 1.6.2 encountered very slow responses from pulling > data from PostgreSQL using JDBC? I can get to