In most cases predicates that you add to jdbcDF will be push down into
oracle, preventing the whole table from being sent over.
df.where("column = 1")
Another common pattern is to save the table to parquet or something for
repeat querying.
Michael
On Fri, Sep 25, 2015 at 3:13 PM, Cui Lin <icecreamlc...@gmail.com> wrote:
>
>
> Hello, All,
>
> I found the examples for JDBC connection are mostly read the whole table
> and then do operations like joining.
>
> val jdbcDF = sqlContext.read.format("jdbc").options(
> Map("url" -> "jdbc:postgresql:dbserver",
> "dbtable" -> "schema.tablename")).load()
>
>
> Sometimes it is not practical since the whole table data is too big and
> not necessary.
>
> What makes sense to me is to use sparksql to get subset data from oracle
> tables using sql-like statement.
> I couldn't find such examples. Can someone show me?
>
>
>
> --
> Best regards!
>
> Lin,Cui
>
>
>
> --
> Best regards!
>
> Lin,Cui
>
