[
https://issues.apache.org/jira/browse/SPARK-5472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14304383#comment-14304383
]
Anand Mohan Tumuluri commented on SPARK-5472:
---------------------------------------------
Thanks again [~rxin] and [~tmyklebu]. My bad, I only checked the table creation
scripts in before and made assumptions.
This would very well satisfy our use case.
The custom partitioning conditions would remove the need to use SQL
conditionals as well.
One more question, how do I 'get' new data that got inserted in the source
table(s)? Would 'refresh table' work for this?
> Add support for reading from and writing to a JDBC database
> -----------------------------------------------------------
>
> Key: SPARK-5472
> URL: https://issues.apache.org/jira/browse/SPARK-5472
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Reporter: Tor Myklebust
> Assignee: Tor Myklebust
> Priority: Blocker
> Fix For: 1.3.0
>
>
> It would be nice to be able to make a table in a JDBC database appear as a
> table in Spark SQL. This would let users, for instance, perform a JOIN
> between a DataFrame in Spark SQL with a table in a Postgres database.
> It might also be nice to be able to go the other direction -- save a
> DataFrame to a database -- for instance in an ETL job.
> Edited to clarify: Both of these tasks are certainly possible to accomplish
> at the moment with a little bit of ad-hoc glue code. However, there is no
> fundamental reason why the user should need to supply the table schema and
> some code for pulling data out of a ResultSet row into a Catalyst Row
> structure when this information can be derived from the schema of the
> database table itself.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
