[
https://issues.apache.org/jira/browse/SPARK-5472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14304588#comment-14304588
]
Tor Myklebust commented on SPARK-5472:
--------------------------------------
If the data in the underlying table changes, this code might not work reliably;
some partitions might have new data and others won't. If you change the schema
of the underlying table after you make it visible to Spark SQL, retrieving data
will (probably) blow up. Whatever behaviour you might observe from this code
when given a changing underlying table will not be behaviour you can rely on.
> Add support for reading from and writing to a JDBC database
> -----------------------------------------------------------
>
> Key: SPARK-5472
> URL: https://issues.apache.org/jira/browse/SPARK-5472
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Reporter: Tor Myklebust
> Assignee: Tor Myklebust
> Priority: Blocker
> Fix For: 1.3.0
>
>
> It would be nice to be able to make a table in a JDBC database appear as a
> table in Spark SQL. This would let users, for instance, perform a JOIN
> between a DataFrame in Spark SQL with a table in a Postgres database.
> It might also be nice to be able to go the other direction -- save a
> DataFrame to a database -- for instance in an ETL job.
> Edited to clarify: Both of these tasks are certainly possible to accomplish
> at the moment with a little bit of ad-hoc glue code. However, there is no
> fundamental reason why the user should need to supply the table schema and
> some code for pulling data out of a ResultSet row into a Catalyst Row
> structure when this information can be derived from the schema of the
> database table itself.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
