[
https://issues.apache.org/jira/browse/SPARK-36801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17417382#comment-17417382
]
Apache Spark commented on SPARK-36801:
--------------------------------------
User 'senthh' has created a pull request for this issue:
https://github.com/apache/spark/pull/34042
> Document change for Spark sql jdbc
> ----------------------------------
>
> Key: SPARK-36801
> URL: https://issues.apache.org/jira/browse/SPARK-36801
> Project: Spark
> Issue Type: Documentation
> Components: Documentation
> Affects Versions: 3.0.0, 3.0.2, 3.0.3, 3.1.0, 3.1.1, 3.1.2
> Reporter: Senthil Kumar
> Priority: Trivial
>
> Reading using Spark SQL jdbc DataSource does not maintain nullable type and
> changes "non nullable" columns to "nullable".
>
> For example:
> mysql> CREATE TABLE Persons(Id int NOT NULL, FirstName varchar(255), LastName
> varchar(255), Age int);
> Query OK, 0 rows affected (0.04 sec)
> mysql> show tables;
> +-------------------+
> | Tables_in_test_db |
> +-------------------+
> | Persons |
> +-------------------+
> 1 row in set (0.00 sec)
> mysql> desc Persons;
> +-----------+--------------+------+-----+---------+-------+
> | Field | Type | Null | Key | Default | Extra |
> +-----------+--------------+------+-----+---------+-------+
> | Id | int | NO | | NULL | |
> | FirstName | varchar(255) | YES | | NULL | |
> | LastName | varchar(255) | YES | | NULL | |
> | Age | int | YES | | NULL | |
> +-----------+--------------+------+-----+---------+-------+
>
>
> {color:#cc7832}val {color}df = spark.read.format({color:#6a8759}"jdbc"{color})
>
> .option({color:#6a8759}"database"{color}{color:#cc7832},{color}{color:#6a8759}"Test_DB"{color})
> .option({color:#6a8759}"user"{color}{color:#cc7832},
> {color}{color:#6a8759}"root"{color})
> .option({color:#6a8759}"password"{color}{color:#cc7832},
> {color}{color:#6a8759}""{color})
> .option({color:#6a8759}"driver"{color}{color:#cc7832},
> {color}{color:#6a8759}"com.mysql.cj.jdbc.Driver"{color})
> .option({color:#6a8759}"url"{color}{color:#cc7832},
> {color}{color:#6a8759}"jdbc:mysql://localhost:3306/Test_DB"{color})
> .option({color:#6a8759}"query"{color}{color:#cc7832},
> {color}{color:#6a8759}"(select * from Persons)"{color})
> .load()
> df.printSchema()
>
> *output:*
>
> root
> |-- Id: integer (nullable = true)
> |-- FirstName: string (nullable = true)
> |-- LastName: string (nullable = true)
> |-- Age: integer (nullable = true)
>
>
> So we need to add a note, in Documentation[1], "All columns are automatically
> converted to be nullable for compatibility reasons."
> Ref:
> [1
> ][https://spark.apache.org/docs/latest/sql-data-sources-jdbc.html#jdbc-to-other-databases]
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
