[GitHub] spark pull request #14523: [SPARK-16936] [SQL] Case Sensitivity Support for ...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14523 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #14523: [SPARK-16936] [SQL] Case Sensitivity Support for ...
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/14523#discussion_r73814553
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/MetadataCacheSuite.scala ---
@@ -85,4 +86,28 @@ class MetadataCacheSuite extends QueryTest with
SharedSQLContext {
assert(newCount > 0 && newCount < 100)
}}
}
+
+ test("case sensitivity support in temporary view refresh") {
+withSQLConf(SQLConf.CASE_SENSITIVE.key -> "false") {
+ withTempView("view_refresh") {
+withTempPath { (location: File) =>
+ // Create a Parquet directory
+ spark.range(start = 0, end = 100, step = 1, numPartitions = 3)
+.write.parquet(location.getAbsolutePath)
+
+ // Read the directory in
+
spark.read.parquet(location.getAbsolutePath).createOrReplaceTempView("view_refresh")
+
+ // Delete a file
+ deleteOneFileInDirectory(location)
+ intercept[SparkException] {sql("select count(*) from
view_refresh").first()}
--- End diff --
nit: use `intercept[..](...)` for one line code.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #14523: [SPARK-16936] [SQL] Case Sensitivity Support for ...
GitHub user gatorsmile opened a pull request: https://github.com/apache/spark/pull/14523 [SPARK-16936] [SQL] Case Sensitivity Support for Refresh Temp Table ### What changes were proposed in this pull request? Currently, the `refreshTable` API is always case sensitive. When users use the view name without the exact case match, the API silently ignores the call. Users might expect the command has been successfully completed. However, when users run the subsequent SQL commands, they might still get the exception, like ``` Job aborted due to stage failure: Task 1 in stage 4.0 failed 1 times, most recent failure: Lost task 1.0 in stage 4.0 (TID 7, localhost): java.io.FileNotFoundException: File file:/private/var/folders/4b/sgmfldk15js406vk7lw5llzwgn/T/spark-bd4b9ea6-9aec-49c5-8f05-01cff426211e/part-r-0-0c84b915-c032-4f2e-abf5-1d48fdbddf38.snappy.parquet does not exist ``` This PR is to fix the issue. ### How was this patch tested? Added a test case. You can merge this pull request into a Git repository by running: $ git pull https://github.com/gatorsmile/spark refreshTempTable Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/14523.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #14523 commit ade173c2397613b2649d6f61e8fe27c2d659d088 Author: gatorsmile Date: 2016-08-07T04:27:41Z fix commit f62fb19791f590a8110d6a7be65987b348dc167a Author: gatorsmile Date: 2016-08-07T05:16:21Z fix2 commit fb0dd0b03640c9456313d8b7a63203607940e683 Author: gatorsmile Date: 2016-08-07T05:35:55Z update the comment --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
