[GitHub] spark pull request: [SPARK-15515] [SPARK-15514] [SQL] Error Handli...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/13283#issuecomment-221711653
  
**[Test build #59296 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59296/consoleFull)**
 for PR 13283 at commit 
[`3ac2b93`](https://github.com/apache/spark/commit/3ac2b930a1629a2fd21602a7ceb22429fb8ce372).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-15515] [SPARK-15514] [SQL] Error Handli...

[gatorsmile]

Github user gatorsmile commented on a diff in the pull request:

https://github.com/apache/spark/pull/13283#discussion_r64652370
  
--- Diff: python/pyspark/sql/utils.py ---
@@ -77,6 +83,8 @@ def deco(*a, **kw):
 raise QueryExecutionException(s.split(': ', 1)[1], 
stackTrace)
 if s.startswith('java.lang.IllegalArgumentException: '):
 raise IllegalArgumentException(s.split(': ', 1)[1], 
stackTrace)
+if s.startswith('java.lang.NoClassDefFoundError: '):
--- End diff --

uh, I see. Will revert it back. Thanks!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-15515] [SPARK-15514] [SQL] Error Handli...

[zsxwing]

Github user zsxwing commented on a diff in the pull request:

https://github.com/apache/spark/pull/13283#discussion_r64651774
  
--- Diff: python/pyspark/sql/utils.py ---
@@ -77,6 +83,8 @@ def deco(*a, **kw):
 raise QueryExecutionException(s.split(': ', 1)[1], 
stackTrace)
 if s.startswith('java.lang.IllegalArgumentException: '):
 raise IllegalArgumentException(s.split(': ', 1)[1], 
stackTrace)
+if s.startswith('java.lang.NoClassDefFoundError: '):
--- End diff --

`NoClassDefFoundError` rarely happens unless we miss something in 
`spark2RemovedClasses`. Hiding it makes us hard to figure out the real place 
quickly.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-15515] [SPARK-15514] [SQL] Error Handli...

[gatorsmile]

Github user gatorsmile commented on a diff in the pull request:

https://github.com/apache/spark/pull/13283#discussion_r64647927
  
--- Diff: 
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala
 ---
@@ -108,21 +108,20 @@ case class DataSource(
   dataSource
 case Failure(error) =>
   if (error.isInstanceOf[ClassNotFoundException]) {
-val className = error.getMessage
-if (spark2RemovedClasses.contains(className)) {
-  throw new ClassNotFoundException(s"$className is removed 
in Spark 2.0. " +
+// error.getMessage is the class name of provider2. 
Instead, we use provider here.
--- End diff --

Sure, will do it. Thanks!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-15515] [SPARK-15514] [SQL] Error Handli...

[gatorsmile]

Github user gatorsmile commented on a diff in the pull request:

https://github.com/apache/spark/pull/13283#discussion_r64648263
  
--- Diff: python/pyspark/sql/utils.py ---
@@ -77,6 +83,8 @@ def deco(*a, **kw):
 raise QueryExecutionException(s.split(': ', 1)[1], 
stackTrace)
 if s.startswith('java.lang.IllegalArgumentException: '):
 raise IllegalArgumentException(s.split(': ', 1)[1], 
stackTrace)
+if s.startswith('java.lang.NoClassDefFoundError: '):
--- End diff --

I think it is still helpful. This is to hide the long java stack trace for 
improving the error reporting. It was introduced in the PR 
https://github.com/apache/spark/pull/7135. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-15515] [SPARK-15514] [SQL] Error Handli...

[zsxwing]

Github user zsxwing commented on a diff in the pull request:

https://github.com/apache/spark/pull/13283#discussion_r64631597
  
--- Diff: 
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala
 ---
@@ -108,21 +108,20 @@ case class DataSource(
   dataSource
 case Failure(error) =>
   if (error.isInstanceOf[ClassNotFoundException]) {
-val className = error.getMessage
-if (spark2RemovedClasses.contains(className)) {
-  throw new ClassNotFoundException(s"$className is removed 
in Spark 2.0. " +
+// error.getMessage is the class name of provider2. 
Instead, we use provider here.
--- End diff --

This is for link issues. But it will be `NoClassDefFoundError`


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-15515] [SPARK-15514] [SQL] Error Handli...

[zsxwing]

Github user zsxwing commented on a diff in the pull request:

https://github.com/apache/spark/pull/13283#discussion_r64631743
  
--- Diff: python/pyspark/sql/utils.py ---
@@ -77,6 +83,8 @@ def deco(*a, **kw):
 raise QueryExecutionException(s.split(': ', 1)[1], 
stackTrace)
 if s.startswith('java.lang.IllegalArgumentException: '):
 raise IllegalArgumentException(s.split(': ', 1)[1], 
stackTrace)
+if s.startswith('java.lang.NoClassDefFoundError: '):
--- End diff --

The Python changes are not necessary. Right?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-15515] [SPARK-15514] [SQL] Error Handli...

[zsxwing]

Github user zsxwing commented on a diff in the pull request:

https://github.com/apache/spark/pull/13283#discussion_r64631435
  
--- Diff: 
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala
 ---
@@ -108,21 +108,20 @@ case class DataSource(
   dataSource
 case Failure(error) =>
   if (error.isInstanceOf[ClassNotFoundException]) {
-val className = error.getMessage
-if (spark2RemovedClasses.contains(className)) {
-  throw new ClassNotFoundException(s"$className is removed 
in Spark 2.0. " +
+// error.getMessage is the class name of provider2. 
Instead, we use provider here.
--- End diff --

In a second thought, I don't think we need this `if` branch. Could you just 
remove it?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-15515] [SPARK-15514] [SQL] Error Handli...

[zsxwing]

Github user zsxwing commented on the pull request:

https://github.com/apache/spark/pull/13283#issuecomment-221665525
  
@gatorsmile I saw it reports `org.apache.spark.sql.AnalysisException: 
Failed to find data source: mydatabase. Please find packages at 
http://spark-packages.org;; line 1 pos 15` for the following statement: 
`sql("select id from `mydatabase`.`file_path`")`. I'm not familiar with the 
table name resolution. Is it correct?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-15515] [SPARK-15514] [SQL] Error Handli...

[gatorsmile]

Github user gatorsmile commented on a diff in the pull request:

https://github.com/apache/spark/pull/13283#discussion_r64626689
  
--- Diff: 
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala
 ---
@@ -132,12 +131,11 @@ case class DataSource(
   }
   }
 } catch {
-  case e: NoClassDefFoundError => // This one won't be caught by 
Scala NonFatal
-// NoClassDefFoundError's class name uses "/" rather than "." 
for packages
-val className = e.getMessage.replaceAll("/", ".")
-if (spark2RemovedClasses.contains(className)) {
-  throw new ClassNotFoundException(s"$className was removed in 
Spark 2.0. " +
-"Please check if your library is compatible with Spark 
2.0", e)
+  case e: NoClassDefFoundError =>
+// e.getMessage is the class name of provider2. Instead, we 
use provider here.
+if (spark2RemovedClasses.contains(provider)) {
--- End diff --

Will revert the changes for `NoClassDefFoundError`. Thanks!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-15515] [SPARK-15514] [SQL] Error Handli...

[zsxwing]

Github user zsxwing commented on the pull request:

https://github.com/apache/spark/pull/13283#issuecomment-221662515
  
> ISSUE 3: Unable to detect incompatibility libraries for Spark 2.0 in Data 
Source Resolution. We report a strange error message:
> SQL Example:
>   select id from 
`org.apache.spark.sql.sources.HadoopFsRelationProvider`.`file_path`
> Error Message: 
>   Table or view not found: 
`org.apache.spark.sql.sources.HadoopFsRelationProvider`.`file_path`

This is not an issue you need to fix. `HadoopFsRelationProvider` is just an 
interface in Spark 1.6. The user should not use it like this. If someone sees 
HadoopFsRelationProvider is not found, it's usually a link issue. E.g., 
`com.databricks.spark.avro.DefaultSource` extends `HadoopFsRelationProvider`, 
however, `HadoopFsRelationProvider` has been removed in 2.0, so when loading 
`com.databricks.spark.avro.DefaultSource`, it will throw 
`NoClassDefFoundError(HadoopFsRelationProvider)` instead of 
`ClassNotFoundException(com.databricks.spark.avro.DefaultSource)`


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-15515] [SPARK-15514] [SQL] Error Handli...

[zsxwing]

Github user zsxwing commented on a diff in the pull request:

https://github.com/apache/spark/pull/13283#discussion_r64622938
  
--- Diff: 
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala
 ---
@@ -132,12 +131,11 @@ case class DataSource(
   }
   }
 } catch {
-  case e: NoClassDefFoundError => // This one won't be caught by 
Scala NonFatal
-// NoClassDefFoundError's class name uses "/" rather than "." 
for packages
-val className = e.getMessage.replaceAll("/", ".")
-if (spark2RemovedClasses.contains(className)) {
-  throw new ClassNotFoundException(s"$className was removed in 
Spark 2.0. " +
-"Please check if your library is compatible with Spark 
2.0", e)
+  case e: NoClassDefFoundError =>
+// e.getMessage is the class name of provider2. Instead, we 
use provider here.
+if (spark2RemovedClasses.contains(provider)) {
--- End diff --

You should not change this. If `provider` is not found, `loadClass` will 
throw `ClassNotFoundException`. If a class used by provider are not found, 
`NoClassDefFoundError` will be thrown.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-15515] [SPARK-15514] [SQL] Error Handli...

[zsxwing]

Github user zsxwing commented on a diff in the pull request:

https://github.com/apache/spark/pull/13283#discussion_r64622417
  
--- Diff: 
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala
 ---
@@ -132,12 +131,11 @@ case class DataSource(
   }
   }
 } catch {
-  case e: NoClassDefFoundError => // This one won't be caught by 
Scala NonFatal
-// NoClassDefFoundError's class name uses "/" rather than "." 
for packages
-val className = e.getMessage.replaceAll("/", ".")
-if (spark2RemovedClasses.contains(className)) {
-  throw new ClassNotFoundException(s"$className was removed in 
Spark 2.0. " +
-"Please check if your library is compatible with Spark 
2.0", e)
+  case e: NoClassDefFoundError =>
+// e.getMessage is the class name of provider2. Instead, we 
use provider here.
+if (spark2RemovedClasses.contains(provider)) {
--- End diff --

In my previous PR, I want to provide a better message for e.g., 
`org.apache.spark.sql.DataFrame` not found. It usually happens when calling 
some method (has a `org.apache.spark.sql.DataFrame` parameter) in a class that 
is compiled with an old Spark. Obviously, here `provider` won't be 
`org.apache.spark.sql.DataFrame`.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-15515] [SPARK-15514] [SQL] Error Handli...

[gatorsmile]

Github user gatorsmile commented on the pull request:

https://github.com/apache/spark/pull/13283#issuecomment-221606617
  
Thank you!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-15515] [SPARK-15514] [SQL] Error Handli...

[rxin]

Github user rxin commented on the pull request:

https://github.com/apache/spark/pull/13283#issuecomment-221495300
  
Just want to say --- this is a really cool change!

(I didn't actually look at the details. Would be great for somebody else to 
do a detailed review.)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-15515] [SPARK-15514] [SQL] Error Handli...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/13283#issuecomment-221466629
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/59249/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-15515] [SPARK-15514] [SQL] Error Handli...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/13283#issuecomment-221466626
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-15515] [SPARK-15514] [SQL] Error Handli...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/13283#issuecomment-221466522
  
**[Test build #59249 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59249/consoleFull)**
 for PR 13283 at commit 
[`f6a4517`](https://github.com/apache/spark/commit/f6a45177609c3c2ab7cd376dd6eb1a4527e693e7).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-15515] [SPARK-15514] [SQL] Error Handli...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/13283#issuecomment-221466099
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-15515] [SPARK-15514] [SQL] Error Handli...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/13283#issuecomment-221466102
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/59254/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-15515] [SPARK-15514] [SQL] Error Handli...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/13283#issuecomment-221466000
  
**[Test build #59254 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59254/consoleFull)**
 for PR 13283 at commit 
[`c313ffc`](https://github.com/apache/spark/commit/c313ffc88488229c45e74cde97210c8686e1c745).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-15515] [SPARK-15514] [SQL] Error Handli...

[gatorsmile]

Github user gatorsmile commented on the pull request:

https://github.com/apache/spark/pull/13283#issuecomment-221457452
  
Just realized this PR is related to 
https://github.com/apache/spark/pull/13201. Also tried to improve the exception 
reporting in Python through hiding the long java stack trace. 

@zsxwing @marmbrus Could you please help me review the changes? Thanks!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-15515] [SPARK-15514] [SQL] Error Handli...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/13283#issuecomment-221457208
  
**[Test build #59254 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59254/consoleFull)**
 for PR 13283 at commit 
[`c313ffc`](https://github.com/apache/spark/commit/c313ffc88488229c45e74cde97210c8686e1c745).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-15515] [SPARK-15514] [SQL] Error Handli...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/13283#issuecomment-221456373
  
**[Test build #59251 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59251/consoleFull)**
 for PR 13283 at commit 
[`26d72ae`](https://github.com/apache/spark/commit/26d72ae697b57fb8ad1a4f0357af0b726a76e443).
 * This patch **fails Python style tests**.
 * This patch merges cleanly.
 * This patch adds the following public classes _(experimental)_:
  * `class ClassNotFoundException(CapturedException):`


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-15515] [SPARK-15514] [SQL] Error Handli...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/13283#issuecomment-221456376
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-15515] [SPARK-15514] [SQL] Error Handli...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/13283#issuecomment-221456377
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/59251/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-15515] [SPARK-15514] [SQL] Error Handli...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/13283#issuecomment-221455992
  
**[Test build #59251 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59251/consoleFull)**
 for PR 13283 at commit 
[`26d72ae`](https://github.com/apache/spark/commit/26d72ae697b57fb8ad1a4f0357af0b726a76e443).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-15515] [SPARK-15514] [SQL] Error Handli...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/13283#issuecomment-221452796
  
**[Test build #59249 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59249/consoleFull)**
 for PR 13283 at commit 
[`f6a4517`](https://github.com/apache/spark/commit/f6a45177609c3c2ab7cd376dd6eb1a4527e693e7).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-15515] [SPARK-15514] [SQL] Error Handli...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/13283#issuecomment-221436957
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/59229/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-15515] [SPARK-15514] [SQL] Error Handli...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/13283#issuecomment-221436953
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-15515] [SPARK-15514] [SQL] Error Handli...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/13283#issuecomment-221436797
  
**[Test build #59229 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59229/consoleFull)**
 for PR 13283 at commit 
[`948340b`](https://github.com/apache/spark/commit/948340b15e504effcbc160deeb2e859a31714f39).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-15515] [SPARK-15514] [SQL] Error Handli...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/13283#issuecomment-221422916
  
**[Test build #59229 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59229/consoleFull)**
 for PR 13283 at commit 
[`948340b`](https://github.com/apache/spark/commit/948340b15e504effcbc160deeb2e859a31714f39).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-15515] [SPARK-15514] [SQL] Error Handli...

[gatorsmile]

GitHub user gatorsmile opened a pull request:

https://github.com/apache/spark/pull/13283

[SPARK-15515] [SPARK-15514] [SQL] Error Handling in Running SQL Directly On 
Files

 What changes were proposed in this pull request?
This PR is to address the following issues: 

- **ISSUE 1:** For ORC source format, we are reporting the strange error 
message when we did not enable Hive support: 
```SQL
SQL Example: 
  select id from `org.apache.spark.sql.hive.orc`.`file_path`
Error Message:
  Table or view not found: `org.apache.spark.sql.hive.orc`.`file_path`
```
Instead, we should issue the error message like:
```
Desired Error Message:
   The ORC data source must be used with Hive support enabled
```
- **ISSUE 2:** For the Avro format, we still report the error message like:

The example query is like
  ```SQL
SQL Example: 
  select id from `avro`.`file_path`
  select id from `com.databricks.spark.avro`.`file_path`
Error Message:
  Table or view not found: `com.databricks.spark.avro`.`file_path`
   ```
The desired message should be like:
```
Desired Error Message:
  Failed to find data source: avro. Please use Spark package 
http://sparkpackages.org/package/databricks/spark-avro;
```

- **ISSUE 3:** Unable to detect incompatibility libraries for Spark 2.0 in 
Data Source Resolution.
```SQL
SQL Example:
  select id from 
`org.apache.spark.sql.sources.HadoopFsRelationProvider`.`file_path`
Error Message: 
  Table or view not found: 
`org.apache.spark.sql.sources.HadoopFsRelationProvider`.`file_path`
```
The desired message should be like:
```
Desired Error Message:
   org.apache.spark.sql.sources.HadoopFsRelationProvider is removed in 
Spark 2.0. Please check if your library is compatible with Spark 2.0
```

 How was this patch tested?
Added test cases to cover all the above issues

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gatorsmile/spark runSQLAgainstFile

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/13283.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #13283


commit 948340b15e504effcbc160deeb2e859a31714f39
Author: gatorsmile 
Date:   2016-05-24T21:52:01Z

fix.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org