[GitHub] spark issue #19343: [SPARK-22121][SQL] Correct database location for namenod...
Github user squito commented on the issue: https://github.com/apache/spark/pull/19343 OK, closing this and the jira --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19343: [SPARK-22121][SQL] Correct database location for namenod...
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19343 Thanks! Maybe we can close it now and revisit it when we have a better way to resolve the file system specific issues? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19343: [SPARK-22121][SQL] Correct database location for namenod...
Github user squito commented on the issue: https://github.com/apache/spark/pull/19343 I don't see much point in putting this in the docs ... it seems too fine-grained a detail to be useful there. I just don't see the users who encounter this exception from going to look at the spot in the docs to figure out exactly what is wrong. I put an example exception in the JIRA, so at least users can find it with a search. Sounds like you feel pretty strongly we should close this as "won't fix"? I'd still prefer to have this in, but will settle for just having the workaround be easily searchable --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19343: [SPARK-22121][SQL] Correct database location for namenod...
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19343 @squito Thank you! Instead of changing the source codes, could we just update the document https://spark.apache.org/docs/2.2.0/sql-programming-guide.html#hive-tables ? This might be enough for this issue. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19343: [SPARK-22121][SQL] Correct database location for namenod...
Github user squito commented on the issue: https://github.com/apache/spark/pull/19343 whoops, sorry I wrote [CORE] out of habit! > Spark SQL might not be deployed in the HDFS system. Conceptually, this HDFS-specific codes should not be part of our HiveExternalCatalog . HiveExternalCatalog is just for using Hive metastore. It does not assume we use HDFS. yes, I totally understand that. Even for users that are on hdfs, this is clear user-error, they should be using hive's metatool to update the database location. Originally I thought this would be unnecessary complication in spark, but with enough complaints I figured maybe spark could just handle it automatically. Is there another place this could go instead? Anyway, if you really feel like this doesn't belong in spark, that is fine. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
