[
https://issues.apache.org/jira/browse/SPARK-15044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1626#comment-1626
]
Soumitra Sulav commented on SPARK-15044:
Facing below issues with respect to same scenario.
[
https://issues.apache.org/jira/browse/SPARK-15044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16033895#comment-16033895
]
Dongjoon Hyun commented on SPARK-15044:
---
Hi, All.
According to SPARK-10198, that option seems to be
[
https://issues.apache.org/jira/browse/SPARK-15044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15779766#comment-15779766
]
Yuming Wang commented on SPARK-15044:
-
I've tested on v2.1.0-rc5, it works fine if
[
https://issues.apache.org/jira/browse/SPARK-15044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15430314#comment-15430314
]
Sean Owen commented on SPARK-15044:
---
The use case here is that someone or something deleted some data
[
https://issues.apache.org/jira/browse/SPARK-15044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15419798#comment-15419798
]
huangyu commented on SPARK-15044:
-
Hi, I know it isn't Spark's fault. However I think maybe it's better
[
https://issues.apache.org/jira/browse/SPARK-15044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15419079#comment-15419079
]
Artur commented on SPARK-15044:
---
I don't know what should we do with this issue. The root cause is invalid
[
https://issues.apache.org/jira/browse/SPARK-15044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15418994#comment-15418994
]
saurabh paliwal commented on SPARK-15044:
-
Hi!
so sorry I mixed it up. So the use case should be
[
https://issues.apache.org/jira/browse/SPARK-15044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15418935#comment-15418935
]
saurabh paliwal commented on SPARK-15044:
-
Hi!
I am so sorry, I mixed it up.
So the premise is
[
https://issues.apache.org/jira/browse/SPARK-15044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15418876#comment-15418876
]
Sean Owen commented on SPARK-15044:
---
The partition would still exist in this case, no?
> spark-sql
[
https://issues.apache.org/jira/browse/SPARK-15044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15418872#comment-15418872
]
saurabh paliwal commented on SPARK-15044:
-
Hi!
I agree with Artur. So let's assume there is a
[
https://issues.apache.org/jira/browse/SPARK-15044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15418862#comment-15418862
]
Sean Owen commented on SPARK-15044:
---
That's the problem. If the semantics were, "query anything that
[
https://issues.apache.org/jira/browse/SPARK-15044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15418841#comment-15418841
]
Artur commented on SPARK-15044:
---
Why does result miss something?
If data doesn't exist in hdfs - it should
[
https://issues.apache.org/jira/browse/SPARK-15044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15418788#comment-15418788
]
Sean Owen commented on SPARK-15044:
---
That seems like worse behavior, because it silently 'succeeds'
[
https://issues.apache.org/jira/browse/SPARK-15044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15418784#comment-15418784
]
Artur commented on SPARK-15044:
---
I know that you should not do this.
But if someone does - spark will not
[
https://issues.apache.org/jira/browse/SPARK-15044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15418746#comment-15418746
]
Sean Owen commented on SPARK-15044:
---
Is it really an error? the files were manually deleted and
[
https://issues.apache.org/jira/browse/SPARK-15044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15418735#comment-15418735
]
Artur commented on SPARK-15044:
---
Tested on spark 2.0.X (master branch) (latest commit:
[
https://issues.apache.org/jira/browse/SPARK-15044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15415118#comment-15415118
]
Artur commented on SPARK-15044:
---
So if some Hive metadata is incorrect - we will always get an exception?
[
https://issues.apache.org/jira/browse/SPARK-15044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15297540#comment-15297540
]
lichenglin commented on SPARK-15044:
This exception is caused by “the HiveContext cache the metadata
[
https://issues.apache.org/jira/browse/SPARK-15044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15287256#comment-15287256
]
saurabh paliwal commented on SPARK-15044:
-
Anyway, as a work-around, I have just caught the
[
https://issues.apache.org/jira/browse/SPARK-15044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15286861#comment-15286861
]
saurabh paliwal commented on SPARK-15044:
-
Only for parquet tables.
set
[
https://issues.apache.org/jira/browse/SPARK-15044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15268252#comment-15268252
]
huangyu commented on SPARK-15044:
-
Oh I know, thank you. Of course, If dropping the partition after
[
https://issues.apache.org/jira/browse/SPARK-15044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15268071#comment-15268071
]
Xin Wu commented on SPARK-15044:
Sorry. What I meant was that after I removed the path manually, then did
[
https://issues.apache.org/jira/browse/SPARK-15044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15267978#comment-15267978
]
huangyu commented on SPARK-15044:
-
Maybe you can try in Spark 1.6.1, because when I tried in Spark 1.6.1
[
https://issues.apache.org/jira/browse/SPARK-15044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15267974#comment-15267974
]
huangyu commented on SPARK-15044:
-
I removed the path by hadoop command,"hadoop fs -rmr
[
https://issues.apache.org/jira/browse/SPARK-15044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15266116#comment-15266116
]
Xin Wu commented on SPARK-15044:
I tried {code}alter table test drop partition (p=1){code} , then the
[
https://issues.apache.org/jira/browse/SPARK-15044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15266042#comment-15266042
]
Niranjan Molkeri` commented on SPARK-15044:
---
I tried to reproduce the error in Spark 1.5.0,
26 matches
Mail list logo