[
https://issues.apache.org/jira/browse/SPARK-24295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17361310#comment-17361310
]
Jungtaek Lim commented on SPARK-24295:
--
[~parshabani]
Please follow up SPARK-27188.
But there're
[
https://issues.apache.org/jira/browse/SPARK-24295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17361245#comment-17361245
]
Parsa Shabani commented on SPARK-24295:
---
Hello all,
Any updates on this? [~kabhwan] are you still
[
https://issues.apache.org/jira/browse/SPARK-24295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17192574#comment-17192574
]
Jungtaek Lim commented on SPARK-24295:
--
[~sta...@gmail.com]
Thanks for sharing the workaround.
[
https://issues.apache.org/jira/browse/SPARK-24295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17192280#comment-17192280
]
Avner Livne commented on SPARK-24295:
-
for those looking for a temporary workaround:
run this code
[
https://issues.apache.org/jira/browse/SPARK-24295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17045011#comment-17045011
]
Jungtaek Lim commented on SPARK-24295:
--
[~iqbal_khattra] [~alfredo-gimenez-bv]
Hi, if you're open
[
https://issues.apache.org/jira/browse/SPARK-24295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16905824#comment-16905824
]
Mohan Parthasarathy commented on SPARK-24295:
-
Can someone explain how it is useful to know
[
https://issues.apache.org/jira/browse/SPARK-24295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16795519#comment-16795519
]
Jungtaek Lim commented on SPARK-24295:
--
[~iqbal_khattra] [~alfredo-gimenez-bv]
I would be really
[
https://issues.apache.org/jira/browse/SPARK-24295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16787511#comment-16787511
]
Jungtaek Lim commented on SPARK-24295:
--
[~alfredo-gimenez-bv]
I meant removing "deleted output
[
https://issues.apache.org/jira/browse/SPARK-24295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16787502#comment-16787502
]
Alfredo Gimenez commented on SPARK-24295:
-
Currently old metadata files are already deleted
[
https://issues.apache.org/jira/browse/SPARK-24295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786569#comment-16786569
]
Jungtaek Lim commented on SPARK-24295:
--
[~iqbal_khattra] [~alfredo-gimenez-bv]
Would we be happy
[
https://issues.apache.org/jira/browse/SPARK-24295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16782510#comment-16782510
]
Alfredo Gimenez commented on SPARK-24295:
-
Our current workaround FWII:
We've added a streaming
[
https://issues.apache.org/jira/browse/SPARK-24295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16781879#comment-16781879
]
Iqbal Singh commented on SPARK-24295:
-
Agree, [~kabhwan]. It's very tricky to purge metadata as it
[
https://issues.apache.org/jira/browse/SPARK-24295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16777240#comment-16777240
]
Jungtaek Lim commented on SPARK-24295:
--
[~alfredo-gimenez-bv]
Unfortunately no, cause I don't think
[
https://issues.apache.org/jira/browse/SPARK-24295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16777197#comment-16777197
]
Alfredo Gimenez commented on SPARK-24295:
-
We've run into the exact same issue, I uploaded a
[
https://issues.apache.org/jira/browse/SPARK-24295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16772588#comment-16772588
]
Jungtaek Lim commented on SPARK-24295:
--
While I submitted the PR to reflect the last comment, I
[
https://issues.apache.org/jira/browse/SPARK-24295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16772322#comment-16772322
]
Jungtaek Lim commented on SPARK-24295:
--
FileStreamSinkLog cannot be removed even we don't leverage
[
https://issues.apache.org/jira/browse/SPARK-24295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16772296#comment-16772296
]
Jungtaek Lim commented on SPARK-24295:
--
Please correct me if I'm missing here. I just skimmed the
[
https://issues.apache.org/jira/browse/SPARK-24295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16638635#comment-16638635
]
Iqbal Singh commented on SPARK-24295:
-
hey [~XuanYuan],
DO we have any plans of pulling this one
[
https://issues.apache.org/jira/browse/SPARK-24295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547873#comment-16547873
]
Li Yuanjian commented on SPARK-24295:
-
Thanks for your detailed explain.
You can check this:
[
https://issues.apache.org/jira/browse/SPARK-24295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16545577#comment-16545577
]
Iqbal Singh commented on SPARK-24295:
-
Hey [~XuanYuan],
We are processing 3000 files every 5
[
https://issues.apache.org/jira/browse/SPARK-24295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16544453#comment-16544453
]
Li Yuanjian commented on SPARK-24295:
-
Could you give more detailed information about how the
21 matches
Mail list logo
