[GitHub] spark issue #12855: [SPARK-10216][SQL] Avoid creating empty files during ove...
Github user milanvdmria commented on the issue: https://github.com/apache/spark/pull/12855 The issue that @jurriaan reported is still there in Spark 2.1.0. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #12855: [SPARK-10216][SQL] Avoid creating empty files during ove...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/12855 @DanielMe Yes, actually it seems a different issue when you use `emptyRDD[Row]`. Apparently, this case does not produce any partitions whereas the code provided by @jurriaan produces some empty partitions. This was reverted because the latter case fails. So.. it seems the former case has been being failed from older versions and the latter is not being failed after this one is reverted. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #12855: [SPARK-10216][SQL] Avoid creating empty files during ove...
Github user DanielMe commented on the issue: https://github.com/apache/spark/pull/12855 I can reproduce the issue that @jurriaan reports on 1.6.0 and on 1.5.2. The issue does not occur on 1.3.1. I have added a comment to the JIRA issue with more detailed instructions how to reproduce: https://issues.apache.org/jira/browse/SPARK-15393 Note that this might mean that this PR did not cause the issue. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
