Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/14627
@rxin, it does not fix any bug but just gets rid of duplicated logics. I
will try to open a separate JIRA in this case in the future to prevent
confusion. Thank you/
---
If your project is set
Github user rxin commented on the issue:
https://github.com/apache/spark/pull/14627
Actually there is a conflict. Does this fix any bug? If not we don't need
to merge it in 2.1.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as
Github user rxin commented on the issue:
https://github.com/apache/spark/pull/14627
Thanks - merging in master/branch-2.1.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enable
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14627
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70520/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14627
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14627
**[Test build #70520 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70520/testReport)**
for PR 14627 at commit
[`47c835d`](https://github.com/apache/spark/commit/4
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14627
**[Test build #70520 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70520/testReport)**
for PR 14627 at commit
[`47c835d`](https://github.com/apache/spark/commit/47
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14627
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68887/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14627
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14627
**[Test build #68887 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68887/consoleFull)**
for PR 14627 at commit
[`b68ce0c`](https://github.com/apache/spark/commit/
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14627
**[Test build #68887 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68887/consoleFull)**
for PR 14627 at commit
[`b68ce0c`](https://github.com/apache/spark/commit/b
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/14627
(I minimised the changes here to make the review easier)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/14627
(gentle ping @liancheng)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wi
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/14627
@liancheng If you are uncertain of `OrcFileOperator.scala` I will
definitely remove this in this PR.
---
If your project is set up for it, you can reply to this email and have your
reply appear
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14627
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14627
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67027/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14627
**[Test build #67027 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67027/consoleFull)**
for PR 14627 at commit
[`bd14038`](https://github.com/apache/spark/commit/
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14627
**[Test build #67027 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67027/consoleFull)**
for PR 14627 at commit
[`bd14038`](https://github.com/apache/spark/commit/b
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/14627
gentle ping @liancheng
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wis
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/14627
This PR removes duplicated logics which I guess is not a safe-guard because
they are inconsistent. I would appreciate if you both @rxin and @liancheng take
a look please.
---
If your project i
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/14627
Could you take a look please @rxin and @liancheng ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14627
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14627
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63714/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14627
**[Test build #63714 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63714/consoleFull)**
for PR 14627 at commit
[`3fa597c`](https://github.com/apache/spark/commit/
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/14627
Ah, yes thank you!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes s
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/14627
Only committers know what is needed at Spark. We are just contributors to
send a pull request as a proposal. You can do anything in your PR. I'm not
against you or this PR. Good luck! :)
---
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/14627
BTW, I am waiting for the Jenkins tests before cc someone. However, as you
are already in here (I appreciate it), I appreciate it if you take a look.
---
If your project is set up for it, you c
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/14627
Yes, it is not. I am sorry for the confusion. It seems your fix is
perfectly fine and correct but it is just a clean-up to get rid of duplicated
logics.
BTW, I am also okay with removin
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/14627
Then, it's not about fixing new or remaining bugs of the previous one
(SPARK-16975). I see!
---
If your project is set up for it, you can reply to this email and have your
reply appear on Git
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/14627
@dongjoon-hyun Thanks for taking a look! Actually, I intended to test your
fix for all data sources just to make sure the issue in `SPARK-16975` is
resolved for all.
As it is a clean-up
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/14627
I am happy to add some more tests but I am a bit confused of what I should
test. Maybe a test for `HadoopFsRelation.listLeafFiles` to make sure
directories are not included for the return value?
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/14627
Hi, @HyukjinKwon .
You test code seems to pass without your code. Could you confirm that first?
---
If your project is set up for it, you can reply to this email and have your
reply appea
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14627
**[Test build #63714 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63714/consoleFull)**
for PR 14627 at commit
[`3fa597c`](https://github.com/apache/spark/commit/3
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/14627
FYI, in most data sources, it would try to read duplicately or fail to
infer the schema (haven't tested this yet) if directories are allowed in
`FileFormat.inferSchema`.
---
If your project i
34 matches
Mail list logo
