Github user ajacques commented on the issue:
https://github.com/apache/spark/pull/21889
Thanks for the response all. @mailman If it's really your preference, I
will create a PR against that branch and close this one. My intention was never
to take away from your efforts, and I still
Github user mallman commented on the issue:
https://github.com/apache/spark/pull/21889
Essentially, this PR was created to take the management of #21320 out of my
hands, with a view towards facilitating its incorporation into Spark 2.4. It
was my suggestion, one based in frustration.
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21889
> I've only taken it as a based to make stylistic changes based on the
code review to help move things along.
This PR doesn't only include stylistic changes. Since stylistic changes do
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21889
**[Test build #4278 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4278/testReport)**
for PR 21889 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21889
**[Test build #4278 has
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4278/testReport)**
for PR 21889 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21889
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21889
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94805/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21889
**[Test build #94805 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94805/testReport)**
for PR 21889 at commit
Github user mallman commented on the issue:
https://github.com/apache/spark/pull/21889
> Due to the urgency of the upcoming 2.4 code freeze, I'm going to open
this PR to collect any feedback. This can be closed if you prefer to continue
to the work in the original PR.
That
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21889
**[Test build #94805 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94805/testReport)**
for PR 21889 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21889
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21889
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94790/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21889
**[Test build #94790 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94790/testReport)**
for PR 21889 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21889
**[Test build #94790 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94790/testReport)**
for PR 21889 at commit
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21889
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21889
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94785/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21889
**[Test build #94785 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94785/testReport)**
for PR 21889 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21889
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21889
**[Test build #94785 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94785/testReport)**
for PR 21889 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21889
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94731/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21889
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21889
**[Test build #94731 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94731/testReport)**
for PR 21889 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21889
**[Test build #94731 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94731/testReport)**
for PR 21889 at commit
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21889
ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user mallman commented on the issue:
https://github.com/apache/spark/pull/21889
@ajacques I added a commit to enable schema pruning by default. It's a
little more complete than your commit to do the same. Please rebase off my
branch and remove your commit.
---
Github user mallman commented on the issue:
https://github.com/apache/spark/pull/21889
>> @mallman, while we wait for the go-no-go, do you have the changes for
the next PR ready? Is there anything you need help with?
> I have the hack I used originally, but I haven't tried
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21889
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94536/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21889
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21889
**[Test build #94536 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94536/testReport)**
for PR 21889 at commit
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21889
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21889
**[Test build #94536 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94536/testReport)**
for PR 21889 at commit
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21889
From a cursory look, the last failure looks unrelated.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21889
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user ajacques commented on the issue:
https://github.com/apache/spark/pull/21889
@gatorsmile Do you think there is a on deterministic failure in this change
that causes it to inconsistently fail?
---
-
To
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21889
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21889
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94499/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21889
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21889
**[Test build #94499 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94499/testReport)**
for PR 21889 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21889
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94503/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21889
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21889
**[Test build #94503 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94503/testReport)**
for PR 21889 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21889
**[Test build #94503 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94503/testReport)**
for PR 21889 at commit
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21889
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21889
**[Test build #94499 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94499/testReport)**
for PR 21889 at commit
Github user mallman commented on the issue:
https://github.com/apache/spark/pull/21889
> @mallman, while we wait for the go-no-go, do you have the changes for the
next PR ready? Is there anything you need help with?
I have the hack I used originally, but I haven't tried
Github user mallman commented on the issue:
https://github.com/apache/spark/pull/21889
@ajacques Please rebase off my branch.
@gatorsmile I don't recall seeing that error before. Any idea for how I can
reproduce and debug?
---
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21889
I hit the following error in my local environment.
```
sbt.ForkMain$ForkError: org.apache.spark.SparkException: Job aborted due to
stage failure: Task 0 in stage 220.0 failed 1 times,
Github user ajacques commented on the issue:
https://github.com/apache/spark/pull/21889
@mallman, while we wait for the go-no-go, do you have the changes for the
next PR ready? Is there anything you need help with?
---
Github user mallman commented on the issue:
https://github.com/apache/spark/pull/21889
Are we waiting for @gatorsmile's go-ahead and merge?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21889
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94409/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21889
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21889
**[Test build #94409 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94409/testReport)**
for PR 21889 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21889
**[Test build #94409 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94409/testReport)**
for PR 21889 at commit
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21889
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21889
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94408/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21889
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21889
**[Test build #94408 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94408/testReport)**
for PR 21889 at commit
Github user mallman commented on the issue:
https://github.com/apache/spark/pull/21889
> just for clarification, so now .. there no outstanding bugs, some tests
are ignored per #21320 (comment) and left comments were mostly addressed. Did i
understand correctly?
The ignored
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21889
**[Test build #94408 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94408/testReport)**
for PR 21889 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21889
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94406/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21889
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21889
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21889
**[Test build #94406 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94406/testReport)**
for PR 21889 at commit
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21889
just for clarification, so now .. there no outstanding bugs, some tests are
ignored per https://github.com/apache/spark/pull/21320#issuecomment-406353694
and left comments were mostly
Github user mallman commented on the issue:
https://github.com/apache/spark/pull/21889
See https://github.com/apache/spark/pull/21320#issuecomment-406353694 for
@gatorsmile's request to move the changes to `ParquetReadSupport.scala` to
another PR.
There was another,
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21889
That comment is before
https://github.com/apache/spark/pull/21889#issuecomment-408330791. I am okay in
general but want to be clear if I'm ignoring his decision or not.
---
Github user ajacques commented on the issue:
https://github.com/apache/spark/pull/21889
>> but @gatorsmile wants to review it in a follow-on PR.
> Where did he say it after the comment above?
It was my interpretation of this comment:
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21889
> but @gatorsmile wants to review it in a follow-on PR.
I need a confirmation from @gatorsmile. I don't want to ignore his decision
here in
> Just FYI, we are unable to merge
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21889
**[Test build #94406 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94406/testReport)**
for PR 21889 at commit
Github user ajacques commented on the issue:
https://github.com/apache/spark/pull/21889
@HyukjinKwon Looks like most of your comments have been already addressed,
but I've gone ahead and made a few more tweaks to help this get merged. Please
let me know if any blocking comments have
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21889
> but @gatorsmile wants to review it in a follow-on PR.
Where did he say it after the comment above?
Also why don't you address my comments if you're going to push more changes
Github user mallman commented on the issue:
https://github.com/apache/spark/pull/21889
> Assuming from #21889 (comment), we shouldn't have any identified bug
here. What kind of bugs left to be fixed?
That bug was address by b50ddb4. We still need to fix the bug underlying
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21889
Assuming from
https://github.com/apache/spark/pull/21889#issuecomment-408330791, we shouldn't
have any identified bug here. What kind of bugs left to be fixed?
---
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21889
Can we address the comments I left on that PR too? Looks that's the only
way to get through this? FWIW, since
https://github.com/apache/spark/commit/51bee7aca13451167fa3e701fcd60f023eae5e61
is
Github user ajacques commented on the issue:
https://github.com/apache/spark/pull/21889
Is there anything I can do to help with this PR?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user ajacques commented on the issue:
https://github.com/apache/spark/pull/21889
Jenkins build successful. Any PR comments/blockers to merge for phase 1?
cc @HyukjinKwon, @gatorsmile, @cloud-fan
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21889
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94252/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21889
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21889
**[Test build #94252 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94252/testReport)**
for PR 21889 at commit
Github user mallman commented on the issue:
https://github.com/apache/spark/pull/21889
> Alright to make sure we're all on the same page, it sounds like we're
ready to merge this PR pending:
>
> * Successful build by Jenkins
> * Any PR comments from a maintainer
>
Github user ajacques commented on the issue:
https://github.com/apache/spark/pull/21889
Alright to make sure we're all on the same page, it sounds like we're ready
to merge this PR pending:
* Successful build by Jenkins
* Any PR comments from a maintainer
This feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21889
**[Test build #94252 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94252/testReport)**
for PR 21889 at commit
Github user mallman commented on the issue:
https://github.com/apache/spark/pull/21889
> @mallman Is it related to this revert in ParquetReadSupport.scala? I
re-added this logic and all 32 tests in ParquetSchemaPruningSuite passed.
Yes. That's what we need to work on in the
Github user ajacques commented on the issue:
https://github.com/apache/spark/pull/21889
@mallman Is it related to [this revert in
ParquetReadSupport](https://github.com/apache/spark/pull/21889/commits/0312a5188f0d6c9fc5304195dbdc703bf0aa3fb7#diff-245e70c1f41e353e34cf29bd00fd9029L86).
Github user mallman commented on the issue:
https://github.com/apache/spark/pull/21889
I've pushed a commit to restore the original test coverage while also
ensuring determinism of the output. Don't ask me how I did it. It's a secret!
The test that was failing before it was
Github user mallman commented on the issue:
https://github.com/apache/spark/pull/21889
> select id, name.middle, address from temp - Works
> select name.middle, address from temp - Fails
> select name.middle from temp - Works
> select name.middle, id, address from temp -
Github user ajacques commented on the issue:
https://github.com/apache/spark/pull/21889
@mallman
`select id, name.middle, address from temp` - **Works**
`select name.middle, address from temp` - **Fails**
---
Github user mallman commented on the issue:
https://github.com/apache/spark/pull/21889
> Test build #94228 has finished for PR 21889 at commit 92901da.
The test failure appears to be unrelated to this PR.
Is it just me or has the test suite become flakier in the past
Github user mallman commented on the issue:
https://github.com/apache/spark/pull/21889
> The tests as committed pass for me, but I removed the order by id and I
got that error. Are you saying it works with the specific query in my comment?
@ajacques Please try this query:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21889
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21889
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94228/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21889
**[Test build #94228 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94228/testReport)**
for PR 21889 at commit
Github user mallman commented on the issue:
https://github.com/apache/spark/pull/21889
> The tests as committed pass for me, but I removed the order by id and I
got that error. Are you saying it works with the specific query in my comment?
Oh! I didn't notice you changed the
Github user ajacques commented on the issue:
https://github.com/apache/spark/pull/21889
The tests as committed pass for me, but I removed the `order by id` and I
got that error. Are you saying it works with the specific query in my comment?
---
Github user mallman commented on the issue:
https://github.com/apache/spark/pull/21889
> @mallman: I've rebased on top of your changes and pushed. I'm seeing the
following:
That test passes for me locally. Also, I inspected your branch and could
not find any errors in the
Github user mallman commented on the issue:
https://github.com/apache/spark/pull/21889
> @mallman: I've rebased on top of your changes and pushed. I'm seeing the
following
That's the test case that I "unignored". It was passing. There must be some
simple explanation. I will
Github user ajacques commented on the issue:
https://github.com/apache/spark/pull/21889
@mallman: I've rebased on top of your changes and pushed. I'm seeing the
following:
Given the following schema:
```
root
|-- id: integer (nullable = true)
|-- name:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21889
**[Test build #94228 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94228/testReport)**
for PR 21889 at commit
Github user ajacques commented on the issue:
https://github.com/apache/spark/pull/21889
@mallman: [This
one](https://github.com/apache/spark/pull/21889/files#diff-0c6c7481232e9637b91c179f1005426aR120)?
I just enabled it on my branch and the test passed. Was it fixed by your
latest
Github user mallman commented on the issue:
https://github.com/apache/spark/pull/21889
> Are there any other blockers to enabling this by default now that
@mallman fixed the currently known broken queries?
The functionality exercised by the ignored test in
1 - 100 of 161 matches
Mail list logo
