Github user MaxGekk commented on the issue:
https://github.com/apache/spark/pull/21296
> ... does this push down the column selection into the reading of data
Yes, it does.
---
-
To unsubscribe, e-mail:
Github user koertkuipers commented on the issue:
https://github.com/apache/spark/pull/21296
if i do not select a schema (and i use inferSchema), and i do a select for
only a few column, does this push down the column selection into the reading of
data (for schema inference and for
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21296
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21296
This PR breaks the tests. Let me revert it. Please resubmit the PR with the
fix. Thanks!
---
-
To unsubscribe, e-mail:
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/21296
thanks, merging to master!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user MaxGekk commented on the issue:
https://github.com/apache/spark/pull/21296
I added the word `parser` to the feature name because as @HyukjinKwon wrote
above we do pruning in type conversion already. This PR enables column pruning
by CSV parser only.
---
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21296
Just for clarification, I'm okay.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/21296
LGTM except a minor comment about the config name
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user MaxGekk commented on the issue:
https://github.com/apache/spark/pull/21296
@cloud-fan @HyukjinKwon Could you look at the PR, please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21296
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90797/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21296
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21296
**[Test build #90797 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90797/testReport)**
for PR 21296 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21296
**[Test build #90797 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90797/testReport)**
for PR 21296 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21296
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21296
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90752/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21296
**[Test build #90752 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90752/testReport)**
for PR 21296 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21296
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90751/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21296
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21296
**[Test build #90751 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90751/testReport)**
for PR 21296 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21296
**[Test build #90752 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90752/testReport)**
for PR 21296 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21296
**[Test build #90751 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90751/testReport)**
for PR 21296 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21296
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21296
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90555/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21296
**[Test build #90555 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90555/testReport)**
for PR 21296 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21296
**[Test build #90555 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90555/testReport)**
for PR 21296 at commit
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21296
Can we update the migration guide then? I want to see if the note makes
sense.
---
-
To unsubscribe, e-mail:
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/21296
I'm fine with breaking the CSV behavior of malformed input a little, as
long as we documented it well in the migration guide. cc @gatorsmile
---
27 matches
Mail list logo
