Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/11956
@kiszk Is this still the issue? I knew you are working on the related part
now.
---
-
To unsubscribe, e-mail: reviews-unsubsc
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/11956
Thank you for pointing it out. #18747 implemented this feature.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/11956
@davies, could you please take a look at this since Spark 2.0.0 has been
successfully released?
cc: @rxin
---
If your project is set up for it, you can reply to this email and have your
reply app
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/11956
@davies, I hope that you have some bandwidth to review PRs. Could you
please review this, too?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub a
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/11956
@davies could you please review this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user davies commented on the issue:
https://github.com/apache/spark/pull/11956
@kiszk I'm sorry that I do not have the bandwidth to review this,
https://github.com/apache/spark/pull/13899/files sounds like an easier approach
(have not looked into the details), how do you think
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/11956
@davies, thank you for your comment. I hope that you will have bandwidth
soon since Spark 2.0 was released.
[this PR](https://github.com/apache/spark/pull/13899/files) does the same
thing. In par
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/11956
@davies I would appreciate it if you would make comments on my questions to
share how we implement a part of
[columnar](https://issues.apache.org/jira/browse/SPARK-15687).
---
If your project is set
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/11956
@davies Would it be possible to share your opinions regarding these design
questions among us?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub a
Github user a-roberts commented on the issue:
https://github.com/apache/spark/pull/11956
@hvanhovell @marmbrus @srowen I see this PR has been open since the 25th of
March and provides substantial performance improvements as mentioned above
without introducing functional regressions, a
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/11956
I'm not qualified to comment as I tend to ignore SQL unless it's a simple
and easy to evaluate change.
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/11956
Current PR #13899 does not support a case that an element is null.
For now, I am working for the followings:
1. Support all of types
2. Make structures serializable
3. Support null cases
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/11956
**[Test build #73611 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73611/consoleFull)**
for PR 11956 at commit
[`fea9f34`](https://github.com/apache/spark/commit/f
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/11956
**[Test build #73611 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73611/consoleFull)**
for PR 11956 at commit
[`fea9f34`](https://github.com/apache/spark/commit/
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/11956
@davies Could you please share your great opinions regarding these design
questions among our community while we know you are busy?
---
If your project is set up for it, you can reply to this email a
Github user davies commented on the issue:
https://github.com/apache/spark/pull/11956
@kiszk The current implementation use ByteBuffer and smart compression
algorithms, it too slow to build the in-memory cache, make it useless. So we'd
like to improve the performance of building phase
Github user davies commented on the issue:
https://github.com/apache/spark/pull/11956
cc @rxin
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the f
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/11956
@davies Thank you for your reply. I understand the future direction. I will
implement this direction by using this PR or another PR. While the future
roadmap may consume more memory, the performance w
Github user davies commented on the issue:
https://github.com/apache/spark/pull/11956
We could compress them in memory with MEMORY_AND_DISK_SER, this could be
controlled by a flag.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/11956
Thank you for your clarification.
[Here](https://trongkhoanguyenblog.wordpress.com/2014/12/14/understand-the-storage-module-in-spark-core/)
is a good summary for me.
We may add property like `spar
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/11956
@davies, could you please rerun Jenkins for PR
[#13899](https://github.com/apache/spark/pull/13899)? I would like to know
which tests were failed by the current implementation.
---
If your project
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/11956
I saw PR #13899. I understood there are two design points for now.
Hopefully, no more points :)
1. `CachedBatch` must be serializable.
2. According to my experiments, we should make a class for
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/11956
**[Test build #71153 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71153/consoleFull)**
for PR 11956 at commit
[`fea9f34`](https://github.com/apache/spark/commit/f
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/11956
**[Test build #71153 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71153/consoleFull)**
for PR 11956 at commit
[`fea9f34`](https://github.com/apache/spark/commit/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/11956
Build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/11956
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71153/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/11956
**[Test build #61460 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61460/consoleFull)**
for PR 11956 at commit
[`697890c`](https://github.com/apache/spark/commit/6
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/11956
**[Test build #61460 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61460/consoleFull)**
for PR 11956 at commit
[`697890c`](https://github.com/apache/spark/commit/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/11956
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61460/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/11956
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/11956
@davies Would it be possible to review this if you have bandwidth for this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your proj
Github user a-roberts commented on the issue:
https://github.com/apache/spark/pull/11956
@robbinspg and I are evaluating this from a functional and performance
perspective, full disclosure: we both work for IBM with @kiszk.
All unit tests pass including the new ones Ishizaki h
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/11956
**[Test build #61915 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61915/consoleFull)**
for PR 11956 at commit
[`d035c42`](https://github.com/apache/spark/commit/d
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/11956
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61915/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/11956
**[Test build #61915 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61915/consoleFull)**
for PR 11956 at commit
[`d035c42`](https://github.com/apache/spark/commit/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/11956
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/11956
**[Test build #61919 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61919/consoleFull)**
for PR 11956 at commit
[`54aabef`](https://github.com/apache/spark/commit/5
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/11956
**[Test build #61919 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61919/consoleFull)**
for PR 11956 at commit
[`54aabef`](https://github.com/apache/spark/commit/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/11956
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61919/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/11956
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/11956
**[Test build #62019 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62019/consoleFull)**
for PR 11956 at commit
[`fea9f34`](https://github.com/apache/spark/commit/f
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/11956
**[Test build #62019 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62019/consoleFull)**
for PR 11956 at commit
[`fea9f34`](https://github.com/apache/spark/commit/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/11956
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/11956
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62019/
Test PASSed.
---
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/11956
Hi @davies could you please take a look at this since Spark 2.0.0 has been
successfully released?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHu
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/11956
@rxin, could you please review this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/11956
**[Test build #60285 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60285/consoleFull)**
for PR 11956 at commit
[`c30d6a1`](https://github.com/apache/spark/commit/c
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/11956
**[Test build #60285 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60285/consoleFull)**
for PR 11956 at commit
[`c30d6a1`](https://github.com/apache/spark/commit/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/11956
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60285/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/11956
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
50 matches
Mail list logo
