Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/20013
thanks, merging to master/2.3!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20013
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85924/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20013
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20013
**[Test build #85924 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85924/testReport)**
for PR 20013 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20013
**[Test build #85924 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85924/testReport)**
for PR 20013 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20013
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85899/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20013
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20013
**[Test build #85899 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85899/testReport)**
for PR 20013 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20013
**[Test build #85899 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85899/testReport)**
for PR 20013 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20013
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20013
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85867/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20013
**[Test build #85867 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85867/testReport)**
for PR 20013 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20013
**[Test build #85867 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85867/testReport)**
for PR 20013 at commit
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/20013
> the memory usage will be much larger with InMemoryStore.
No it won't. The in-memory store does not use extra memory for indices. It
sorts the data when the index is read, and discards that
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/20013
The major concern is that with these code changes, the memory usage will
be much larger with `InMemoryStore`.
Also building so many new indexes just for getting `computedQuantiles`,
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20013
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20013
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85809/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20013
**[Test build #85809 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85809/testReport)**
for PR 20013 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20013
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20013
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85806/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20013
**[Test build #85806 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85806/testReport)**
for PR 20013 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20013
**[Test build #85809 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85809/testReport)**
for PR 20013 at commit
Github user sameeragarwal commented on the issue:
https://github.com/apache/spark/pull/20013
@vanzin Thanks for the great work! Given this is a sizable change, I'll
like get this in before cutting RC1.
---
-
To
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20013
**[Test build #85806 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85806/testReport)**
for PR 20013 at commit
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/20013
> I suggest running more tests(real workloads)
I've tested all of these PRs attached to SPARK-18085 with lots of real and
synthetic app logs.
---
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/20013
Tried run with event logs of TPCDS benchmark(scale 1) on my local setup,
the UI works.
The leveldb size is about 50% larger.
---
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/20013
The code looks good. But it is a lot of changes in SHS, I suggest running
more tests(real workloads) before merge.
---
-
Github user squito commented on the issue:
https://github.com/apache/spark/pull/20013
lgtm
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20013
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20013
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85737/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20013
**[Test build #85737 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85737/testReport)**
for PR 20013 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20013
**[Test build #85737 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85737/testReport)**
for PR 20013 at commit
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/20013
> I wonder which changes double the disk usage?
It's the new indices, more explicitly the values, not the keys. I tried
changing the disk layout to write all the indices in a new namespace
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/20013
@vanzin I haven't run the code. I wonder which changes double the disk
usage? The new indices or the cached quantiles?
---
Github user ajbozarth commented on the issue:
https://github.com/apache/spark/pull/20013
I did a read through, I didn't see any issues from a code standpoint but I
haven't spent much time in your new code so another pair of eyes would be good.
I also didn't check out and run the code
Github user ajbozarth commented on the issue:
https://github.com/apache/spark/pull/20013
I'll take a look at this either today or tomorrow
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/20013
@squito
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20013
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85614/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20013
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20013
**[Test build #85614 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85614/testReport)**
for PR 20013 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20013
**[Test build #85614 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85614/testReport)**
for PR 20013 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20013
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85289/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20013
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20013
**[Test build #85289 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85289/testReport)**
for PR 20013 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20013
**[Test build #85289 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85289/testReport)**
for PR 20013 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20013
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20013
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85073/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20013
**[Test build #85073 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85073/testReport)**
for PR 20013 at commit
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/20013
I did some non-scientific test locally and this change causes the disk
store for an app to grow to about double its previous size; while not great,
given how much new data is being written, that's
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20013
**[Test build #85073 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85073/testReport)**
for PR 20013 at commit
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/20013
Some previous comments for this PR can be found at:
https://github.com/vanzin/spark/pull/41
---
-
To unsubscribe, e-mail:
51 matches
Mail list logo
