Github user HeartSaVioR commented on the issue:
https://github.com/apache/spark/pull/21733
Thanks all for reviewing and thanks @tdas for merging this in!
---
-
To unsubscribe, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21733
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21733
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95003/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21733
**[Test build #95003 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95003/testReport)**
for PR 21733 at commit
Github user tdas commented on the issue:
https://github.com/apache/spark/pull/21733
Good point. That can be minor Pr.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21733
**[Test build #95003 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95003/testReport)**
for PR 21733 at commit
Github user HeartSaVioR commented on the issue:
https://github.com/apache/spark/pull/21733
@tdas Removed the .crc files. Thanks for reviewing!
Btw, it might be good to find the way to add `.crc` files to `.gitignore`
if `.crc` files are mostly useless all the times. Might worth to
Github user tdas commented on the issue:
https://github.com/apache/spark/pull/21733
LGTM. Will merge when tests pass. :)
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user tdas commented on the issue:
https://github.com/apache/spark/pull/21733
This looks good!! Only one comment, please don't add the .crc files. They
are useless and adds unnecessarily clutter.
---
-
To
Github user HeartSaVioR commented on the issue:
https://github.com/apache/spark/pull/21733
@tdas Kindly reminder.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21733
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21733
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94474/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21733
**[Test build #94474 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94474/testReport)**
for PR 21733 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21733
**[Test build #94474 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94474/testReport)**
for PR 21733 at commit
Github user HeartSaVioR commented on the issue:
https://github.com/apache/spark/pull/21733
retest this, please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21733
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21733
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94469/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21733
**[Test build #94469 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94469/testReport)**
for PR 21733 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21733
**[Test build #94469 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94469/testReport)**
for PR 21733 at commit
Github user HeartSaVioR commented on the issue:
https://github.com/apache/spark/pull/21733
@tdas Addressed review comments. Please take a look again. Thanks in
advance.
---
-
To unsubscribe, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21733
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94403/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21733
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21733
**[Test build #94403 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94403/testReport)**
for PR 21733 at commit
Github user HeartSaVioR commented on the issue:
https://github.com/apache/spark/pull/21733
Also added javadoc as well. Most of contents are from StateStore but I
didn't copy the note to implementation for state store since it is duplicated.
Please let me know if we want to add
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21733
**[Test build #94403 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94403/testReport)**
for PR 21733 at commit
Github user HeartSaVioR commented on the issue:
https://github.com/apache/spark/pull/21733
@tdas
Done running perf. test with 4 more tests:
> BenchmarkMovingAggregationsListenerKeyMuchBigger
rate: 16
version | input rows per second | processed rows
Github user HeartSaVioR commented on the issue:
https://github.com/apache/spark/pull/21733
@tdas Kindly reminder. I'll take the doc step when you say it's OK to go.
---
-
To unsubscribe, e-mail:
Github user HeartSaVioR commented on the issue:
https://github.com/apache/spark/pull/21733
@tdas
I found the spare time to run performance tests though I've run only one
app for now... I couldn't run the tests concurrently. Please let me know if you
are not confident with
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21733
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93903/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21733
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21733
**[Test build #93903 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93903/testReport)**
for PR 21733 at commit
Github user HeartSaVioR commented on the issue:
https://github.com/apache/spark/pull/21733
@tdas
I've applied your review comments except documentation. (Will add WIP to
the PR's title if it sounds clearer) There may be something you can add the
review comments and so I'd like
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21733
**[Test build #93903 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93903/testReport)**
for PR 21733 at commit
Github user HeartSaVioR commented on the issue:
https://github.com/apache/spark/pull/21733
@tdas
Thanks for the detailed review! I'll follow up your comments and update the
patch.
Btw, If my memory is right, I tried out increasing "rate" while
benchmarking, but rate
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21733
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21733
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93757/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21733
**[Test build #93757 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93757/testReport)**
for PR 21733 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21733
**[Test build #93757 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93757/testReport)**
for PR 21733 at commit
Github user HeartSaVioR commented on the issue:
https://github.com/apache/spark/pull/21733
retest this, please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21733
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93324/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21733
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21733
**[Test build #93324 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93324/testReport)**
for PR 21733 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21733
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21733
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93315/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21733
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21733
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93318/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21733
**[Test build #93318 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93318/testReport)**
for PR 21733 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21733
**[Test build #93315 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93315/testReport)**
for PR 21733 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21733
**[Test build #93324 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93324/testReport)**
for PR 21733 at commit
Github user HeartSaVioR commented on the issue:
https://github.com/apache/spark/pull/21733
Add tests for StatefulOperatorsHelper itself as well. (Sorry for pushing
commits multiple times which trigger multiple builds. It might be ideal if
older test builds are terminated once newer
Github user HeartSaVioR commented on the issue:
https://github.com/apache/spark/pull/21733
Now I'd like to propose changing default behavior to apply new path but
keeping backward compatibility, so applied it to the patch. I'm still open on
decision to apply it as advanced option as
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21733
**[Test build #93315 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93315/testReport)**
for PR 21733 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21733
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21733
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93277/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21733
**[Test build #93277 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93277/testReport)**
for PR 21733 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21733
**[Test build #93277 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93277/testReport)**
for PR 21733 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21733
Build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21733
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21733
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93222/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21733
**[Test build #93222 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93222/testReport)**
for PR 21733 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21733
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93221/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21733
**[Test build #93221 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93221/testReport)**
for PR 21733 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21733
**[Test build #93222 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93222/testReport)**
for PR 21733 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21733
**[Test build #93221 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93221/testReport)**
for PR 21733 at commit
Github user HeartSaVioR commented on the issue:
https://github.com/apache/spark/pull/21733
@arunmahadevan @jose-torres
Github user HeartSaVioR commented on the issue:
https://github.com/apache/spark/pull/21733
I guess we would have to treat reducing state memory size to have worth to
do: as described in above commit, we already optimized in
HDFSBackedStateStoreProvider for reducing state store disk
Github user jose-torres commented on the issue:
https://github.com/apache/spark/pull/21733
We could still save the value of the option to offsetSeqMetadata and error
if it's changed. The value of using an option would just be that there's no
global default; a poweruser can set the
Github user HeartSaVioR commented on the issue:
https://github.com/apache/spark/pull/21733
@arunmahadevan
I'm actually in favor of changing default behavior, just not 100% sure the
result would be promising for exhaustive use cases. I might need to prepare
more kinds of
Github user arunmahadevan commented on the issue:
https://github.com/apache/spark/pull/21733
@HeartSaVioR , the results looks promising. I am wondering if theres a way
to make this default option than introducing new configs. Since this is
internal details anyway theres no need to
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21733
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92791/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21733
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21733
**[Test build #92791 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92791/testReport)**
for PR 21733 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21733
**[Test build #92791 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92791/testReport)**
for PR 21733 at commit
Github user HeartSaVioR commented on the issue:
https://github.com/apache/spark/pull/21733
retest this, please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21733
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92755/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21733
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21733
**[Test build #92755 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92755/testReport)**
for PR 21733 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21733
**[Test build #92755 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92755/testReport)**
for PR 21733 at commit
Github user HeartSaVioR commented on the issue:
https://github.com/apache/spark/pull/21733
retest this, please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21733
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21733
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92738/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21733
**[Test build #92738 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92738/testReport)**
for PR 21733 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21733
**[Test build #92738 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92738/testReport)**
for PR 21733 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21733
**[Test build #92735 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92735/testReport)**
for PR 21733 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21733
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92735/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21733
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21733
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21733
**[Test build #92735 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92735/testReport)**
for PR 21733 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21733
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92734/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21733
**[Test build #92734 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92734/testReport)**
for PR 21733 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21733
**[Test build #92734 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92734/testReport)**
for PR 21733 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21733
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user HeartSaVioR commented on the issue:
https://github.com/apache/spark/pull/21733
cc. @tdas @zsxwing @jose-torres @jerryshao @arunmahadevan @HyukjinKwon
---
-
To unsubscribe, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21733
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
94 matches
Mail list logo
