Github user squito commented on the issue:
https://github.com/apache/spark/pull/23058
merged to master, thanks @wypoon
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-ma
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23058
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional comma
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23058
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99420/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23058
**[Test build #99420 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99420/testReport)**
for PR 23058 at commit
[`86ba18c`](https://github.com/apache/spark/commit/8
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23058
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/5491/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23058
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional comma
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23058
**[Test build #99420 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99420/testReport)**
for PR 23058 at commit
[`86ba18c`](https://github.com/apache/spark/commit/86
Github user squito commented on the issue:
https://github.com/apache/spark/pull/23058
> causes any performance degradation compared to memory mapping
@ankuriitg good question, though if you look at what the old code was
doing, it wasn't memory mapping the file, it was reading
Github user ankuriitg commented on the issue:
https://github.com/apache/spark/pull/23058
The change looks good to me. I understand that this change uses memory
efficiently but I am wondering whether it causes any performance degradation
compared to memory mapping. If yes, can we measu
Github user squito commented on the issue:
https://github.com/apache/spark/pull/23058
@mridulm @jerryshao @Ngone51 @vanzin just checking if you want to look at
this before I merge, will leave open a bit.
---
-
To un
Github user squito commented on the issue:
https://github.com/apache/spark/pull/23058
@attilapiros yes, something like that would be possible. I was thinking
you'd just use the existing serializer methods to do it, soemthing like:
```scala
val buffer = getRemoteManagedBuf
Github user wypoon commented on the issue:
https://github.com/apache/spark/pull/23058
Thanks @squito. I updated the testing section of the PR.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For ad
Github user squito commented on the issue:
https://github.com/apache/spark/pull/23058
lgtm
I looked more into the lifecycle of the buffers and when they get
`disposed`, and it looks fine to me. (In fact I think there is no need for the
`dispose` in the first place, as hinted
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23058
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional comma
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23058
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99091/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23058
**[Test build #99091 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99091/testReport)**
for PR 23058 at commit
[`125d746`](https://github.com/apache/spark/commit/1
Github user wypoon commented on the issue:
https://github.com/apache/spark/pull/23058
> can we also make the same change to `TaskResultGetter`?
I had a conversation off-line with Imran. As we end up deserializing the
value of the task result into a ByteBuffer anyway, this chan
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23058
**[Test build #99091 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99091/testReport)**
for PR 23058 at commit
[`125d746`](https://github.com/apache/spark/commit/12
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23058
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional comma
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23058
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/5203/
Github user squito commented on the issue:
https://github.com/apache/spark/pull/23058
can we also make the same change to `TaskResultGetter`? We could avoid the
same inefficiency for large task results that also get fetched to disk. (That
would actually get us closer to removing the
Github user squito commented on the issue:
https://github.com/apache/spark/pull/23058
@attilapiros can you review this please?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional command
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23058
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional comma
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23058
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98923/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23058
**[Test build #98923 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98923/testReport)**
for PR 23058 at commit
[`2516ec6`](https://github.com/apache/spark/commit/2
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23058
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional comma
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23058
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/5083/
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23058
**[Test build #98923 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98923/testReport)**
for PR 23058 at commit
[`2516ec6`](https://github.com/apache/spark/commit/25
28 matches
Mail list logo