Github user squito commented on the issue:
https://github.com/apache/spark/pull/23058
merged to master, thanks @wypoon
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23058
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23058
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99420/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23058
**[Test build #99420 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99420/testReport)**
for PR 23058 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23058
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23058
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23058
**[Test build #99420 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99420/testReport)**
for PR 23058 at commit
Github user squito commented on the issue:
https://github.com/apache/spark/pull/23058
> causes any performance degradation compared to memory mapping
@ankuriitg good question, though if you look at what the old code was
doing, it wasn't memory mapping the file, it was reading
Github user ankuriitg commented on the issue:
https://github.com/apache/spark/pull/23058
The change looks good to me. I understand that this change uses memory
efficiently but I am wondering whether it causes any performance degradation
compared to memory mapping. If yes, can we
Github user squito commented on the issue:
https://github.com/apache/spark/pull/23058
@mridulm @jerryshao @Ngone51 @vanzin just checking if you want to look at
this before I merge, will leave open a bit.
---
-
To
Github user squito commented on the issue:
https://github.com/apache/spark/pull/23058
@attilapiros yes, something like that would be possible. I was thinking
you'd just use the existing serializer methods to do it, soemthing like:
```scala
val buffer =
Github user wypoon commented on the issue:
https://github.com/apache/spark/pull/23058
Thanks @squito. I updated the testing section of the PR.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user squito commented on the issue:
https://github.com/apache/spark/pull/23058
lgtm
I looked more into the lifecycle of the buffers and when they get
`disposed`, and it looks fine to me. (In fact I think there is no need for the
`dispose` in the first place, as
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23058
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23058
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99091/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23058
**[Test build #99091 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99091/testReport)**
for PR 23058 at commit
Github user wypoon commented on the issue:
https://github.com/apache/spark/pull/23058
> can we also make the same change to `TaskResultGetter`?
I had a conversation off-line with Imran. As we end up deserializing the
value of the task result into a ByteBuffer anyway, this
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23058
**[Test build #99091 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99091/testReport)**
for PR 23058 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23058
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23058
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user squito commented on the issue:
https://github.com/apache/spark/pull/23058
can we also make the same change to `TaskResultGetter`? We could avoid the
same inefficiency for large task results that also get fetched to disk. (That
would actually get us closer to removing
Github user squito commented on the issue:
https://github.com/apache/spark/pull/23058
@attilapiros can you review this please?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23058
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23058
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98923/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23058
**[Test build #98923 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98923/testReport)**
for PR 23058 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23058
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23058
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23058
**[Test build #98923 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98923/testReport)**
for PR 23058 at commit
28 matches
Mail list logo
