Github user advancedxy commented on the pull request:
https://github.com/apache/spark/pull/4783#issuecomment-77671093
@shivaram I update the comment, please review again. And still unrelated
test failure.
---
If your project is set up for it, you can reply to this email and have your
Github user advancedxy commented on a diff in the pull request:
https://github.com/apache/spark/pull/4783#discussion_r25992824
--- Diff: core/src/main/scala/org/apache/spark/util/SizeEstimator.scala ---
@@ -49,6 +49,11 @@ private[spark] object SizeEstimator extends Logging {
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4783#issuecomment-77670754
[Test build #28363 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28363/consoleFull)
for PR 4783 at commit
[`cb0af95`](https://githu
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4783#issuecomment-77625383
[Test build #28351 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28351/consoleFull)
for PR 4783 at commit
[`a55f3e1`](https://gith
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4783#issuecomment-77625394
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/28
Github user shivaram commented on a diff in the pull request:
https://github.com/apache/spark/pull/4783#discussion_r25969952
--- Diff: core/src/main/scala/org/apache/spark/util/SizeEstimator.scala ---
@@ -257,21 +262,56 @@ private[spark] object SizeEstimator extends Logging {
Github user shivaram commented on a diff in the pull request:
https://github.com/apache/spark/pull/4783#discussion_r25969836
--- Diff: core/src/main/scala/org/apache/spark/util/SizeEstimator.scala ---
@@ -283,4 +323,7 @@ private[spark] object SizeEstimator extends Logging {
Github user shivaram commented on a diff in the pull request:
https://github.com/apache/spark/pull/4783#discussion_r25969730
--- Diff: core/src/main/scala/org/apache/spark/util/SizeEstimator.scala ---
@@ -257,21 +262,56 @@ private[spark] object SizeEstimator extends Logging {
Github user shivaram commented on a diff in the pull request:
https://github.com/apache/spark/pull/4783#discussion_r25969639
--- Diff: core/src/main/scala/org/apache/spark/util/SizeEstimator.scala ---
@@ -257,21 +262,56 @@ private[spark] object SizeEstimator extends Logging {
Github user shivaram commented on a diff in the pull request:
https://github.com/apache/spark/pull/4783#discussion_r25969626
--- Diff: core/src/main/scala/org/apache/spark/util/SizeEstimator.scala ---
@@ -257,21 +262,56 @@ private[spark] object SizeEstimator extends Logging {
Github user shivaram commented on a diff in the pull request:
https://github.com/apache/spark/pull/4783#discussion_r25969222
--- Diff: core/src/main/scala/org/apache/spark/util/SizeEstimator.scala ---
@@ -257,21 +262,56 @@ private[spark] object SizeEstimator extends Logging {
Github user shivaram commented on a diff in the pull request:
https://github.com/apache/spark/pull/4783#discussion_r25969340
--- Diff: core/src/main/scala/org/apache/spark/util/SizeEstimator.scala ---
@@ -257,21 +262,56 @@ private[spark] object SizeEstimator extends Logging {
Github user shivaram commented on a diff in the pull request:
https://github.com/apache/spark/pull/4783#discussion_r25968891
--- Diff: core/src/main/scala/org/apache/spark/util/SizeEstimator.scala ---
@@ -49,6 +49,11 @@ private[spark] object SizeEstimator extends Logging {
pr
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4783#issuecomment-77615616
[Test build #28351 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28351/consoleFull)
for PR 4783 at commit
[`a55f3e1`](https://githu
Github user shivaram commented on the pull request:
https://github.com/apache/spark/pull/4783#issuecomment-77615372
Jenkins, retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4783#issuecomment-77312744
[Test build #28277 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28277/consoleFull)
for PR 4783 at commit
[`a55f3e1`](https://gith
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4783#issuecomment-77312747
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/28
Github user advancedxy commented on the pull request:
https://github.com/apache/spark/pull/4783#issuecomment-77308814
@shivaram, I add a bunch of comments in the code. I think it's time for you
to review it now. And the previous failures are not related to this code
change. Wonder why
Github user advancedxy commented on a diff in the pull request:
https://github.com/apache/spark/pull/4783#discussion_r25840258
--- Diff: core/src/main/scala/org/apache/spark/util/SizeEstimator.scala ---
@@ -283,4 +323,7 @@ private[spark] object SizeEstimator extends Logging {
Github user advancedxy commented on a diff in the pull request:
https://github.com/apache/spark/pull/4783#discussion_r25840206
--- Diff: core/src/main/scala/org/apache/spark/util/SizeEstimator.scala ---
@@ -257,21 +262,56 @@ private[spark] object SizeEstimator extends Logging {
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4783#issuecomment-77308110
[Test build #28277 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28277/consoleFull)
for PR 4783 at commit
[`a55f3e1`](https://githu
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4783#issuecomment-77135811
[Test build #28259 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28259/consoleFull)
for PR 4783 at commit
[`4b8aa90`](https://githu
Github user advancedxy commented on the pull request:
https://github.com/apache/spark/pull/4783#issuecomment-76352556
@shivaram I updated the gist, you can look at the result.
[gist](https://gist.github.com/advancedxy/2ae7c9cc7629f3aeb679)
Also you can just download the shell
Github user shivaram commented on the pull request:
https://github.com/apache/spark/pull/4783#issuecomment-76262495
@advancedxy -- Yeah it sounds good to me to use 4-byte shell size alignment
without CompressedOops and 8 bytes otherwise (we can wait to see if there are
other opinions)
Github user advancedxy commented on the pull request:
https://github.com/apache/spark/pull/4783#issuecomment-76190779
@shivaram -- I followed the
[JDK-8024912](https://bugs.openjdk.java.net/browse/JDK-8024912),
[JDK-8024913](https://bugs.openjdk.java.net/browse/JDK-8024913) and
[JDK-
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4783#issuecomment-76140128
[Test build #27992 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27992/consoleFull)
for PR 4783 at commit
[`8a127dc`](https://gith
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4783#issuecomment-76140139
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/27
Github user advancedxy commented on the pull request:
https://github.com/apache/spark/pull/4783#issuecomment-76139869
Ok, Thanks for the update. I will look into the jol and the example when I
get some spare time.
---
If your project is set up for it, you can reply to this email and
Github user shivaram commented on the pull request:
https://github.com/apache/spark/pull/4783#issuecomment-76138824
@advancedxy -- Thanks for the detailed writeup and the gist. I can see the
problem but I am not sure this is the complete fix. You are right that
shellSizes don't need
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4783#issuecomment-76135015
[Test build #27992 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27992/consoleFull)
for PR 4783 at commit
[`8a127dc`](https://githu
Github user shivaram commented on the pull request:
https://github.com/apache/spark/pull/4783#issuecomment-76134709
Jenkins, this is ok to test
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4783#issuecomment-76131588
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your pro
Github user advancedxy commented on a diff in the pull request:
https://github.com/apache/spark/pull/4783#discussion_r25408258
--- Diff:
core/src/test/scala/org/apache/spark/util/SizeEstimatorSuite.scala ---
@@ -59,6 +59,13 @@ class SizeEstimatorSuite
assertResult(24)(Size
GitHub user advancedxy opened a pull request:
https://github.com/apache/spark/pull/4783
[SPARK-6030][CORE] Don't alignSize the computed shellSize for classes in
the getClassInfo method
SizeEstimator gives wrong result for Integer on 64bit JVM with
UseCompressedOops on, this pr fixe
34 matches
Mail list logo
