[GitHub] spark pull request: [SPARK-7734][DataFrame] support struct type in...

2015-05-25 Thread chenghao-intel
Github user chenghao-intel commented on the pull request: https://github.com/apache/spark/pull/6266#issuecomment-105137973 `explode` only takes the `ArrayType` or `MapType` as child type. Not for `StructType`, See

[GitHub] spark pull request: SPARK-4550. In sort-based shuffle, store map o...

2015-05-25 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/4450#discussion_r30963780 --- Diff: core/src/main/scala/org/apache/spark/util/collection/PartitionedSerializedPairBuffer.scala --- @@ -0,0 +1,254 @@ +/* + * Licensed to the

[GitHub] spark pull request: SPARK-4550. In sort-based shuffle, store map o...

2015-05-25 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/4450#discussion_r30964306 --- Diff: core/src/main/scala/org/apache/spark/util/collection/PartitionedSerializedPairBuffer.scala --- @@ -0,0 +1,254 @@ +/* + * Licensed to the

[GitHub] spark pull request: [Spark-7780][MLLIB] Intercept in logisticregre...

2015-05-25 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/6386#discussion_r30964263 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/classification/LogisticRegression.scala --- @@ -363,4 +370,34 @@ class LogisticRegressionWithLBFGS

[GitHub] spark pull request: SPARK-4550. In sort-based shuffle, store map o...

2015-05-25 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/4450#discussion_r30964120 --- Diff: core/src/main/scala/org/apache/spark/util/collection/PartitionedSerializedPairBuffer.scala --- @@ -0,0 +1,254 @@ +/* + * Licensed to the

[GitHub] spark pull request: [SPARK-7734][DataFrame] support struct type in...

2015-05-25 Thread cloud-fan
Github user cloud-fan closed the pull request at: https://github.com/apache/spark/pull/6266 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-7734][DataFrame] support struct type in...

2015-05-25 Thread cloud-fan
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/6266#issuecomment-10513 @chenghao-intel , Thanks for reminding! Closing. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: SPARK-4550. In sort-based shuffle, store map o...

2015-05-25 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/4450#discussion_r30966102 --- Diff: core/src/main/scala/org/apache/spark/util/collection/ExternalSorter.scala --- @@ -306,14 +297,10 @@ private[spark] class ExternalSorter[K, V, C](

[GitHub] spark pull request: Close HBaseAdmin at the end of HBaseTest

2015-05-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/6381 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: Add test which shows Kryo buffer size configur...

2015-05-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/6390 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-7042][BUILD] use the standard akka arti...

2015-05-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6341#issuecomment-105153564 [Test build #33463 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33463/consoleFull) for PR 6341 at commit

[GitHub] spark pull request: [Spark-7780][MLLIB] Intercept in logisticregre...

2015-05-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6386#issuecomment-105153577 [Test build #33462 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33462/consoleFull) for PR 6386 at commit

[GitHub] spark pull request: [SQL] SPARK-6981: Factor out SparkPlanner and ...

2015-05-25 Thread evacchi
Github user evacchi commented on the pull request: https://github.com/apache/spark/pull/6356#issuecomment-105153556 @chenghao-intel to me both solutions are fine, see PR #6122; however, this solution would retain binary compatibility --- If your project is set up for it, you can

[GitHub] spark pull request: [SPARK-4352][YARN][WIP] Incorporate locality p...

2015-05-25 Thread jerryshao
GitHub user jerryshao opened a pull request: https://github.com/apache/spark/pull/6394 [SPARK-4352][YARN][WIP] Incorporate locality preferences in dynamic allocation requests Currently there's no locality preference for container request in YARN mode, this will affect the

[GitHub] spark pull request: [SPARK-7392][Core] bugfix: Kryo buffer size ca...

2015-05-25 Thread liyezhang556520
Github user liyezhang556520 commented on a diff in the pull request: https://github.com/apache/spark/pull/5934#discussion_r30971371 --- Diff: core/src/test/scala/org/apache/spark/serializer/KryoSerializerSuite.scala --- @@ -32,6 +32,36 @@ class KryoSerializerSuite extends FunSuite

[GitHub] spark pull request: [Spark-7514][MLlib] Add MinMaxScaler to featur...

2015-05-25 Thread hhbyyh
Github user hhbyyh commented on the pull request: https://github.com/apache/spark/pull/6039#issuecomment-105187284 Hi @jkbradley Sorry for the delay. I've removed the MLlib version and added UT. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [SPARK-7850][BUILD] Hive 0.12.0 profile in POM...

2015-05-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6393#issuecomment-105187352 [Test build #33461 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33461/consoleFull) for PR 6393 at commit

[GitHub] spark pull request: [Spark-7514][MLlib] Add MinMaxScaler to featur...

2015-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6039#issuecomment-105188263 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [minor] [sql] Partial revert of e683182c3e.

2015-05-25 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/6243#issuecomment-105150813 I'm going to commit this soon if there are no further comments. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: [SPARK-7850][BUILD] Hive 0.12.0 profile in POM...

2015-05-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6393#issuecomment-105151706 [Test build #33461 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33461/consoleFull) for PR 6393 at commit

[GitHub] spark pull request: [SPARK-7717][WebUI] Only showing total memory ...

2015-05-25 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/6317#discussion_r30966235 --- Diff: core/src/main/scala/org/apache/spark/deploy/master/ui/MasterPage.scala --- @@ -75,6 +75,7 @@ private[ui] class MasterPage(parent: MasterWebUI)

[GitHub] spark pull request: [Spark-7780][MLLIB] Intercept in logisticregre...

2015-05-25 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/6386#discussion_r30966227 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/classification/LogisticRegression.scala --- @@ -363,4 +370,34 @@ class LogisticRegressionWithLBFGS

[GitHub] spark pull request: [SPARK-4352][YARN][WIP] Incorporate locality p...

2015-05-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6394#issuecomment-105165893 [Test build #33464 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33464/consoleFull) for PR 6394 at commit

[GitHub] spark pull request: [SPARK-5681][Streaming] Use a lock to eliminat...

2015-05-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6294#issuecomment-105173571 [Test build #33465 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33465/consoleFull) for PR 6294 at commit

[GitHub] spark pull request: [SPARK-7850][BUILD] Hive 0.12.0 profile in POM...

2015-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6393#issuecomment-105187420 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-7850][BUILD] Hive 0.12.0 profile in POM...

2015-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6393#issuecomment-105187418 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-7850][BUILD] Hive 0.12.0 profile in POM...

2015-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6393#issuecomment-105150613 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-7850][BUILD] Hive 0.12.0 profile in POM...

2015-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6393#issuecomment-105150605 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [minor] [sql] Partial revert of e683182c3e.

2015-05-25 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/6243#issuecomment-105152769 I'm still confused. Is our Jenkins maven build broken? If not, what builds are broken? Is this related to the catalyst module, or the package name? If it is about

[GitHub] spark pull request: [SPARK-7850][BUILD] Hive 0.12.0 profile in POM...

2015-05-25 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/6393#issuecomment-105152989 I think this is the intent - but I will let @marmbrus confirm. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: [SPARK-7042][BUILD] use the standard akka arti...

2015-05-25 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/6341#issuecomment-105152780 Jenkins, test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SQL] SPARK-6981: Factor out SparkPlanner and ...

2015-05-25 Thread evacchi
Github user evacchi commented on a diff in the pull request: https://github.com/apache/spark/pull/6356#discussion_r30966486 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SQLContext.scala --- @@ -166,9 +166,9 @@ class SQLContext(@transient val sparkContext: SparkContext)

[GitHub] spark pull request: [SQL] SPARK-6981: Factor out SparkPlanner and ...

2015-05-25 Thread evacchi
Github user evacchi commented on a diff in the pull request: https://github.com/apache/spark/pull/6356#discussion_r30967193 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SQLContext.scala --- @@ -1321,3 +1201,137 @@ object SQLContext { } } } + +

[GitHub] spark pull request: [Spark-7780][MLLIB] Intercept in logisticregre...

2015-05-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6386#issuecomment-105169568 [Test build #33462 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33462/consoleFull) for PR 6386 at commit

[GitHub] spark pull request: [Spark-7780][MLLIB] Intercept in logisticregre...

2015-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6386#issuecomment-105169578 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [Spark-7780][MLLIB] Intercept in logisticregre...

2015-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6386#issuecomment-105169576 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-5681][Streaming] Use a lock to eliminat...

2015-05-25 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/6294#discussion_r30970195 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/receiver/ReceiverSupervisor.scala --- @@ -136,12 +141,19 @@ private[streaming] abstract class

[GitHub] spark pull request: [SPARK-7042][BUILD] use the standard akka arti...

2015-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6341#issuecomment-105176239 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-7042][BUILD] use the standard akka arti...

2015-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6341#issuecomment-105176234 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-7042][BUILD] use the standard akka arti...

2015-05-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6341#issuecomment-105176203 [Test build #33463 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33463/consoleFull) for PR 6341 at commit

[GitHub] spark pull request: [Spark-7514][MLlib] Add MinMaxScaler to featur...

2015-05-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6039#issuecomment-105189183 [Test build #33466 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33466/consoleFull) for PR 6039 at commit

[GitHub] spark pull request: [SPARK-7850][BUILD] Hive 0.12.0 profile in POM...

2015-05-25 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/6393#issuecomment-105149958 I'm OK with that if this is true and the intent, but then do we need `hive-0.13.1` anymore either? --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request: [SPARK-7604] [MLlib] Python API for PCA and PC...

2015-05-25 Thread yanboliang
Github user yanboliang commented on the pull request: https://github.com/apache/spark/pull/6315#issuecomment-105149865 @mengxr @jkbradley --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-7850][BUILD] Hive 0.12.0 profile in POM...

2015-05-25 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/6393#issuecomment-105149762 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-7042][BUILD] use the standard akka arti...

2015-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6341#issuecomment-105153375 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-7042][BUILD] use the standard akka arti...

2015-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6341#issuecomment-105153385 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [Spark-7780][MLLIB] Intercept in logisticregre...

2015-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6386#issuecomment-105153369 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [Spark-7780][MLLIB] Intercept in logisticregre...

2015-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6386#issuecomment-105153386 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-7850][BUILD] Hive 0.12.0 profile in POM...

2015-05-25 Thread witgo
Github user witgo commented on the pull request: https://github.com/apache/spark/pull/6393#issuecomment-105154271 The code is useless ? https://github.com/apache/spark/tree/master/sql/hive --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: [SPARK-4352][YARN][WIP] Incorporate locality p...

2015-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6394#issuecomment-105164632 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-4352][YARN][WIP] Incorporate locality p...

2015-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6394#issuecomment-105164596 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-5681][Streaming] Use a lock to eliminat...

2015-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6294#issuecomment-105172981 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-5681][Streaming] Use a lock to eliminat...

2015-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6294#issuecomment-105172926 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [Spark-7514][MLlib] Add MinMaxScaler to featur...

2015-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6039#issuecomment-105188316 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-6307][Core] Speed up RDD.cartesian by c...

2015-05-25 Thread viirya
Github user viirya commented on the pull request: https://github.com/apache/spark/pull/5572#issuecomment-105197340 @squito Thanks. I've committed `CartesianRDD`. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-6307][Core] Speed up RDD.cartesian by c...

2015-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5572#issuecomment-105197388 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-6307][Core] Speed up RDD.cartesian by c...

2015-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5572#issuecomment-105197380 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-6307][Core] Speed up RDD.cartesian by c...

2015-05-25 Thread viirya
Github user viirya commented on the pull request: https://github.com/apache/spark/pull/5572#issuecomment-105199065 @squito I supposed that because we don't do any special to them, the newly cached blocks are treated as same as other cached blocks? I think just only applying idea 1

[GitHub] spark pull request: [SPARK-4352][YARN][WIP] Incorporate locality p...

2015-05-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6394#issuecomment-105199884 [Test build #33464 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33464/consoleFull) for PR 6394 at commit

[GitHub] spark pull request: [SPARK-4352][YARN][WIP] Incorporate locality p...

2015-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6394#issuecomment-105199920 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4352][YARN][WIP] Incorporate locality p...

2015-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6394#issuecomment-105199918 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-5681][Streaming] Use a lock to eliminat...

2015-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6294#issuecomment-105205521 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-6307][Core] Speed up RDD.cartesian by c...

2015-05-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5572#issuecomment-105213725 [Test build #33467 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33467/consoleFull) for PR 5572 at commit

[GitHub] spark pull request: [SPARK-6307][Core] Speed up RDD.cartesian by c...

2015-05-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5572#issuecomment-105197636 [Test build #33467 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33467/consoleFull) for PR 5572 at commit

[GitHub] spark pull request: [SPARK-5681][Streaming] Use a lock to eliminat...

2015-05-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6294#issuecomment-105205508 [Test build #33465 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33465/consoleFull) for PR 6294 at commit

[GitHub] spark pull request: [SPARK-5681][Streaming] Use a lock to eliminat...

2015-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6294#issuecomment-105205519 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-6307][Core] Speed up RDD.cartesian by c...

2015-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5572#issuecomment-105213737 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-6307][Core] Speed up RDD.cartesian by c...

2015-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5572#issuecomment-105213734 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [Spark-7514][MLlib] Add MinMaxScaler to featur...

2015-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6039#issuecomment-105215836 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [Spark-7514][MLlib] Add MinMaxScaler to featur...

2015-05-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6039#issuecomment-105215796 [Test build #33466 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33466/consoleFull) for PR 6039 at commit

[GitHub] spark pull request: [Spark-7514][MLlib] Add MinMaxScaler to featur...

2015-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6039#issuecomment-105215843 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-6287][MESOS] Add dynamic allocation to ...

2015-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4984#issuecomment-105239193 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-6287][MESOS] Add dynamic allocation to ...

2015-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4984#issuecomment-105239177 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-6287][MESOS] Add dynamic allocation to ...

2015-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4984#issuecomment-105239806 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-6287][MESOS] Add dynamic allocation to ...

2015-05-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4984#issuecomment-105239527 [Test build #33468 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33468/consoleFull) for PR 4984 at commit

[GitHub] spark pull request: [SPARK-6287][MESOS] Add dynamic allocation to ...

2015-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4984#issuecomment-105245088 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-6287][MESOS] Add dynamic allocation to ...

2015-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4984#issuecomment-105245090 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-6287][MESOS] Add dynamic allocation to ...

2015-05-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4984#issuecomment-105245086 [Test build #33469 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33469/consoleFull) for PR 4984 at commit

[GitHub] spark pull request: [SPARK-7854][test] refine Kryo test suite

2015-05-25 Thread liyezhang556520
GitHub user liyezhang556520 opened a pull request: https://github.com/apache/spark/pull/6395 [SPARK-7854][test] refine Kryo test suite this modification is accroding @JoshRosen 's comments, for details, please refer to

[GitHub] spark pull request: [SPARK-7854][test] refine Kryo test suite

2015-05-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6395#issuecomment-105255847 [Test build #33471 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33471/consoleFull) for PR 6395 at commit

[GitHub] spark pull request: [SPARK-6307][Core] Speed up RDD.cartesian by c...

2015-05-25 Thread cloud-fan
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/5572#issuecomment-105255778 Hi @squito , I don't think calling `rdd2.iterator.toArray` in `CartesianRDD.compute` is a good idea. Every spark task will run `RDD.compute` and one node may have

[GitHub] spark pull request: [SPARK-6307][Core] Speed up RDD.cartesian by c...

2015-05-25 Thread squito
Github user squito commented on the pull request: https://github.com/apache/spark/pull/5572#issuecomment-105257051 @viirya @cloud-fan good point, I hadn't thought about multiple tasks on one executor that are all pulling the same partition of `rdd2`. Still, I'm very worried about

[GitHub] spark pull request: [SPARK-6307][Core] Speed up RDD.cartesian by c...

2015-05-25 Thread viirya
Github user viirya commented on the pull request: https://github.com/apache/spark/pull/5572#issuecomment-105257082 @squito you said: con: newly cached blocks, effectively adding some replication, but using extra memory. Users will most likely find that extra caching quite

[GitHub] spark pull request: [SPARK-7853] [SQL] Fix bug of class loader iss...

2015-05-25 Thread chenghao-intel
GitHub user chenghao-intel opened a pull request: https://github.com/apache/spark/pull/6396 [SPARK-7853] [SQL] Fix bug of class loader issue in Spark SQL ``` bin/spark-sql --jars ./sql/hive/src/test/resources/hive-hcatalog-core-0.13.1.jar CREATE TABLE t1(a string, b string)

[GitHub] spark pull request: [SPARK-7842] [SQL] Makes task committing/abort...

2015-05-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/6378 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-7684] [SQL] Refactoring MetastoreDataSo...

2015-05-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6353#issuecomment-105264899 [Test build #33474 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33474/consoleFull) for PR 6353 at commit

[GitHub] spark pull request: [SPARK-6287][MESOS] Add dynamic allocation to ...

2015-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4984#issuecomment-105239805 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-6287][MESOS] Add dynamic allocation to ...

2015-05-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4984#issuecomment-105239803 [Test build #33468 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33468/consoleFull) for PR 4984 at commit

[GitHub] spark pull request: [SPARK-6287][MESOS] Add dynamic allocation to ...

2015-05-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4984#issuecomment-105244629 [Test build #33469 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33469/consoleFull) for PR 4984 at commit

[GitHub] spark pull request: [SPARK-6287][MESOS] Add dynamic allocation to ...

2015-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4984#issuecomment-105244163 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-6287][MESOS] Add dynamic allocation to ...

2015-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4984#issuecomment-105244180 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-3181][MLLIB]: Add Robust Regression Alg...

2015-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2096#issuecomment-105261355 Build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-7684] [SQL] Refactoring MetastoreDataSo...

2015-05-25 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/6353#issuecomment-105264188 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-7853] [SQL] Fix bug of class loader iss...

2015-05-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6396#issuecomment-105267397 [Test build #33472 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33472/consoleFull) for PR 6396 at commit

[GitHub] spark pull request: [SPARK-7853] [SQL] Fix bug of class loader iss...

2015-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6396#issuecomment-105267432 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-7853] [SQL] Fix bug of class loader iss...

2015-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6396#issuecomment-105267434 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-6287][MESOS] Add dynamic allocation to ...

2015-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4984#issuecomment-105245486 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-6287][MESOS] Add dynamic allocation to ...

2015-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4984#issuecomment-105245461 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-6307][Core] Speed up RDD.cartesian by c...

2015-05-25 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/5572#discussion_r30983933 --- Diff: core/src/main/scala/org/apache/spark/CacheManager.scala --- @@ -48,7 +49,13 @@ private[spark] class CacheManager(blockManager: BlockManager)

[GitHub] spark pull request: [SPARK-7684] [SQL] Invoking HiveContext.newTem...

2015-05-25 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/6359#issuecomment-105255147 I'm merging this to master and branch-1.4. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

  1   2   3   4   >