[GitHub] spark pull request: [SPARK-10990] [SQL] improve unrolling of compl...

2015-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9016#issuecomment-146716893 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-10665][SQL] Query planner infra for loc...

2015-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9039#issuecomment-146716906 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-10990] [SQL] improve unrolling of compl...

2015-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9016#issuecomment-146716905 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-10665][SQL] Query planner infra for loc...

2015-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9039#issuecomment-146716888 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: SPARK-10771: Implement the shuffle encryption ...

2015-10-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8880#discussion_r41584063 --- Diff: core/src/main/scala/org/apache/spark/crypto/CryptoInputStream.scala --- @@ -0,0 +1,439 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: SPARK-10771: Implement the shuffle encryption ...

2015-10-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8880#discussion_r41584084 --- Diff: core/src/main/scala/org/apache/spark/crypto/CryptoInputStream.scala --- @@ -0,0 +1,439 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-8170][Python]Add signal handler to trap...

2015-10-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9033#issuecomment-146717125 [Test build #1863 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1863/consoleFull) for PR 9033 at commit [`a77b32a`](https://git

[GitHub] spark pull request: SPARK-10771: Implement the shuffle encryption ...

2015-10-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8880#discussion_r41584112 --- Diff: core/src/main/scala/org/apache/spark/crypto/CryptoInputStream.scala --- @@ -0,0 +1,439 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: SPARK-10771: Implement the shuffle encryption ...

2015-10-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8880#discussion_r41584139 --- Diff: core/src/main/scala/org/apache/spark/crypto/CryptoInputStream.scala --- @@ -0,0 +1,439 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-10599] Lower communication for block ma...

2015-10-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8757#issuecomment-146717274 [Test build #1864 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1864/consoleFull) for PR 8757 at commit [`ae98edc`](https://git

[GitHub] spark pull request: [SPARK-11007] [SQL] Adds dictionary aware Parq...

2015-10-08 Thread liancheng
GitHub user liancheng opened a pull request: https://github.com/apache/spark/pull/9040 [SPARK-11007] [SQL] Adds dictionary aware Parquet decimal converters For Parquet decimal columns that are encoded using plain-dictionary encoding, we can make the upper level converter aware of th

[GitHub] spark pull request: [SPARK-10990] [SQL] improve unrolling of compl...

2015-10-08 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/9016#discussion_r41584197 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/columnar/ColumnType.scala --- @@ -34,7 +34,8 @@ import org.apache.spark.unsafe.types.UTF8String

[GitHub] spark pull request: SPARK-10771: Implement the shuffle encryption ...

2015-10-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8880#discussion_r41584178 --- Diff: core/src/main/scala/org/apache/spark/crypto/CryptoInputStream.scala --- @@ -0,0 +1,439 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-10990] [SQL] improve unrolling of compl...

2015-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9016#issuecomment-146718024 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: SPARK-10771: Implement the shuffle encryption ...

2015-10-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8880#discussion_r41584272 --- Diff: core/src/main/scala/org/apache/spark/crypto/CryptoInputStream.scala --- @@ -0,0 +1,439 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-11007] [SQL] Adds dictionary aware Parq...

2015-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9040#issuecomment-146717980 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-11007] [SQL] Adds dictionary aware Parq...

2015-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9040#issuecomment-146718007 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-10990] [SQL] improve unrolling of compl...

2015-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9016#issuecomment-146718027 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43

[GitHub] spark pull request: SPARK-10771: Implement the shuffle encryption ...

2015-10-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8880#discussion_r41584296 --- Diff: core/src/main/scala/org/apache/spark/crypto/CryptoInputStream.scala --- @@ -0,0 +1,439 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: SPARK-10771: Implement the shuffle encryption ...

2015-10-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8880#discussion_r41584339 --- Diff: core/src/main/scala/org/apache/spark/crypto/CryptoOutputStream.scala --- @@ -0,0 +1,224 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: SPARK-10771: Implement the shuffle encryption ...

2015-10-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8880#discussion_r41584330 --- Diff: core/src/main/scala/org/apache/spark/crypto/CryptoOutputStream.scala --- @@ -0,0 +1,224 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-10990] [SQL] improve unrolling of compl...

2015-10-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9016#issuecomment-146718346 [Test build #43441 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43441/consoleFull) for PR 9016 at commit [`96661a8`](https://gith

[GitHub] spark pull request: SPARK-10771: Implement the shuffle encryption ...

2015-10-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8880#discussion_r41584460 --- Diff: core/src/main/scala/org/apache/spark/crypto/CryptoOutputStream.scala --- @@ -0,0 +1,224 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-10665][SQL] Query planner infra for loc...

2015-10-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9039#issuecomment-146718362 [Test build #43442 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43442/consoleFull) for PR 9039 at commit [`a6c8b52`](https://gith

[GitHub] spark pull request: [SPARK-10665][SQL] Query planner infra for loc...

2015-10-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9039#issuecomment-146718419 [Test build #43442 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43442/console) for PR 9039 at commit [`a6c8b52`](https://github.

[GitHub] spark pull request: [SPARK-10665][SQL] Query planner infra for loc...

2015-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9039#issuecomment-146718423 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43

[GitHub] spark pull request: [SPARK-10665][SQL] Query planner infra for loc...

2015-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9039#issuecomment-146718422 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-11007] [SQL] Adds dictionary aware Parq...

2015-10-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9040#issuecomment-146718396 [Test build #43443 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43443/consoleFull) for PR 9040 at commit [`bd694d6`](https://gith

[GitHub] spark pull request: [SPARK-10956] Common MemoryManager interface f...

2015-10-08 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/9000#issuecomment-146718407 LGTM, by the way, pending updates to address comments. The majority of the unaddressed comments are minor and nitpicky, but I think that https://github.com/apache/spar

[GitHub] spark pull request: SPARK-10771: Implement the shuffle encryption ...

2015-10-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8880#discussion_r41584529 --- Diff: core/src/main/scala/org/apache/spark/crypto/CryptoOutputStream.scala --- @@ -0,0 +1,224 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: SPARK-10771: Implement the shuffle encryption ...

2015-10-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8880#discussion_r41584604 --- Diff: core/src/main/scala/org/apache/spark/crypto/CryptoOutputStream.scala --- @@ -0,0 +1,224 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-10772][Streaming][Scala]: NullPointerEx...

2015-10-08 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/8881#discussion_r41584948 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/dstream/TransformedDStream.scala --- @@ -38,6 +39,11 @@ class TransformedDStream[U: ClassTag] (

[GitHub] spark pull request: [SPARK-8170][Python]Add signal handler to trap...

2015-10-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9033#issuecomment-146719675 [Test build #1863 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1863/console) for PR 9033 at commit [`a77b32a`](https://github

[GitHub] spark pull request: SPARK-10771: Implement the shuffle encryption ...

2015-10-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8880#discussion_r41585235 --- Diff: core/src/main/scala/org/apache/spark/crypto/CryptoOutputStream.scala --- @@ -0,0 +1,224 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-10599] Lower communication for block ma...

2015-10-08 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/8757#issuecomment-146720466 Making a pass now --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this fe

[GitHub] spark pull request: SPARK-10771: Implement the shuffle encryption ...

2015-10-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8880#discussion_r41585457 --- Diff: core/src/main/scala/org/apache/spark/crypto/CryptoOutputStream.scala --- @@ -0,0 +1,224 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-10914] UnsafeRow serialization breaks w...

2015-10-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9030#issuecomment-146721142 [Test build #43434 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43434/console) for PR 9030 at commit [`9b79e6f`](https://github.

[GitHub] spark pull request: [SPARK-10914] UnsafeRow serialization breaks w...

2015-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9030#issuecomment-146721248 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43

[GitHub] spark pull request: [SPARK-10914] UnsafeRow serialization breaks w...

2015-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9030#issuecomment-146721247 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: SPARK-10771: Implement the shuffle encryption ...

2015-10-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8880#discussion_r41585744 --- Diff: core/src/main/scala/org/apache/spark/crypto/CryptoCodec.scala --- @@ -0,0 +1,96 @@ +/* + * Licensed to the Apache Software Foundation (ASF) un

[GitHub] spark pull request: SPARK-10771: Implement the shuffle encryption ...

2015-10-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8880#discussion_r41585773 --- Diff: core/src/main/scala/org/apache/spark/crypto/CryptoCodec.scala --- @@ -0,0 +1,96 @@ +/* + * Licensed to the Apache Software Foundation (ASF) un

[GitHub] spark pull request: SPARK-10771: Implement the shuffle encryption ...

2015-10-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8880#discussion_r41585866 --- Diff: core/src/main/scala/org/apache/spark/crypto/CryptoCodec.scala --- @@ -0,0 +1,96 @@ +/* + * Licensed to the Apache Software Foundation (ASF) un

[GitHub] spark pull request: SPARK-10771: Implement the shuffle encryption ...

2015-10-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8880#discussion_r41585923 --- Diff: core/src/main/scala/org/apache/spark/crypto/CryptoInputStream.scala --- @@ -0,0 +1,439 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-11019][streaming][flume] Gracefully shu...

2015-10-08 Thread harishreedharan
GitHub user harishreedharan opened a pull request: https://github.com/apache/spark/pull/9041 [SPARK-11019][streaming][flume] Gracefully shutdown Flume receiver th… …reads. Wait for a minute for the receiver threads to shutdown before interrupting them. You can merge th

[GitHub] spark pull request: SPARK-10771: Implement the shuffle encryption ...

2015-10-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8880#discussion_r41586007 --- Diff: core/src/main/scala/org/apache/spark/crypto/Decryptor.scala --- @@ -0,0 +1,67 @@ +/* + * Licensed to the Apache Software Foundation (ASF) unde

[GitHub] spark pull request: SPARK-10771: Implement the shuffle encryption ...

2015-10-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8880#discussion_r41586058 --- Diff: core/src/main/scala/org/apache/spark/crypto/JceAesCtrCryptoCodec.scala --- @@ -0,0 +1,132 @@ +/* + * Licensed to the Apache Software Foundati

[GitHub] spark pull request: SPARK-10771: Implement the shuffle encryption ...

2015-10-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8880#discussion_r41586081 --- Diff: core/src/main/scala/org/apache/spark/crypto/JceAesCtrCryptoCodec.scala --- @@ -0,0 +1,132 @@ +/* + * Licensed to the Apache Software Foundati

[GitHub] spark pull request: SPARK-10771: Implement the shuffle encryption ...

2015-10-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8880#discussion_r41586127 --- Diff: core/src/main/scala/org/apache/spark/crypto/JceAesCtrCryptoCodec.scala --- @@ -0,0 +1,132 @@ +/* + * Licensed to the Apache Software Foundati

[GitHub] spark pull request: [SPARK-11019][streaming][flume] Gracefully shu...

2015-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9041#issuecomment-146722776 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-11019][streaming][flume] Gracefully shu...

2015-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9041#issuecomment-146722754 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: SPARK-10771: Implement the shuffle encryption ...

2015-10-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8880#discussion_r41586239 --- Diff: core/src/main/scala/org/apache/spark/crypto/JceAesCtrCryptoCodec.scala --- @@ -0,0 +1,132 @@ +/* + * Licensed to the Apache Software Foundati

[GitHub] spark pull request: [SPARK-10599] Lower communication for block ma...

2015-10-08 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/8757#discussion_r41586252 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/distributed/BlockMatrix.scala --- @@ -60,6 +60,7 @@ private[mllib] class GridPartitioner(

[GitHub] spark pull request: SPARK-10771: Implement the shuffle encryption ...

2015-10-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8880#discussion_r41586275 --- Diff: core/src/main/scala/org/apache/spark/crypto/JceAesCtrCryptoCodec.scala --- @@ -0,0 +1,132 @@ +/* + * Licensed to the Apache Software Foundati

[GitHub] spark pull request: [SPARK-10599] Lower communication for block ma...

2015-10-08 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/8757#discussion_r41586259 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/distributed/BlockMatrix.scala --- @@ -352,6 +353,30 @@ class BlockMatrix @Since("1.3.0") (

[GitHub] spark pull request: [SPARK-10599] Lower communication for block ma...

2015-10-08 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/8757#discussion_r41586254 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/distributed/BlockMatrix.scala --- @@ -352,6 +353,30 @@ class BlockMatrix @Since("1.3.0") (

[GitHub] spark pull request: [SPARK-10599] Lower communication for block ma...

2015-10-08 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/8757#discussion_r41586262 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/distributed/BlockMatrix.scala --- @@ -352,6 +353,30 @@ class BlockMatrix @Since("1.3.0") (

[GitHub] spark pull request: [SPARK-11019][streaming][flume] Gracefully shu...

2015-10-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9041#issuecomment-146723009 [Test build #43444 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43444/consoleFull) for PR 9041 at commit [`dba200f`](https://gith

[GitHub] spark pull request: [SPARK-10599] Lower communication for block ma...

2015-10-08 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/8757#issuecomment-146723162 Minor comments only. Other than that, it looks fine to me. @avulanov In your "Block-diagonal matrix multiplication" tests, do you know if data were shuffled d

[GitHub] spark pull request: [SPARK-8654] [SQL] Analysis exception when usi...

2015-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9036#issuecomment-146724653 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-8654] [SQL] Analysis exception when usi...

2015-10-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9036#issuecomment-146724381 [Test build #43435 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43435/console) for PR 9036 at commit [`6a8933b`](https://github.

[GitHub] spark pull request: [SPARK-8654] [SQL] Analysis exception when usi...

2015-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9036#issuecomment-146724656 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43

[GitHub] spark pull request: [SPARK-10914] UnsafeRow serialization breaks w...

2015-10-08 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/9030#issuecomment-146724781 Merging this in master & branch-1.5. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does n

[GitHub] spark pull request: SPARK-10771: Implement the shuffle encryption ...

2015-10-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8880#discussion_r41586546 --- Diff: core/src/main/scala/org/apache/spark/shuffle/BlockStoreShuffleReader.scala --- @@ -48,14 +53,29 @@ private[spark] class BlockStoreShuffleReader[K, C]

[GitHub] spark pull request: [SPARK-10599] Lower communication for block ma...

2015-10-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8757#issuecomment-146724888 [Test build #1864 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1864/console) for PR 8757 at commit [`ae98edc`](https://github

[GitHub] spark pull request: SPARK-10771: Implement the shuffle encryption ...

2015-10-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8880#discussion_r41586581 --- Diff: core/src/main/scala/org/apache/spark/shuffle/BlockStoreShuffleReader.scala --- @@ -48,14 +53,29 @@ private[spark] class BlockStoreShuffleReader[K, C]

[GitHub] spark pull request: [SPARK-10914] UnsafeRow serialization breaks w...

2015-10-08 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/9030 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-11019][streaming][flume] Gracefully shu...

2015-10-08 Thread harishreedharan
Github user harishreedharan commented on the pull request: https://github.com/apache/spark/pull/9041#issuecomment-146724990 /cc @tdas --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this fea

[GitHub] spark pull request: [SPARK-10810] [SPARK-10902] [SQL] Improve sess...

2015-10-08 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/8909#issuecomment-146725227 Non-hive parts LGTM! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have thi

[GitHub] spark pull request: SPARK-10771: Implement the shuffle encryption ...

2015-10-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8880#discussion_r41586770 --- Diff: core/src/main/scala/org/apache/spark/storage/DiskBlockObjectWriter.scala --- @@ -80,12 +83,28 @@ private[spark] class DiskBlockObjectWriter(

[GitHub] spark pull request: [SPARK-10905][SparkR]: Export freqItems() for ...

2015-10-08 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/8962#discussion_r41586805 --- Diff: R/pkg/R/DataFrame.R --- @@ -1873,3 +1872,30 @@ setMethod("as.data.frame", collect(x) } ) + +#' freqI

[GitHub] spark pull request: SPARK-10771: Implement the shuffle encryption ...

2015-10-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8880#discussion_r41586814 --- Diff: core/src/main/scala/org/apache/spark/storage/ShuffleBlockFetcherIterator.scala --- @@ -21,13 +21,17 @@ import java.io.InputStream import java.u

[GitHub] spark pull request: [SPARK-10905][SparkR]: Export freqItems() for ...

2015-10-08 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/8962#discussion_r41586812 --- Diff: R/pkg/R/DataFrame.R --- @@ -1873,3 +1872,30 @@ setMethod("as.data.frame", collect(x) } ) + +#' freqI

[GitHub] spark pull request: [SPARK-10956] Common MemoryManager interface f...

2015-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9000#issuecomment-146725522 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-10956] Common MemoryManager interface f...

2015-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9000#issuecomment-146725531 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-10956] Common MemoryManager interface f...

2015-10-08 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/9000#discussion_r41586984 --- Diff: core/src/main/scala/org/apache/spark/storage/MemoryStore.scala --- @@ -489,17 +478,21 @@ private[spark] class MemoryStore(blockManager: BlockMana

[GitHub] spark pull request: [SPARK-10250][CORE] External group by to handl...

2015-10-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8438#issuecomment-146725794 **[Test build #43422 timed out](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43422/console)** for PR 8438 at commit [`860811d`](https://github

[GitHub] spark pull request: [SPARK-10250][CORE] External group by to handl...

2015-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8438#issuecomment-146725821 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43

[GitHub] spark pull request: SPARK-10771: Implement the shuffle encryption ...

2015-10-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8880#discussion_r41587038 --- Diff: core/src/test/scala/org/apache/spark/crypto/JceAesCtrCryptoCodecSuite.scala --- @@ -0,0 +1,72 @@ +/* + * Licensed to the Apache Software Foun

[GitHub] spark pull request: SPARK-10771: Implement the shuffle encryption ...

2015-10-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8880#discussion_r41587132 --- Diff: core/src/test/scala/org/apache/spark/crypto/JceAesCtrCryptoCodecSuite.scala --- @@ -0,0 +1,72 @@ +/* + * Licensed to the Apache Software Foun

[GitHub] spark pull request: [SPARK-10250][CORE] External group by to handl...

2015-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8438#issuecomment-146725820 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-11019][streaming][flume] Gracefully shu...

2015-10-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9041#issuecomment-146725895 [Test build #43444 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43444/console) for PR 9041 at commit [`dba200f`](https://github.

[GitHub] spark pull request: [SPARK-10810] [SPARK-10902] [SQL] Improve sess...

2015-10-08 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/8909#issuecomment-146725869 Merged into master, new comments will be addressed in follow-up PR, thanks you all! --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: [SPARK-8654] [SQL] Analysis exception when usi...

2015-10-08 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/9036#discussion_r41587125 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/HiveTypeCoercionSuite.scala --- @@ -470,7 +470,8 @@ class HiveTypeCoercionSuit

[GitHub] spark pull request: [SPARK-11019][streaming][flume] Gracefully shu...

2015-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9041#issuecomment-146725914 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: SPARK-10771: Implement the shuffle encryption ...

2015-10-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8880#discussion_r41587190 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala --- @@ -17,6 +17,7 @@ package org.apache.spark.deploy.yarn

[GitHub] spark pull request: [SPARK-10810] [SPARK-10902] [SQL] Improve sess...

2015-10-08 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/8909 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-10956] Common MemoryManager interface f...

2015-10-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9000#issuecomment-146725936 [Test build #43445 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43445/consoleFull) for PR 9000 at commit [`6e2e870`](https://gith

[GitHub] spark pull request: SPARK-10771: Implement the shuffle encryption ...

2015-10-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8880#discussion_r41587181 --- Diff: core/src/test/scala/org/apache/spark/crypto/JceAesCtrCryptoCodecSuite.scala --- @@ -0,0 +1,72 @@ +/* + * Licensed to the Apache Software Foun

[GitHub] spark pull request: [SPARK-11019][streaming][flume] Gracefully shu...

2015-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9041#issuecomment-146725915 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43

[GitHub] spark pull request: [SPARK-10991][ML] logistic regression training...

2015-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9037#issuecomment-146726078 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-10991][ML] logistic regression training...

2015-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9037#issuecomment-146726063 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: SPARK-10771: Implement the shuffle encryption ...

2015-10-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8880#discussion_r41587282 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala --- @@ -608,6 +612,18 @@ private[spark] class ApplicationMaster( }

[GitHub] spark pull request: SPARK-10771: Implement the shuffle encryption ...

2015-10-08 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/8880#issuecomment-146726420 Hi @winningsix, I need to read the buffer handling code again, but it could use some cleanup to make it easier to follow. Also, just pointing out style violations

[GitHub] spark pull request: [SPARK-10990] [SQL] improve unrolling of compl...

2015-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9016#issuecomment-146726579 Build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have thi

[GitHub] spark pull request: [SPARK-10767][PYSPARK] Make pyspark shared par...

2015-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9017#issuecomment-146726586 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-10990] [SQL] improve unrolling of compl...

2015-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9016#issuecomment-146726584 Build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this f

[GitHub] spark pull request: [SPARK-10767][PYSPARK] Make pyspark shared par...

2015-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9017#issuecomment-146726577 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-10991][ML] logistic regression training...

2015-10-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9037#issuecomment-146726783 [Test build #43446 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43446/consoleFull) for PR 9037 at commit [`62fa50e`](https://gith

[GitHub] spark pull request: [SPARK-10956] Common MemoryManager interface f...

2015-10-08 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/9000#discussion_r41587618 --- Diff: core/src/test/scala/org/apache/spark/storage/BlockManagerSuite.scala --- @@ -1040,17 +1044,23 @@ class BlockManagerSuite extends SparkFunSuite wi

[GitHub] spark pull request: [SPARK-10767][PYSPARK] Make pyspark shared par...

2015-10-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9017#issuecomment-146726910 [Test build #43447 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43447/consoleFull) for PR 9017 at commit [`929d850`](https://gith

<    2   3   4   5   6   7   8   9   >