[GitHub] spark pull request: SPARK-4843 [YARN] Squash ExecutorRunnableUtil ...

2015-01-05 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/3696 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: SPARK-4843 [YARN] Squash ExecutorRunnableUtil ...

2015-01-05 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/3696#issuecomment-68834508 LGTM, so I'm going to merge this into `master` (1.3.0). Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request: [SPARK-4987] [SQL] parquet timestamp type supp...

2015-01-05 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/3820#issuecomment-68834477 Thanks for doing this, I've been getting a ton of requests for this feature! Can you also add this to the sql programming guide? --- If your project is set up f

[GitHub] spark pull request: [SPARK-4987] [SQL] parquet timestamp type supp...

2015-01-05 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/3820#discussion_r22511350 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SQLConf.scala --- @@ -141,6 +142,12 @@ private[sql] trait SQLConf { getConf(PARQUET_BINARY_AS_

[GitHub] spark pull request: SPARK-4226: Add support for subqueries in wher...

2015-01-05 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/3888#issuecomment-68833871 This is simpler, but it has several disadvantages to the other approach: - The InSet it collected to the driver and thus could cause OOMs when large - I don't th

[GitHub] spark pull request: Update HiveMetastoreCatalog.scala(override the...

2015-01-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3898#issuecomment-68833443 [Test build #25091 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25091/consoleFull) for PR 3898 at commit [`8d910aa`](https://githu

[GitHub] spark pull request: Update HiveMetastoreCatalog.scala(override the...

2015-01-05 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/3898#issuecomment-68833371 Finally, please file a JIRA and add it to the PR title. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub a

[GitHub] spark pull request: Update HiveMetastoreCatalog.scala(override the...

2015-01-05 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/3898#discussion_r22511002 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala --- @@ -427,6 +427,13 @@ private[hive] case class MetastoreRelation

[GitHub] spark pull request: Update HiveMetastoreCatalog.scala(override the...

2015-01-05 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/3898#issuecomment-68833170 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature ena

[GitHub] spark pull request: [SPARK-5068][SQL]fix bug query data when path ...

2015-01-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3891#issuecomment-68833133 [Test build #25090 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25090/consoleFull) for PR 3891 at commit [`55636f3`](https://githu

[GitHub] spark pull request: [SPARK-5068][SQL]fix bug query data when path ...

2015-01-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3891#issuecomment-68833139 [Test build #25090 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25090/consoleFull) for PR 3891 at commit [`55636f3`](https://gith

[GitHub] spark pull request: Update HiveMetastoreCatalog.scala(override the...

2015-01-05 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/3898#issuecomment-68833152 Can you also add a regression test to [`CachedTableSuite`](https://github.com/apache/spark/blob/master/sql/hive/src/test/scala/org/apache/spark/sql/hive/CachedTableSuite.

[GitHub] spark pull request: [SPARK-5068][SQL]fix bug query data when path ...

2015-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3891#issuecomment-68833142 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25

[GitHub] spark pull request: [SPARK-5068][SQL]fix bug query data when path ...

2015-01-05 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/3891#issuecomment-68832961 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature ena

[GitHub] spark pull request: [SPARK-4991][CORE] Worker should reconnect to ...

2015-01-05 Thread liyezhang556520
Github user liyezhang556520 commented on the pull request: https://github.com/apache/spark/pull/3825#issuecomment-68828355 @markhamstra , thanks for reminder, I'll update this PR by making a try to introduce the supervision. --- If your project is set up for it, you can reply to this

[GitHub] spark pull request: [SPARK-5096] Use sbt tasks instead of vals to ...

2015-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3905#issuecomment-68827013 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25

[GitHub] spark pull request: [SPARK-5096] Use sbt tasks instead of vals to ...

2015-01-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3905#issuecomment-68827011 [Test build #25089 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25089/consoleFull) for PR 3905 at commit [`fd63aae`](https://gith

[GitHub] spark pull request: Support for Mesos DockerInfo

2015-01-05 Thread hellertime
Github user hellertime commented on the pull request: https://github.com/apache/spark/pull/3074#issuecomment-68826850 Alright. I've added a test to the `MesosSchedulerBackendSuite` which checks that the spark conf properties correctly populate the DockerInfo fields. --- If your proj

[GitHub] spark pull request: [SPARK-4574][SQL] Adding support for defining ...

2015-01-05 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/3431#discussion_r22508765 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/sources/NewTableScanSuite.scala --- @@ -0,0 +1,136 @@ +/* +* Licensed to the Apache Software Foun

[GitHub] spark pull request: SPARK-5017 [MLlib] - Use SVD to compute determ...

2015-01-05 Thread tgaloppo
Github user tgaloppo commented on a diff in the pull request: https://github.com/apache/spark/pull/3871#discussion_r22508610 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/stat/impl/MultivariateGaussian.scala --- @@ -17,23 +17,84 @@ package org.apache.spark.mllib

[GitHub] spark pull request: [SPARK-5032] [graphx] Remove GraphX MIMA exclu...

2015-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3856#issuecomment-68826220 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25

[GitHub] spark pull request: [SPARK-5032] [graphx] Remove GraphX MIMA exclu...

2015-01-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3856#issuecomment-68826217 [Test build #25088 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25088/consoleFull) for PR 3856 at commit [`1eea2f6`](https://gith

[GitHub] spark pull request: [SPARK-1953][YARN]yarn client mode Application...

2015-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3607#issuecomment-68825514 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25

[GitHub] spark pull request: [SPARK-1953][YARN]yarn client mode Application...

2015-01-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3607#issuecomment-68825511 [Test build #25087 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25087/consoleFull) for PR 3607 at commit [`3bf70cc`](https://gith

[GitHub] spark pull request: [SPARK-4631] unit test for MQTT

2015-01-05 Thread Bilna
Github user Bilna commented on a diff in the pull request: https://github.com/apache/spark/pull/3844#discussion_r22507984 --- Diff: external/mqtt/src/test/scala/org/apache/spark/streaming/mqtt/MQTTStreamSuite.scala --- @@ -17,31 +17,111 @@ package org.apache.spark.strea

[GitHub] spark pull request: [SPARK-4574][SQL] Adding support for defining ...

2015-01-05 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/3431#discussion_r22507890 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/ddl.scala --- @@ -50,6 +50,7 @@ private[sql] class DDLParser extends StandardTokenParsers with Pa

[GitHub] spark pull request: [SPARK-1600] Refactor FileInputStream tests to...

2015-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3801#issuecomment-68824928 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25

[GitHub] spark pull request: [SPARK-1600] Refactor FileInputStream tests to...

2015-01-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3801#issuecomment-68824923 [Test build #25086 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25086/consoleFull) for PR 3801 at commit [`e4494f4`](https://gith

[GitHub] spark pull request: [SPARK-4912][SQL] Persistent tables for the Sp...

2015-01-05 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/3752#discussion_r22507568 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala --- @@ -55,8 +56,61 @@ private[hive] class HiveMetastoreCatalog(hive: Hi

[GitHub] spark pull request: SPARK-5017 [MLlib] - Use SVD to compute determ...

2015-01-05 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/3871#discussion_r22507242 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/stat/impl/MultivariateGaussian.scala --- @@ -17,23 +17,84 @@ package org.apache.spark.mlli

[GitHub] spark pull request: [SPARK-5096] Use sbt tasks instead of vals to ...

2015-01-05 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/3905#issuecomment-68823449 /cc @andrewor14 I'm throwing away the special casing for network-yarn/assembly... but even on master: ``` > network-yarn/assembly [error] Expect

[GitHub] spark pull request: [SPARK-5096] Use sbt tasks instead of vals to ...

2015-01-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3905#issuecomment-68823445 [Test build #25089 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25089/consoleFull) for PR 3905 at commit [`fd63aae`](https://githu

[GitHub] spark pull request: [SPARK-5096] Use sbt tasks instead of vals to ...

2015-01-05 Thread marmbrus
GitHub user marmbrus opened a pull request: https://github.com/apache/spark/pull/3905 [SPARK-5096] Use sbt tasks instead of vals to get hadoop version You can merge this pull request into a Git repository by running: $ git pull https://github.com/marmbrus/spark effectivePom A

[GitHub] spark pull request: [SPARK-1507][YARN]specify num of cores for AM

2015-01-05 Thread WangTaoTheTonic
Github user WangTaoTheTonic commented on the pull request: https://github.com/apache/spark/pull/3806#issuecomment-68822983 Sorry but I already file a PR that splits spark.driver.memory in https://github.com/apache/spark/pull/3607. Could you please check if anyone already did t

[GitHub] spark pull request: [SPARK-5032] [graphx] Remove GraphX MIMA exclu...

2015-01-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3856#issuecomment-68822597 [Test build #25088 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25088/consoleFull) for PR 3856 at commit [`1eea2f6`](https://githu

[GitHub] spark pull request: [SPARK-5032] [graphx] Remove GraphX MIMA exclu...

2015-01-05 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/3856#issuecomment-68822521 @pwendell OK, I moved the rm's. Thanks for your help! (This should be correct...but running the test may not actually test correctness since the caches have already b

[GitHub] spark pull request: [SPARK-1953][YARN]yarn client mode Application...

2015-01-05 Thread WangTaoTheTonic
Github user WangTaoTheTonic commented on the pull request: https://github.com/apache/spark/pull/3607#issuecomment-68821904 @vanzin Oh thanks for that. @andrewor14 Updated. Could you check if it is ok now? --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request: [SPARK-1953][YARN]yarn client mode Application...

2015-01-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3607#issuecomment-68821743 [Test build #25087 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25087/consoleFull) for PR 3607 at commit [`3bf70cc`](https://githu

[GitHub] spark pull request: [SPARK-4961] [CORE] Put HadoopRDD.getPartition...

2015-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3794#issuecomment-68821077 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25

[GitHub] spark pull request: [SPARK-1600] Refactor FileInputStream tests to...

2015-01-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3801#issuecomment-68821073 [Test build #25086 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25086/consoleFull) for PR 3801 at commit [`e4494f4`](https://githu

[GitHub] spark pull request: [SPARK-4961] [CORE] Put HadoopRDD.getPartition...

2015-01-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3794#issuecomment-68821076 **[Test build #25076 timed out](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25076/consoleFull)** for PR 3794 at commit [`74c1dec`](https://git

[GitHub] spark pull request: [SPARK-5063] Useful error messages for nested ...

2015-01-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3884#issuecomment-68820890 [Test build #25085 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25085/consoleFull) for PR 3884 at commit [`b39e041`](https://gith

[GitHub] spark pull request: [SPARK-5063] Useful error messages for nested ...

2015-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3884#issuecomment-68820897 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25

[GitHub] spark pull request: [SPARK-4286] Integrate external shuffle servic...

2015-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3861#issuecomment-68820573 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25

[GitHub] spark pull request: [SPARK-4286] Integrate external shuffle servic...

2015-01-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3861#issuecomment-68820568 [Test build #25083 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25083/consoleFull) for PR 3861 at commit [`c601c3f`](https://gith

[GitHub] spark pull request: [SPARK-4286] Integrate external shuffle servic...

2015-01-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3861#issuecomment-68820372 [Test build #25082 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25082/consoleFull) for PR 3861 at commit [`ae1d91a`](https://gith

[GitHub] spark pull request: [SPARK-5063] Useful error messages for nested ...

2015-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3884#issuecomment-68820376 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25

[GitHub] spark pull request: [SPARK-5063] Useful error messages for nested ...

2015-01-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3884#issuecomment-68820371 [Test build #25084 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25084/consoleFull) for PR 3884 at commit [`99cc09f`](https://gith

[GitHub] spark pull request: [SPARK-4286] Integrate external shuffle servic...

2015-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3861#issuecomment-68820378 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25

[GitHub] spark pull request: [SPARK-5032] [graphx] Remove GraphX MIMA exclu...

2015-01-05 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/3856#issuecomment-68819654 Okay looks like that was the issue! Let's just put this clean-up in run-tests. It looks like we already do some stuff there: https://github.com/apache/spark/blob

[GitHub] spark pull request: [Minor] Fix comments for GraphX 2D partitionin...

2015-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3904#issuecomment-68819099 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pro

[GitHub] spark pull request: [SPARK-4920][UI]: back port the PR-3763 to bra...

2015-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3768#issuecomment-68818962 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25

[GitHub] spark pull request: [SPARK-4920][UI]: back port the PR-3763 to bra...

2015-01-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3768#issuecomment-68818953 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25080/consoleFull) for PR 3768 at commit [`ec2365f`](https://github.com/a

[GitHub] spark pull request: [SPARK-4387][PySpark] Refactoring python profi...

2015-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3901#issuecomment-68818946 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25

[GitHub] spark pull request: [SPARK-4387][PySpark] Refactoring python profi...

2015-01-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3901#issuecomment-68818939 [Test build #25077 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25077/consoleFull) for PR 3901 at commit [`b4a9306`](https://gith

[GitHub] spark pull request: [SPARK-4014] Add TaskContext.attemptNumber and...

2015-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3849#issuecomment-68818892 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25

[GitHub] spark pull request: [SPARK-4014] Add TaskContext.attemptNumber and...

2015-01-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3849#issuecomment-68818889 [Test build #25075 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25075/consoleFull) for PR 3849 at commit [`1d43aa6`](https://gith

[GitHub] spark pull request: [Minor] Fix comments for GraphX 2D partitionin...

2015-01-05 Thread kj-ki
GitHub user kj-ki opened a pull request: https://github.com/apache/spark/pull/3904 [Minor] Fix comments for GraphX 2D partitioning strategy The sum of vertices on matrix (v0 to v11) is 12. And, I think one same block overlaps in this strategy. This is minor PR, so I didn't

[GitHub] spark pull request: [SPARK-5032] [graphx] Remove GraphX MIMA exclu...

2015-01-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3856#issuecomment-68818808 [Test build #25079 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25079/consoleFull) for PR 3856 at commit [`527ccd9`](https://gith

[GitHub] spark pull request: [SPARK-5032] [graphx] Remove GraphX MIMA exclu...

2015-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3856#issuecomment-68818812 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25

[GitHub] spark pull request: SPARK-4843 [YARN] Squash ExecutorRunnableUtil ...

2015-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3696#issuecomment-68818796 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25

[GitHub] spark pull request: SPARK-4843 [YARN] Squash ExecutorRunnableUtil ...

2015-01-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3696#issuecomment-68818792 [Test build #25078 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25078/consoleFull) for PR 3696 at commit [`486716f`](https://gith

[GitHub] spark pull request: [SPARK-4048] Enhance and extend hadoop-provide...

2015-01-05 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/2982#issuecomment-68817799 Hey @vanzin - I took a look through this. Many of the changes seem like a good idea and a nice restructuring of the build. Thanks for keeping this up to date for so long

[GitHub] spark pull request: [Core][ SPARK-5065]Fix: BroadCast can still wo...

2015-01-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3885#issuecomment-68817668 [Test build #25081 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25081/consoleFull) for PR 3885 at commit [`5eac6f2`](https://gith

[GitHub] spark pull request: [Core][ SPARK-5065]Fix: BroadCast can still wo...

2015-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3885#issuecomment-68817675 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25

[GitHub] spark pull request: [Core][ SPARK-5065]Fix: BroadCast can still wo...

2015-01-05 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/3885#issuecomment-68817539 @SaintBacchus Some of the others _don't_ work well: calling `sc.parallelize` would throw a confusing NullPointerException, for example. I took the shotgun-blast approa

[GitHub] spark pull request: [Core][ SPARK-5065]Fix: BroadCast can still wo...

2015-01-05 Thread SaintBacchus
Github user SaintBacchus commented on the pull request: https://github.com/apache/spark/pull/3885#issuecomment-68817359 Ok, @JoshRosen . Your PR has covered mine. I will open until yours have merged into the master branch. But I still have a question: do we need add so many `assert

[GitHub] spark pull request: [SPARK-4048] Enhance and extend hadoop-provide...

2015-01-05 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/2982#discussion_r22505034 --- Diff: sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/CliSuite.scala --- @@ -69,8 +69,11 @@ class CliSuite extends FunSuite

[GitHub] spark pull request: [SPARK-5063] Useful error messages for nested ...

2015-01-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3884#issuecomment-68817012 [Test build #25085 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25085/consoleFull) for PR 3884 at commit [`b39e041`](https://githu

[GitHub] spark pull request: [Core][ SPARK-5065]Fix: BroadCast can still wo...

2015-01-05 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/3885#issuecomment-68816578 @SaintBacchus I've incorporated the change in this PR into my PR in #3884, plus I've also added tests to prevent other SparkContext methods from being called after the

[GitHub] spark pull request: [SPARK-5063] Useful error messages for nested ...

2015-01-05 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/3884#issuecomment-68816630 I've added some additional tests to prevent users from calling methods on a stopped SparkContext, since this usually resulted in confusing NullPointerExceptions. ---

[GitHub] spark pull request: [SPARK-5063] Useful error messages for nested ...

2015-01-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3884#issuecomment-68816618 [Test build #25084 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25084/consoleFull) for PR 3884 at commit [`99cc09f`](https://githu

[GitHub] spark pull request: SPARK-5017 [MLlib] - Use SVD to compute determ...

2015-01-05 Thread tgaloppo
Github user tgaloppo commented on a diff in the pull request: https://github.com/apache/spark/pull/3871#discussion_r22504722 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/stat/impl/MultivariateGaussian.scala --- @@ -17,23 +17,84 @@ package org.apache.spark.mllib

[GitHub] spark pull request: [SPARK-4048] Enhance and extend hadoop-provide...

2015-01-05 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/2982#discussion_r22504587 --- Diff: examples/pom.xml --- @@ -98,143 +98,145 @@ ${project.version} - org.eclipse.jetty - jetty-server

[GitHub] spark pull request: [SPARK-4286] Integrate external shuffle servic...

2015-01-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3861#issuecomment-68815846 [Test build #25083 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25083/consoleFull) for PR 3861 at commit [`c601c3f`](https://githu

[GitHub] spark pull request: [SPARK-5032] [graphx] Remove GraphX MIMA exclu...

2015-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3856#issuecomment-68815522 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25

[GitHub] spark pull request: [SPARK-5032] [graphx] Remove GraphX MIMA exclu...

2015-01-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3856#issuecomment-68815517 [Test build #25074 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25074/consoleFull) for PR 3856 at commit [`802e252`](https://gith

[GitHub] spark pull request: [SPARK-4286] Integrate external shuffle servic...

2015-01-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3861#issuecomment-68815496 [Test build #25082 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25082/consoleFull) for PR 3861 at commit [`ae1d91a`](https://githu

[GitHub] spark pull request: [SPARK-4508] [SQL] build native date type to c...

2015-01-05 Thread adrian-wang
Github user adrian-wang commented on the pull request: https://github.com/apache/spark/pull/3732#issuecomment-68815238 For oracle's doc about `java.sql.Date`, In https://docs.oracle.com/javase/6/docs/api/java/sql/Date.html To conform with the definition of SQL DATE, the millisecond

[GitHub] spark pull request: [SPARK-4964] [Streaming] Exactly-once semantic...

2015-01-05 Thread tdas
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/3798#issuecomment-68815205 Great! Keep me posted. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this fea

[GitHub] spark pull request: [SPARK-5068][SQL]fix bug query data when path ...

2015-01-05 Thread jeanlyn
Github user jeanlyn commented on the pull request: https://github.com/apache/spark/pull/3891#issuecomment-68814331 Thanks for suggestion! I would optimize this and commit later. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as w

[GitHub] spark pull request: [Core][ SPARK-5065]Fix: BroadCast can still wo...

2015-01-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3885#issuecomment-68813981 [Test build #25081 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25081/consoleFull) for PR 3885 at commit [`5eac6f2`](https://githu

[GitHub] spark pull request: [Core][ SPARK-5065]Fix: BroadCast can still wo...

2015-01-05 Thread SaintBacchus
Github user SaintBacchus commented on the pull request: https://github.com/apache/spark/pull/3885#issuecomment-68813900 Hi, @JoshRosen I had added the test in my PR.But I can't find where having confilict with your PR #3884 . Can you point it out for me? --- If your project is set up

[GitHub] spark pull request: [SPARK-5068][SQL]fix bug query data when path ...

2015-01-05 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/3891#issuecomment-68813712 Okay, that is reasonable and we should probably support this. So then the question is can we do this check on the executor in parallel (or just catch the exception if it

[GitHub] spark pull request: [SPARK-4920][UI]: back port the PR-3763 to bra...

2015-01-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3768#issuecomment-68813622 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25080/consoleFull) for PR 3768 at commit [`ec2365f`](https://github.com/ap

[GitHub] spark pull request: [SPARK-5032] [graphx] Remove GraphX MIMA exclu...

2015-01-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3856#issuecomment-68813591 [Test build #25079 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25079/consoleFull) for PR 3856 at commit [`527ccd9`](https://githu

[GitHub] spark pull request: SPARK-5017 [MLlib] - Use SVD to compute determ...

2015-01-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3871#issuecomment-68813341 [Test build #25069 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25069/consoleFull) for PR 3871 at commit [`383b5b3`](https://gith

[GitHub] spark pull request: SPARK-5017 [MLlib] - Use SVD to compute determ...

2015-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3871#issuecomment-68813348 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25

[GitHub] spark pull request: [SPARK-4014] Add TaskContext.attemptNumber and...

2015-01-05 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/3849#issuecomment-68813324 @pwendell I've updated this based on our previous discussions; please take another look. --- If your project is set up for it, you can reply to this email and have you

[GitHub] spark pull request: [SPARK-3847] Use portable hashcode for Java en...

2015-01-05 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/3795#issuecomment-68813288 @aarondav @pwendell Does one of you want to do the final sign-off on this? --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: SPARK-4843 [YARN] Squash ExecutorRunnableUtil ...

2015-01-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3696#issuecomment-68813229 [Test build #25078 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25078/consoleFull) for PR 3696 at commit [`486716f`](https://githu

[GitHub] spark pull request: [SPARK-4920][UI]: back port the PR-3763 to bra...

2015-01-05 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/3768#issuecomment-68813235 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not ha

[GitHub] spark pull request: [SPARK-4387][PySpark] Refactoring python profi...

2015-01-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3901#issuecomment-68813226 [Test build #25077 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25077/consoleFull) for PR 3901 at commit [`b4a9306`](https://githu

[GitHub] spark pull request: [SPARK-4504] Minor bug fixes in bin/run-exampl...

2015-01-05 Thread witgo
Github user witgo closed the pull request at: https://github.com/apache/spark/pull/3069 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabl

[GitHub] spark pull request: [SPARK-1600] Refactor FileInputStream tests to...

2015-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3801#issuecomment-68813146 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25

[GitHub] spark pull request: SPARK-4843 [YARN] Squash ExecutorRunnableUtil ...

2015-01-05 Thread ksakellis
Github user ksakellis commented on the pull request: https://github.com/apache/spark/pull/3696#issuecomment-68813105 @JoshRosen yup, moved lazy val env = prepareEnvironment to be last statement so it is the same as before. --- If your project is set up for it, you can reply to this e

[GitHub] spark pull request: [SPARK-1600] Refactor FileInputStream tests to...

2015-01-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3801#issuecomment-68813139 [Test build #25071 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25071/consoleFull) for PR 3801 at commit [`8340bd0`](https://gith

[GitHub] spark pull request: [SPARK-5068][SQL]fix bug query data when path ...

2015-01-05 Thread jeanlyn
Github user jeanlyn commented on the pull request: https://github.com/apache/spark/pull/3891#issuecomment-68813094 Yes,hive is work in this situation.I found this issue from our production environment when i try to use spark-sql to test some sql which run in hive original. I am not fa

[GitHub] spark pull request: [SPARK-4737] Task set manager properly handles...

2015-01-05 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/3638#issuecomment-68812961 Thanks for bringing this up to date. The only comments that I have left at this point are mostly minor style issues that I can fix myself on merge. --- If your projec

[GitHub] spark pull request: [SPARK-4048] Enhance and extend hadoop-provide...

2015-01-05 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/2982#discussion_r22503323 --- Diff: bin/compute-classpath.cmd --- @@ -109,6 +115,24 @@ if "x%YARN_CONF_DIR%"=="x" goto no_yarn_conf_dir set CLASSPATH=%CLASSPATH%;%YARN_CONF_DIR%

  1   2   3   4   >