[GitHub] spark pull request #17077: [SPARK-16931][PYTHON][SQL] Add Python wrapper for...

2017-04-10 Thread zero323
Github user zero323 commented on a diff in the pull request: https://github.com/apache/spark/pull/17077#discussion_r110699779 --- Diff: python/pyspark/sql/tests.py --- @@ -2167,6 +2167,61 @@ def test_BinaryType_serialization(self): df = self.spark.createDataFrame(data,

[GitHub] spark pull request #16774: [SPARK-19357][ML] Adding parallel model evaluatio...

2017-04-10 Thread MLnick
Github user MLnick commented on a diff in the pull request: https://github.com/apache/spark/pull/16774#discussion_r110717717 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tuning/CrossValidator.scala --- @@ -100,31 +108,60 @@ class CrossValidator @Since("1.2.0")

[GitHub] spark pull request #17591: [SPARK-20280][CORE] FileStatusCache Weigher integ...

2017-04-10 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/17591#discussion_r110692150 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/FileIndexSuite.scala --- @@ -220,6 +221,32 @@ class FileIndexSuite extends

[GitHub] spark pull request #17591: [SPARK-20280][CORE] FileStatusCache Weigher integ...

2017-04-10 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/17591#discussion_r110692271 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileStatusCache.scala --- @@ -94,27 +94,46 @@ private class

[GitHub] spark pull request #17566: [SPARK-19518][SQL] IGNORE NULLS in first / last i...

2017-04-10 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17566 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #17330: [SPARK-19993][SQL] Caching logical plans containing subq...

2017-04-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17330 **[Test build #75664 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75664/testReport)** for PR 17330 at commit

[GitHub] spark issue #17587: [SPARK-20274][SQL] support compatible array element type...

2017-04-10 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/17587 @cloud-fan Good to see what plan is generated in your comment. People (especially I) will forgot what plan we generated in the future. When I saw the comment in `upCastToExpectedType`, I

[GitHub] spark issue #17591: [SPARK-20280][CORE] FileStatusCache Weigher integer over...

2017-04-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17591 **[Test build #75665 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75665/testReport)** for PR 17591 at commit

[GitHub] spark issue #17077: [SPARK-16931][PYTHON][SQL] Add Python wrapper for bucket...

2017-04-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17077 **[Test build #75666 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75666/testReport)** for PR 17077 at commit

[GitHub] spark issue #17566: [SPARK-19518][SQL] IGNORE NULLS in first / last in SQL

2017-04-10 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/17566 LGTM - merging to master. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #17330: [SPARK-19993][SQL] Caching logical plans containi...

2017-04-10 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/17330#discussion_r110693397 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/subquery.scala --- @@ -59,6 +58,13 @@ abstract class

[GitHub] spark issue #17077: [SPARK-16931][PYTHON][SQL] Add Python wrapper for bucket...

2017-04-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17077 **[Test build #75667 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75667/testReport)** for PR 17077 at commit

[GitHub] spark pull request #17308: [SPARK-19968][SS] Use a cached instance of `Kafka...

2017-04-10 Thread BenFradet
Github user BenFradet commented on a diff in the pull request: https://github.com/apache/spark/pull/17308#discussion_r110685760 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/CachedKafkaProducer.scala --- @@ -0,0 +1,70 @@ +/* + * Licensed

[GitHub] spark issue #17593: [SPARK-20279][WEB-UI]In web ui,'Only showing 200' should...

2017-04-10 Thread ajbozarth
Github user ajbozarth commented on the issue: https://github.com/apache/spark/pull/17593 I agree with @srowen we left it that way since sorting can change --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark issue #17592: [SPARK-20243][TESTS] DebugFilesystem.assertNoOpenStreams...

2017-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17592 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17591: [SPARK-20280][CORE] FileStatusCache Weigher integer over...

2017-04-10 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/17591 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #17520: [WIP][SPARK-19712][SQL] Move PullupCorrelatedPredicates ...

2017-04-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17520 **[Test build #75668 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75668/testReport)** for PR 17520 at commit

[GitHub] spark pull request #17077: [SPARK-16931][PYTHON][SQL] Add Python wrapper for...

2017-04-10 Thread zero323
Github user zero323 commented on a diff in the pull request: https://github.com/apache/spark/pull/17077#discussion_r110693499 --- Diff: python/pyspark/sql/tests.py --- @@ -2167,6 +2167,61 @@ def test_BinaryType_serialization(self): df = self.spark.createDataFrame(data,

[GitHub] spark pull request #17585: [SPARK-20273] [SQL] Disallow Non-deterministic Fi...

2017-04-10 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17585 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #17364: [SPARK-20038] [SQL]: FileFormatWriter.ExecuteWriteTask.r...

2017-04-10 Thread steveloughran
Github user steveloughran commented on the issue: https://github.com/apache/spark/pull/17364 @squito Is this ready to go in? Like I warned, I'm not going to add tests for this, not on its own --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark issue #17550: [SPARK-20240][SQL] SparkSQL support limitations of max d...

2017-04-10 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17550 We do not add the new things into 1.6 branch. Please open the PR using the master branch --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request #17480: [SPARK-20079][Core][yarn] Re registration of AM h...

2017-04-10 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/17480#discussion_r110716862 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -249,7 +249,14 @@ private[spark] class ExecutorAllocationManager(

[GitHub] spark pull request #17592: [SPARK-20243][TESTS] DebugFilesystem.assertNoOpen...

2017-04-10 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17592 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #16774: [SPARK-19357][ML] Adding parallel model evaluatio...

2017-04-10 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/16774#discussion_r110713910 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tuning/CrossValidator.scala --- @@ -100,31 +108,60 @@ class CrossValidator @Since("1.2.0")

[GitHub] spark issue #17592: [SPARK-20243][TESTS] DebugFilesystem.assertNoOpenStreams...

2017-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17592 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75662/ Test PASSed. ---

[GitHub] spark issue #17592: [SPARK-20243][TESTS] DebugFilesystem.assertNoOpenStreams...

2017-04-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17592 **[Test build #75662 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75662/testReport)** for PR 17592 at commit

[GitHub] spark issue #17077: [SPARK-16931][PYTHON][SQL] Add Python wrapper for bucket...

2017-04-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17077 **[Test build #75667 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75667/testReport)** for PR 17077 at commit

[GitHub] spark issue #17077: [SPARK-16931][PYTHON][SQL] Add Python wrapper for bucket...

2017-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17077 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17077: [SPARK-16931][PYTHON][SQL] Add Python wrapper for bucket...

2017-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17077 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75667/ Test PASSed. ---

[GitHub] spark pull request #17436: [SPARK-20101][SQL] Use OffHeapColumnVector when "...

2017-04-10 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/17436#discussion_r110714213 --- Diff: core/src/main/scala/org/apache/spark/SparkConf.scala --- @@ -67,6 +67,9 @@ class SparkConf(loadDefaults: Boolean) extends Cloneable with Logging

[GitHub] spark issue #17592: [SPARK-20243][TESTS] DebugFilesystem.assertNoOpenStreams...

2017-04-10 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/17592 Merging to master. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #17591: [SPARK-20280][CORE] FileStatusCache Weigher integ...

2017-04-10 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/17591#discussion_r110692833 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileStatusCache.scala --- @@ -94,27 +94,46 @@ private class

[GitHub] spark pull request #17077: [SPARK-16931][PYTHON][SQL] Add Python wrapper for...

2017-04-10 Thread zero323
Github user zero323 commented on a diff in the pull request: https://github.com/apache/spark/pull/17077#discussion_r110692936 --- Diff: python/pyspark/sql/tests.py --- @@ -2167,6 +2167,61 @@ def test_BinaryType_serialization(self): df = self.spark.createDataFrame(data,

[GitHub] spark issue #12823: [SPARK-14985][ML] Update LinearRegression, LogisticRegre...

2017-04-10 Thread BenFradet
Github user BenFradet commented on the issue: https://github.com/apache/spark/pull/12823 ping @jkbradley if you could take a look, that'd be great. If you have the time, there is also the #17431 segue. --- If your project is set up for it, you can reply to this email and

[GitHub] spark issue #17077: [SPARK-16931][PYTHON][SQL] Add Python wrapper for bucket...

2017-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17077 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17077: [SPARK-16931][PYTHON][SQL] Add Python wrapper for bucket...

2017-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17077 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75666/ Test FAILed. ---

[GitHub] spark issue #17077: [SPARK-16931][PYTHON][SQL] Add Python wrapper for bucket...

2017-04-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17077 **[Test build #75666 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75666/testReport)** for PR 17077 at commit

[GitHub] spark pull request #16774: [SPARK-19357][ML] Adding parallel model evaluatio...

2017-04-10 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/16774#discussion_r110712166 --- Diff: docs/ml-tuning.md --- @@ -55,6 +55,9 @@ for multiclass problems. The default metric used to choose the best `ParamMap` c method in each

[GitHub] spark issue #17592: [SPARK-20243][TESTS] DebugFilesystem.assertNoOpenStreams...

2017-04-10 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/17592 Should this go into branch-2.1 as well? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17330: [SPARK-19993][SQL] Caching logical plans containing subq...

2017-04-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17330 **[Test build #75664 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75664/testReport)** for PR 17330 at commit

[GitHub] spark issue #17595: [SPARK-20283][SQL] Add preOptimizationBatches

2017-04-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17595 **[Test build #75671 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75671/testReport)** for PR 17595 at commit

[GitHub] spark issue #17445: [SPARK-20115] [CORE] Fix DAGScheduler to recompute all t...

2017-04-10 Thread umehrot2
Github user umehrot2 commented on the issue: https://github.com/apache/spark/pull/17445 @kayousterhout @mridulm @rxin @lins05 Can you take a look at this PR ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #17597: [SPARK-20285][Tests]Increase the pyspark streaming test ...

2017-04-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17597 **[Test build #75673 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75673/testReport)** for PR 17597 at commit

[GitHub] spark issue #17527: [SPARK-20156][CORE][SQL][STREAMING][MLLIB] Java String t...

2017-04-10 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/17527 Merged to master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #17594: [SPARK-20282][SS][Tests]Write the commit log first to fi...

2017-04-10 Thread marmbrus
Github user marmbrus commented on the issue: https://github.com/apache/spark/pull/17594 LGTM, for fixing the issue with the test. We should separately decide if this is really the behavior we want for the commit log. --- If your project is set up for it, you can reply to this email

[GitHub] spark issue #17596: [SPARK-12837][SQL] reduce the serialized size of accumul...

2017-04-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17596 **[Test build #75672 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75672/testReport)** for PR 17596 at commit

[GitHub] spark issue #17596: [SPARK-12837][SQL] reduce the serialized size of accumul...

2017-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17596 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75672/ Test FAILed. ---

[GitHub] spark issue #17596: [SPARK-12837][SQL] reduce the serialized size of accumul...

2017-04-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17596 **[Test build #75672 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75672/testReport)** for PR 17596 at commit

[GitHub] spark issue #17596: [SPARK-12837][SQL] reduce the serialized size of accumul...

2017-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17596 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17597: [SPARK-20285][Tests]Increase the pyspark streaming test ...

2017-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17597 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17597: [SPARK-20285][Tests]Increase the pyspark streaming test ...

2017-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17597 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75673/ Test PASSed. ---

[GitHub] spark issue #17595: [SPARK-20283][SQL] Add preOptimizationBatches

2017-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17595 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75671/ Test FAILed. ---

[GitHub] spark issue #17595: [SPARK-20283][SQL] Add preOptimizationBatches

2017-04-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17595 **[Test build #75671 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75671/testReport)** for PR 17595 at commit

[GitHub] spark issue #17595: [SPARK-20283][SQL] Add preOptimizationBatches

2017-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17595 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #17495: [SPARK-20172][Core] Add file permission check whe...

2017-04-10 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/17495#discussion_r110722790 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -320,14 +321,15 @@ private[history] class

[GitHub] spark issue #17591: [SPARK-20280][CORE] FileStatusCache Weigher integer over...

2017-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17591 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17591: [SPARK-20280][CORE] FileStatusCache Weigher integer over...

2017-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17591 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75665/ Test PASSed. ---

[GitHub] spark issue #17591: [SPARK-20280][CORE] FileStatusCache Weigher integer over...

2017-04-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17591 **[Test build #75665 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75665/testReport)** for PR 17591 at commit

[GitHub] spark pull request #17594: Write the log first to fix a race contion in test...

2017-04-10 Thread zsxwing
GitHub user zsxwing opened a pull request: https://github.com/apache/spark/pull/17594 Write the log first to fix a race contion in tests ## What changes were proposed in this pull request? This PR fixes the following failure: ``` sbt.ForkMain$ForkError:

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-10 Thread ioana-delaney
Github user ioana-delaney commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110741246 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/CostBasedJoinReorder.scala --- @@ -327,3 +345,104 @@ object

[GitHub] spark issue #17598: [SPARK-20284][CORE] Make {Des,S}erializationStream exten...

2017-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17598 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #17597: [SPARK-20285][Tests]Increase the pyspark streaming test ...

2017-04-10 Thread brkyvz
Github user brkyvz commented on the issue: https://github.com/apache/spark/pull/17597 LGTM! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark pull request #17597: [SPARK-20285][Tests]Increase the pyspark streamin...

2017-04-10 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17597 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #17594: [SPARK-20282][SS][Tests]Write the commit log first to fi...

2017-04-10 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/17594 Thanks! Merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #17594: [SPARK-20282][SS][Tests]Write the commit log firs...

2017-04-10 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/17594#discussion_r110735241 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamExecution.scala --- @@ -304,8 +304,8 @@ class StreamExecution(

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-10 Thread ioana-delaney
Github user ioana-delaney commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110742657 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/CostBasedJoinReorder.scala --- @@ -218,28 +220,44 @@ object

[GitHub] spark issue #17594: [SPARK-20282][SS][Tests]Write the commit log first to fi...

2017-04-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17594 **[Test build #75670 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75670/testReport)** for PR 17594 at commit

[GitHub] spark issue #17596: [SPARK-12837][SQL] reduce the serialized size of accumul...

2017-04-10 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17596 cc @rxin @davies @andrewor14 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #17596: [SPARK-12837][SQL] reduce the serialized size of ...

2017-04-10 Thread cloud-fan
GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/17596 [SPARK-12837][SQL] reduce the serialized size of accumulator ## What changes were proposed in this pull request? When sending accumulator updates back to driver, the network overhead is

[GitHub] spark pull request #17598: [SPARK-20284][CORE] Make {Des,S}erializationStrea...

2017-04-10 Thread superbobry
GitHub user superbobry opened a pull request: https://github.com/apache/spark/pull/17598 [SPARK-20284][CORE] Make {Des,S}erializationStream extend Closeable ## What changes were proposed in this pull request? This PR allows to use `SerializationStream` and

[GitHub] spark issue #17591: [SPARK-20280][CORE] FileStatusCache Weigher integer over...

2017-04-10 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/17591 Merging to master. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-10 Thread ioana-delaney
Github user ioana-delaney commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110740755 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/CostBasedJoinReorder.scala --- @@ -54,8 +54,6 @@ case class

[GitHub] spark issue #17520: [WIP][SPARK-19712][SQL] Move PullupCorrelatedPredicates ...

2017-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17520 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75668/ Test PASSed. ---

[GitHub] spark issue #17520: [WIP][SPARK-19712][SQL] Move PullupCorrelatedPredicates ...

2017-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17520 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17568: [SPARK-20254][SQL] Remove unnecessary data conversion fo...

2017-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17568 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75669/ Test PASSed. ---

[GitHub] spark issue #17568: [SPARK-20254][SQL] Remove unnecessary data conversion fo...

2017-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17568 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #17495: [SPARK-20172][Core] Add file permission check whe...

2017-04-10 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/17495#discussion_r110721933 --- Diff: core/src/test/scala/org/apache/spark/deploy/history/FsHistoryProviderSuite.scala --- @@ -571,6 +572,34 @@ class FsHistoryProviderSuite extends

[GitHub] spark issue #17330: [SPARK-19993][SQL] Caching logical plans containing subq...

2017-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17330 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17330: [SPARK-19993][SQL] Caching logical plans containing subq...

2017-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17330 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75664/ Test PASSed. ---

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-10 Thread ioana-delaney
Github user ioana-delaney commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110742361 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/CostBasedJoinReorder.scala --- @@ -327,3 +345,104 @@ object

[GitHub] spark issue #17527: [SPARK-20156][CORE][SQL][STREAMING][MLLIB] Java String t...

2017-04-10 Thread nihavend
Github user nihavend commented on the issue: https://github.com/apache/spark/pull/17527 Thank you very much all of you for all your efforts. Sometimes, facing the same issue different platforms and looking for a way to set jvm options for locale explicitly. But many times there is no

[GitHub] spark issue #17566: [SPARK-19518][SQL] IGNORE NULLS in first / last in SQL

2017-04-10 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/17566 Thank you @hvanhovell. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #17597: [SPARK-20285][Tests]Increase the pyspark streamin...

2017-04-10 Thread zsxwing
GitHub user zsxwing opened a pull request: https://github.com/apache/spark/pull/17597 [SPARK-20285][Tests]Increase the pyspark streaming test timeout to 30 seconds ## What changes were proposed in this pull request? Saw the following failure locally: ```

[GitHub] spark issue #17595: [SPARK-20283][SQL] Add preOptimizationBatches

2017-04-10 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/17595 LGTM - pending jenkins. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #17591: [SPARK-20280][CORE] FileStatusCache Weigher integ...

2017-04-10 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17591 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #17520: [WIP][SPARK-19712][SQL] Move PullupCorrelatedPredicates ...

2017-04-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17520 **[Test build #75668 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75668/testReport)** for PR 17520 at commit

[GitHub] spark issue #17597: [SPARK-20285][Tests]Increase the pyspark streaming test ...

2017-04-10 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/17597 Thanks! Merging to master, 2.1 and 2.0. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17582: [SPARK-20239][Core] Improve HistoryServer's ACL mechanis...

2017-04-10 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/17582 > user configured with "spark.admin.acls" (or group) or "spark.ui.view.acls" (or group), or the user who started SHS could list all the applications, otherwise none of them can be listed So

[GitHub] spark issue #17568: [SPARK-20254][SQL] Remove unnecessary data conversion fo...

2017-04-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17568 **[Test build #75669 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75669/testReport)** for PR 17568 at commit

[GitHub] spark pull request #17594: [SPARK-20282][SS][Tests]Write the commit log firs...

2017-04-10 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/17594#discussion_r110735710 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamExecution.scala --- @@ -304,8 +304,8 @@ class StreamExecution(

[GitHub] spark pull request #17595: [SPARK-20283][SQL] Add preOptimizationBatches

2017-04-10 Thread rxin
GitHub user rxin opened a pull request: https://github.com/apache/spark/pull/17595 [SPARK-20283][SQL] Add preOptimizationBatches ## What changes were proposed in this pull request? We currently have postHocOptimizationBatches, but not preOptimizationBatches. This patch adds

[GitHub] spark issue #16347: [SPARK-18934][SQL] Writing to dynamic partitions does no...

2017-04-10 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/16347 is this still a problem? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #17527: [SPARK-20156][CORE][SQL][STREAMING][MLLIB] Java S...

2017-04-10 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17527 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #17597: [SPARK-20285][Tests]Increase the pyspark streaming test ...

2017-04-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17597 **[Test build #75673 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75673/testReport)** for PR 17597 at commit

[GitHub] spark issue #17546: [SPARK-20233] [SQL] Apply star-join filter heuristics to...

2017-04-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17546 **[Test build #75674 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75674/testReport)** for PR 17546 at commit

[GitHub] spark pull request #17594: [SPARK-20282][SS][Tests]Write the commit log firs...

2017-04-10 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/17594#discussion_r110760826 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamExecution.scala --- @@ -304,8 +304,8 @@ class StreamExecution(

[GitHub] spark issue #17568: [SPARK-20254][SQL] Remove unnecessary data conversion fo...

2017-04-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17568 **[Test build #75669 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75669/testReport)** for PR 17568 at commit

[GitHub] spark issue #17594: [SPARK-20282][SS][Tests]Write the commit log first to fi...

2017-04-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17594 **[Test build #75670 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75670/testReport)** for PR 17594 at commit

[GitHub] spark issue #17594: [SPARK-20282][SS][Tests]Write the commit log first to fi...

2017-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17594 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75670/ Test PASSed. ---

[GitHub] spark issue #17594: [SPARK-20282][SS][Tests]Write the commit log first to fi...

2017-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17594 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

  1   2   3   4   5   >