[GitHub] spark issue #19813: [SPARK-22600][SQL] Fix 64kb limit for deeply nested expr...

2017-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19813 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19813: [SPARK-22600][SQL] Fix 64kb limit for deeply nested expr...

2017-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19813 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84371/ Test PASSed. ---

[GitHub] spark pull request #19758: [SPARK-3162][MLlib] Local Tree Training Pt 1: Ref...

2017-11-30 Thread smurching
Github user smurching commented on a diff in the pull request: https://github.com/apache/spark/pull/19758#discussion_r154284858 --- Diff: mllib/src/test/scala/org/apache/spark/ml/tree/impl/TreeSplitUtilsSuite.scala --- @@ -0,0 +1,280 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #19813: [SPARK-22600][SQL] Fix 64kb limit for deeply nested expr...

2017-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19813 **[Test build #84371 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84371/testReport)** for PR 19813 at commit

[GitHub] spark issue #19840: [SPARK-22640][PYSPARK][YARN]switch python exec on execut...

2017-11-30 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19840 I'm a little concerned about such changes, this may be misconfigured to introduce the discrepancy between driver python and executor python, at least we should honor this configuration

[GitHub] spark issue #19840: [SPARK-22640][PYSPARK][YARN]switch python exec on execut...

2017-11-30 Thread yaooqinn
Github user yaooqinn commented on the issue: https://github.com/apache/spark/pull/19840 Yes, you are right. we should use same python executables. But the **same** might mean binary same not just same path --- - To

[GitHub] spark issue #19857: [SPARK-22667][ML] Fix model-specific optimization suppor...

2017-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19857 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84373/ Test PASSed. ---

[GitHub] spark issue #19857: [SPARK-22667][ML] Fix model-specific optimization suppor...

2017-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19857 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19857: [SPARK-22667][ML] Fix model-specific optimization suppor...

2017-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19857 **[Test build #84373 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84373/testReport)** for PR 19857 at commit

[GitHub] spark issue #19764: [SPARK-22539][SQL] Add second order for rangepartitioner...

2017-11-30 Thread caneGuy
Github user caneGuy commented on the issue: https://github.com/apache/spark/pull/19764 @gczsjdy Added a simple example. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #19840: [SPARK-22640][PYSPARK][YARN]switch python exec on execut...

2017-11-30 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19840 Oh, I see. You're running in client mode. So this one `--conf spark.yarn.appMasterEnv.PYSPARK_PYTHON=py3.zip/py3/bin/python` is useless. So I guess the behavior is expected. Because

[GitHub] spark issue #19857: [SPARK-22667][ML] Fix model-specific optimization suppor...

2017-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19857 **[Test build #84373 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84373/testReport)** for PR 19857 at commit

[GitHub] spark issue #19813: [SPARK-22600][SQL] Fix 64kb limit for deeply nested expr...

2017-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19813 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19813: [SPARK-22600][SQL] Fix 64kb limit for deeply nested expr...

2017-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19813 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84370/ Test FAILed. ---

[GitHub] spark issue #19813: [SPARK-22600][SQL] Fix 64kb limit for deeply nested expr...

2017-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19813 **[Test build #84370 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84370/testReport)** for PR 19813 at commit

[GitHub] spark issue #19824: [SPARK][STREAMING] Invoke onBatchCompletion() only when ...

2017-11-30 Thread CodingCat
Github user CodingCat commented on the issue: https://github.com/apache/spark/pull/19824 `What I want to say is that if a Job is failed, we should consider the Batch as not completed.` isn't #16542 doing the same thing? ---

[GitHub] spark issue #19857: [SPARK-22667][ML] Fix model-specific optimization suppor...

2017-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19857 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84372/ Test FAILed. ---

[GitHub] spark issue #19857: [SPARK-22667][ML] Fix model-specific optimization suppor...

2017-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19857 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19857: [SPARK-22667][ML] Fix model-specific optimization suppor...

2017-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19857 **[Test build #84372 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84372/testReport)** for PR 19857 at commit

[GitHub] spark issue #19857: [SPARK-22667][ML] Fix model-specific optimization suppor...

2017-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19857 **[Test build #84372 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84372/testReport)** for PR 19857 at commit

[GitHub] spark issue #19840: [SPARK-22640][PYSPARK][YARN]switch python exec on execut...

2017-11-30 Thread yaooqinn
Github user yaooqinn commented on the issue: https://github.com/apache/spark/pull/19840 https://user-images.githubusercontent.com/8326978/33471349-e570953e-d6a7-11e7-9fec-74963efe37d2.png;> @jerryshao ENVs are specified ok by yarn, but the `pythonExec` is generated in

[GitHub] spark issue #19857: [SPARK-22667][ML] Fix model-specific optimization suppor...

2017-11-30 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/19857 @MrBago @jkbradley I think this PR need to be reviewed and merged first, before reviewing #19627 Because this PR change some critical code path. ---

[GitHub] spark pull request #19857: [SPARK-22667][ML] Fix model-specific optimization...

2017-11-30 Thread WeichenXu123
GitHub user WeichenXu123 opened a pull request: https://github.com/apache/spark/pull/19857 [SPARK-22667][ML] Fix model-specific optimization support for ML tuning: Python API ## What changes were proposed in this pull request? Python CrossValidator/TrainValidationSplit:

[GitHub] spark issue #19717: [SPARK-22646] [Submission] Spark on Kubernetes - basic s...

2017-11-30 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/19717 @vanzin @mridulm @jerryshao would love your comment. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #19717: [SPARK-22646] [Submission] Spark on Kubernetes - basic s...

2017-11-30 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/19717 will do a pass on the latest over the weekend --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #19717: [SPARK-22646] [Submission] Spark on Kubernetes - basic s...

2017-11-30 Thread liyinan926
Github user liyinan926 commented on the issue: https://github.com/apache/spark/pull/19717 /cc @felixcheung --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19054: [SPARK-18067] Avoid shuffling child if join keys are sup...

2017-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19054 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19054: [SPARK-18067] Avoid shuffling child if join keys are sup...

2017-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19054 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84368/ Test PASSed. ---

[GitHub] spark issue #19054: [SPARK-18067] Avoid shuffling child if join keys are sup...

2017-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19054 **[Test build #84368 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84368/testReport)** for PR 19054 at commit

[GitHub] spark issue #19840: [SPARK-22640][PYSPARK][YARN]switch python exec on execut...

2017-11-30 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19840 I think in YARN we have several different ways to set `PYSPARK_PYTHON`, I guess your issue is that which one should take priority? Can you please: 1. Define a consistent ordering

[GitHub] spark issue #19813: [SPARK-22600][SQL] Fix 64kb limit for deeply nested expr...

2017-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19813 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19813: [SPARK-22600][SQL] Fix 64kb limit for deeply nested expr...

2017-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19813 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84367/ Test PASSed. ---

[GitHub] spark issue #19813: [SPARK-22600][SQL] Fix 64kb limit for deeply nested expr...

2017-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19813 **[Test build #84367 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84367/testReport)** for PR 19813 at commit

[GitHub] spark pull request #18324: [SPARK-21045][PYSPARK]Fixed executor blocked beca...

2017-11-30 Thread dataknocker
Github user dataknocker commented on a diff in the pull request: https://github.com/apache/spark/pull/18324#discussion_r154274837 --- Diff: python/pyspark/worker.py --- @@ -177,8 +180,11 @@ def process(): process() except Exception: try: +

[GitHub] spark issue #19856: [SPARK-22664] The logs about "Connected to Zookeeper" in...

2017-11-30 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19856 >I think the log can't reflect the behavior of consumer connection,because consumer.create doesn't do any connect,it only construct a ZookeeperConsumerConnector instance That's not true,

[GitHub] spark issue #19856: [SPARK-22664] The logs about "Connected to Zookeeper" in...

2017-11-30 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19856 Actually there's no issue here, IMHO I think your understanding of this log is slightly different from the original purpose. ---

[GitHub] spark issue #19856: [SPARK-22664] The logs about "Connected to Zookeeper" in...

2017-11-30 Thread liu-zhaokun
Github user liu-zhaokun commented on the issue: https://github.com/apache/spark/pull/19856 @jerryshao I think the log can't reflect the behavior of consumer connection,because consumer.create doesn't do any connect,it only construct a ZookeeperConsumerConnector instance,so the

[GitHub] spark pull request #18692: [SPARK-21417][SQL] Infer join conditions using pr...

2017-11-30 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18692#discussion_r154272738 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/joins.scala --- @@ -152,3 +153,62 @@ object EliminateOuterJoin extends

[GitHub] spark issue #19856: [SPARK-22664] The logs about "Connected to Zookeeper" in...

2017-11-30 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19856 I guess the original purpose of such log is to reflect the behavior of consumer connection. It is not super necessary to do such trivial change. Also `ReliableKafkaReceiver` is not recommended

[GitHub] spark issue #19856: [SPARK-22664] The logs about "Connected to Zookeeper" in...

2017-11-30 Thread liu-zhaokun
Github user liu-zhaokun commented on the issue: https://github.com/apache/spark/pull/19856 @srowen Please help merge this PR as it has passed all tests. --- - To unsubscribe, e-mail:

[GitHub] spark issue #19840: [SPARK-22640][PYSPARK][YARN]switch python exec on execut...

2017-11-30 Thread yaooqinn
Github user yaooqinn commented on the issue: https://github.com/apache/spark/pull/19840 i guess specifing `PYSPARK_PYTHON=~/anaconda3/envs/py3/bin/python` overwrites spark.executorEnv.PYSPARK_PYTHON by

[GitHub] spark pull request #19783: [SPARK-21322][SQL] support histogram in filter ca...

2017-11-30 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/19783#discussion_r154270654 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/EstimationUtils.scala --- @@ -114,4 +114,197 @@ object

[GitHub] spark issue #19783: [SPARK-21322][SQL] support histogram in filter cardinali...

2017-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19783 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84365/ Test PASSed. ---

[GitHub] spark issue #19831: [SPARK-22626][SQL] Wrong Hive table statistics may trigg...

2017-11-30 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/19831 Yes, I saw some of these tables in my cluster, but the user did not manually modify this parameter: ``` # Detailed Table Information Databasedw Table

[GitHub] spark issue #19783: [SPARK-21322][SQL] support histogram in filter cardinali...

2017-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19783 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19783: [SPARK-21322][SQL] support histogram in filter cardinali...

2017-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19783 **[Test build #84365 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84365/testReport)** for PR 19783 at commit

[GitHub] spark issue #19845: [SPARK-22651][PYTHON][ML] Prevent initiating multiple Hi...

2017-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19845 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84369/ Test PASSed. ---

[GitHub] spark issue #19845: [SPARK-22651][PYTHON][ML] Prevent initiating multiple Hi...

2017-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19845 **[Test build #84369 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84369/testReport)** for PR 19845 at commit

[GitHub] spark issue #19845: [SPARK-22651][PYTHON][ML] Prevent initiating multiple Hi...

2017-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19845 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19631: [SPARK-22372][core, yarn] Make cluster submission use Sp...

2017-11-30 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19631 LGTM. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19813: [SPARK-22600][SQL] Fix 64kb limit for deeply nested expr...

2017-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19813 **[Test build #84371 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84371/testReport)** for PR 19813 at commit

[GitHub] spark issue #19845: [SPARK-22651][PYTHON][ML] Prevent initiating multiple Hi...

2017-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19845 **[Test build #84369 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84369/testReport)** for PR 19845 at commit

[GitHub] spark issue #19813: [SPARK-22600][SQL] Fix 64kb limit for deeply nested expr...

2017-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19813 **[Test build #84370 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84370/testReport)** for PR 19813 at commit

[GitHub] spark pull request #19813: [SPARK-22600][SQL] Fix 64kb limit for deeply nest...

2017-11-30 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19813#discussion_r154267815 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/ExpressionCodegen.scala --- @@ -0,0 +1,248 @@ +/* + *

[GitHub] spark pull request #19813: [SPARK-22600][SQL] Fix 64kb limit for deeply nest...

2017-11-30 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19813#discussion_r154267761 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -1028,12 +1053,18 @@ class

[GitHub] spark pull request #19823: [SPARK-22601][SQL] Data load is getting displayed...

2017-11-30 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19823 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19823: [SPARK-22601][SQL] Data load is getting displayed succes...

2017-11-30 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19823 Thanks! Merged to master/2.2 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #19813: [SPARK-22600][SQL] Fix 64kb limit for deeply nest...

2017-11-30 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19813#discussion_r154266910 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -1028,12 +1053,18 @@ class CodegenContext

[GitHub] spark pull request #19813: [SPARK-22600][SQL] Fix 64kb limit for deeply nest...

2017-11-30 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19813#discussion_r154266575 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/ExpressionCodegen.scala --- @@ -0,0 +1,248 @@ +/* + *

[GitHub] spark pull request #19854: SPARK-22660:Use position() and limit() to fix amb...

2017-11-30 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19854#discussion_r154266377 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/ScriptTransformationExec.scala --- @@ -424,7 +426,9 @@ case class HiveScriptIOSchema

[GitHub] spark pull request #19845: [SPARK-22651][PYTHON][ML] Prevent initiating mult...

2017-11-30 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19845#discussion_r154266335 --- Diff: python/pyspark/ml/tests.py --- @@ -1837,6 +1837,29 @@ def test_read_images(self):

[GitHub] spark pull request #19854: SPARK-22660:Use position() and limit() to fix amb...

2017-11-30 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19854#discussion_r154266308 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/ScriptTransformationExec.scala --- @@ -412,7 +412,9 @@ case class HiveScriptIOSchema

[GitHub] spark issue #19811: [SPARK-18016][SQL] Code Generation: Constant Pool Limit ...

2017-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19811 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84364/ Test FAILed. ---

[GitHub] spark issue #19811: [SPARK-18016][SQL] Code Generation: Constant Pool Limit ...

2017-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19811 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19854: SPARK-22660:Use position() and limit() to fix amb...

2017-11-30 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19854#discussion_r154266187 --- Diff: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaTestUtils.scala --- @@ -296,7 +296,8 @@ class

[GitHub] spark pull request #19854: SPARK-22660:Use position() and limit() to fix amb...

2017-11-30 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19854#discussion_r154266242 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/ScriptTransformationExec.scala --- @@ -412,7 +412,9 @@ case class HiveScriptIOSchema

[GitHub] spark issue #19811: [SPARK-18016][SQL] Code Generation: Constant Pool Limit ...

2017-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19811 **[Test build #84364 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84364/testReport)** for PR 19811 at commit

[GitHub] spark pull request #19854: SPARK-22660:Use position() and limit() to fix amb...

2017-11-30 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19854#discussion_r154265720 --- Diff: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaTestUtils.scala --- @@ -296,7 +296,8 @@ class

[GitHub] spark issue #19823: [SPARK-22601][SQL] Data load is getting displayed succes...

2017-11-30 Thread sujith71955
Github user sujith71955 commented on the issue: https://github.com/apache/spark/pull/19823 Thanks all for the review and guidance. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19845: [SPARK-22651][PYTHON][ML] Prevent initiating multiple Hi...

2017-11-30 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19845 @HyukjinKwon Thanks. I forgot the Hive support is needed to test it. The added test looks good. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #19845: [SPARK-22651][PYTHON][ML] Prevent initiating mult...

2017-11-30 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19845#discussion_r154262931 --- Diff: python/pyspark/ml/tests.py --- @@ -1837,6 +1837,29 @@ def test_read_images(self): self.assertEqual(ImageSchema.undefinedImageType,

[GitHub] spark issue #19054: [SPARK-18067] Avoid shuffling child if join keys are sup...

2017-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19054 **[Test build #84368 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84368/testReport)** for PR 19054 at commit

[GitHub] spark issue #19631: [SPARK-22372][core, yarn] Make cluster submission use Sp...

2017-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19631 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19631: [SPARK-22372][core, yarn] Make cluster submission use Sp...

2017-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19631 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84363/ Test PASSed. ---

[GitHub] spark issue #19631: [SPARK-22372][core, yarn] Make cluster submission use Sp...

2017-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19631 **[Test build #84363 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84363/testReport)** for PR 19631 at commit

[GitHub] spark issue #19813: [SPARK-22600][SQL] Fix 64kb limit for deeply nested expr...

2017-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19813 **[Test build #84367 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84367/testReport)** for PR 19813 at commit

[GitHub] spark issue #19813: [SPARK-22600][SQL] Fix 64kb limit for deeply nested expr...

2017-11-30 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19813 @mgaido91 Thanks for the review. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #19813: [SPARK-22600][SQL] Fix 64kb limit for deeply nest...

2017-11-30 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19813#discussion_r154261964 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/ExpressionCodegen.scala --- @@ -0,0 +1,259 @@ +/* + *

[GitHub] spark issue #19848: [SPARK-22162] Executors and the driver should use consis...

2017-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19848 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19848: [SPARK-22162] Executors and the driver should use consis...

2017-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19848 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84362/ Test PASSed. ---

[GitHub] spark issue #19054: [SPARK-18067] Avoid shuffling child if join keys are sup...

2017-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19054 **[Test build #84366 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84366/testReport)** for PR 19054 at commit

[GitHub] spark issue #19054: [SPARK-18067] Avoid shuffling child if join keys are sup...

2017-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19054 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84366/ Test FAILed. ---

[GitHub] spark issue #19848: [SPARK-22162] Executors and the driver should use consis...

2017-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19848 **[Test build #84362 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84362/testReport)** for PR 19848 at commit

[GitHub] spark issue #19054: [SPARK-18067] Avoid shuffling child if join keys are sup...

2017-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19054 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19054: [SPARK-18067] Avoid shuffling child if join keys are sup...

2017-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19054 **[Test build #84366 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84366/testReport)** for PR 19054 at commit

[GitHub] spark issue #19840: [SPARK-22640][PYSPARK][YARN]switch python exec on execut...

2017-11-30 Thread yaooqinn
Github user yaooqinn commented on the issue: https://github.com/apache/spark/pull/19840 @ueshin cluster mode working, client not --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19850: [SPARK-22653] executorAddress registered in Coars...

2017-11-30 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19850 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19850: [SPARK-22653] executorAddress registered in CoarseGraine...

2017-11-30 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19850 good catch! merging to master! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #19813: [SPARK-22600][SQL] Fix 64kb limit for deeply nest...

2017-11-30 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19813#discussion_r154257537 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Expression.scala --- @@ -115,9 +116,35 @@ abstract class Expression extends

[GitHub] spark pull request #19813: [SPARK-22600][SQL] Fix 64kb limit for deeply nest...

2017-11-30 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19813#discussion_r154257472 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/ExpressionCodegen.scala --- @@ -0,0 +1,259 @@ +/* + *

[GitHub] spark pull request #18692: [SPARK-21417][SQL] Infer join conditions using pr...

2017-11-30 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18692#discussion_r154256590 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/joins.scala --- @@ -152,3 +153,62 @@ object EliminateOuterJoin extends

[GitHub] spark issue #19783: [SPARK-21322][SQL] support histogram in filter cardinali...

2017-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19783 **[Test build #84365 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84365/testReport)** for PR 19783 at commit

[GitHub] spark pull request #19848: [SPARK-22162] Executors and the driver should use...

2017-11-30 Thread rezasafi
Github user rezasafi commented on a diff in the pull request: https://github.com/apache/spark/pull/19848#discussion_r154256205 --- Diff: core/src/main/scala/org/apache/spark/mapred/SparkHadoopMapRedUtil.scala --- @@ -70,7 +70,8 @@ object SparkHadoopMapRedUtil extends Logging {

[GitHub] spark issue #19850: [SPARK-22653] executorAddress registered in CoarseGraine...

2017-11-30 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/19850 Please update the PR title: ``` [SPARK-22653][CORE] executorAddress registered in CoarseGrainedSchedulerBackend.executorDataMap should not be null ``` ---

[GitHub] spark pull request #19783: [SPARK-21322][SQL] support histogram in filter ca...

2017-11-30 Thread ron8hu
Github user ron8hu commented on a diff in the pull request: https://github.com/apache/spark/pull/19783#discussion_r154255068 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/statsEstimation/FilterEstimationSuite.scala --- @@ -578,6 +590,112 @@ class

[GitHub] spark pull request #19783: [SPARK-21322][SQL] support histogram in filter ca...

2017-11-30 Thread ron8hu
Github user ron8hu commented on a diff in the pull request: https://github.com/apache/spark/pull/19783#discussion_r154254419 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/statsEstimation/FilterEstimationSuite.scala --- @@ -359,7 +371,7 @@ class

[GitHub] spark pull request #19783: [SPARK-21322][SQL] support histogram in filter ca...

2017-11-30 Thread ron8hu
Github user ron8hu commented on a diff in the pull request: https://github.com/apache/spark/pull/19783#discussion_r154254145 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/FilterEstimation.scala --- @@ -784,11 +879,16 @@ case

[GitHub] spark issue #19811: [SPARK-18016][SQL] Code Generation: Constant Pool Limit ...

2017-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19811 **[Test build #84364 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84364/testReport)** for PR 19811 at commit

[GitHub] spark issue #19824: [SPARK][STREAMING] Invoke onBatchCompletion() only when ...

2017-11-30 Thread victor-wong
Github user victor-wong commented on the issue: https://github.com/apache/spark/pull/19824 @CodingCat Yes, this PR wants to solve the same issue in https://github.com/apache/spark/pull/16542, but I think this is a better way to solve it. If a Job failed, I think we should not

[GitHub] spark pull request #19783: [SPARK-21322][SQL] support histogram in filter ca...

2017-11-30 Thread ron8hu
Github user ron8hu commented on a diff in the pull request: https://github.com/apache/spark/pull/19783#discussion_r154252063 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/FilterEstimation.scala --- @@ -513,10 +560,9 @@ case

  1   2   3   4   >