[GitHub] spark issue #18730: [SPARK-21527][CORE] Use buffer limit in order to use JAV...

2017-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18730 **[Test build #81020 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81020/testReport)** for PR 18730 at commit

[GitHub] spark issue #18266: [SPARK-20427][SQL] Read JDBC table use custom schema

2017-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18266 **[Test build #81019 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81019/testReport)** for PR 18266 at commit

[GitHub] spark pull request #18971: [SPARK-21764][TESTS] Fix tests failures on Window...

2017-08-22 Thread sarutak
Github user sarutak commented on a diff in the pull request: https://github.com/apache/spark/pull/18971#discussion_r134659662 --- Diff: core/src/test/scala/org/apache/spark/scheduler/ReplayListenerSuite.scala --- @@ -151,7 +153,10 @@ class ReplayListenerSuite extends SparkFunSuite

[GitHub] spark issue #18945: Add option to convert nullable int columns to float colu...

2017-08-22 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18945 BTW, I think it'd be nicer if we can go with the approach above ^ (checking the null in data and setting the correct type). I am okay with any form for the approach above as we have a decent

[GitHub] spark pull request #19025: [SPARK-21813][core] Modify TaskMemoryManager.MAXI...

2017-08-22 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/19025#discussion_r134657843 --- Diff: core/src/main/java/org/apache/spark/memory/TaskMemoryManager.java --- @@ -74,7 +74,7 @@ * Maximum supported data page size (in bytes). In

[GitHub] spark pull request #19025: [SPARK-21813][core] Modify TaskMemoryManager.MAXI...

2017-08-22 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/19025#discussion_r134657758 --- Diff: core/src/main/java/org/apache/spark/memory/TaskMemoryManager.java --- @@ -53,7 +53,7 @@ * retrieve the base object. * * This

[GitHub] spark issue #19021: [SPARK-21603][SQL][FOLLOW-UP] Change the default value o...

2017-08-22 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19021 cc @rednaxelafx --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark pull request #18953: [SPARK-20682][SQL] Update ORC data source based o...

2017-08-22 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/18953#discussion_r134657662 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcUtils.scala --- @@ -0,0 +1,316 @@ +/* + * Licensed to the Apache Software

[GitHub] spark issue #19021: [SPARK-21603][SQL][FOLLOW-UP] Change the default value o...

2017-08-22 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19021 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark pull request #19021: [SPARK-21603][SQL][FOLLOW-UP] Change the default ...

2017-08-22 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19021#discussion_r134657177 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -577,10 +577,10 @@ object SQLConf { .doc("The maximum

[GitHub] spark issue #18730: [SPARK-21527][CORE] Use buffer limit in order to use JAV...

2017-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18730 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #18730: [SPARK-21527][CORE] Use buffer limit in order to ...

2017-08-22 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18730#discussion_r134656903 --- Diff: core/src/main/scala/org/apache/spark/util/io/ChunkedByteBuffer.scala --- @@ -63,6 +65,19 @@ private[spark] class ChunkedByteBuffer(var chunks:

[GitHub] spark issue #18730: [SPARK-21527][CORE] Use buffer limit in order to use JAV...

2017-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18730 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81018/ Test FAILed. ---

[GitHub] spark issue #18730: [SPARK-21527][CORE] Use buffer limit in order to use JAV...

2017-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18730 **[Test build #81018 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81018/testReport)** for PR 18730 at commit

[GitHub] spark issue #18730: [SPARK-21527][CORE] Use buffer limit in order to use JAV...

2017-08-22 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18730 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #18730: [SPARK-21527][CORE] Use buffer limit in order to use JAV...

2017-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18730 **[Test build #81018 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81018/testReport)** for PR 18730 at commit

[GitHub] spark issue #18730: [SPARK-21527][CORE] Use buffer limit in order to use JAV...

2017-08-22 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18730 looks reasonable, do you have some performance numbers? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark issue #18266: [SPARK-20427][SQL] Read JDBC table use custom schema

2017-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18266 **[Test build #81017 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81017/testReport)** for PR 18266 at commit

[GitHub] spark issue #18266: [SPARK-20427][SQL] Read JDBC table use custom schema

2017-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18266 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81017/ Test FAILed. ---

[GitHub] spark issue #18266: [SPARK-20427][SQL] Read JDBC table use custom schema

2017-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18266 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18266: [SPARK-20427][SQL] Read JDBC table use custom schema

2017-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18266 **[Test build #81017 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81017/testReport)** for PR 18266 at commit

[GitHub] spark issue #18953: [SPARK-20682][SQL] Update ORC data source based on Apach...

2017-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18953 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18953: [SPARK-20682][SQL] Update ORC data source based on Apach...

2017-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18953 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81013/ Test PASSed. ---

[GitHub] spark issue #18953: [SPARK-20682][SQL] Update ORC data source based on Apach...

2017-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18953 **[Test build #81013 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81013/testReport)** for PR 18953 at commit

[GitHub] spark issue #19017: SPARK-21804: json_tuple returns null values within repea...

2017-08-22 Thread jmchung
Github user jmchung commented on the issue: https://github.com/apache/spark/pull/19017 @HyukjinKwon @viirya I replaced the functional transformations with a while loop. What do you think about this? Thanks. --- If your project is set up for it, you can reply to this email and

[GitHub] spark issue #19017: SPARK-21804: json_tuple returns null values within repea...

2017-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19017 **[Test build #81016 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81016/testReport)** for PR 19017 at commit

[GitHub] spark pull request #17373: [SPARK-12664][ML] Expose probability in mlp model

2017-08-22 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17373 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #17373: [SPARK-12664][ML] Expose probability in mlp model

2017-08-22 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/17373 Merging with master Thanks @WeichenXu123 ! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #18961: [SPARK-21746][SQL]there is an java.lang.IllegalArgumentE...

2017-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18961 **[Test build #81011 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81011/testReport)** for PR 18961 at commit

[GitHub] spark issue #18266: [SPARK-20427][SQL] Read JDBC table use custom schema

2017-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18266 **[Test build #81015 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81015/testReport)** for PR 18266 at commit

[GitHub] spark issue #18266: [SPARK-20427][SQL] Read JDBC table use custom schema

2017-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18266 **[Test build #81015 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81015/testReport)** for PR 18266 at commit

[GitHub] spark issue #17373: [SPARK-12664][ML] Expose probability in mlp model

2017-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17373 **[Test build #3895 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3895/testReport)** for PR 17373 at commit

[GitHub] spark issue #18957: [SPARK-21744][CORE] Add retry logic for new broadcast in...

2017-08-22 Thread caneGuy
Github user caneGuy commented on the issue: https://github.com/apache/spark/pull/18957 Yes i agree with you @jiangxb1987 since we do not have retry logic for broadcast.Actually,my original idea is when length of Array 'localDirs' in DiskBlockManager is larger than 1.We can retry to

[GitHub] spark issue #16774: [SPARK-19357][ML] Adding parallel model evaluation in ML...

2017-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16774 **[Test build #3896 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3896/testReport)** for PR 16774 at commit

[GitHub] spark issue #18962: [SPARK-21714][CORE][YARN] Avoiding re-uploading remote r...

2017-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18962 **[Test build #81014 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81014/testReport)** for PR 18962 at commit

[GitHub] spark pull request #18492: [SPARK-19326] Speculated task attempts do not get...

2017-08-22 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18492 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #18492: [SPARK-19326] Speculated task attempts do not get launch...

2017-08-22 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18492 thanks, merging to master! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #19025: [SPARK-21813][core] Modify TaskMemoryManager.MAXIMUM_PAG...

2017-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19025 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #19025: [SPARK-21813][core] Modify TaskMemoryManager.MAXI...

2017-08-22 Thread Geek-He
GitHub user Geek-He opened a pull request: https://github.com/apache/spark/pull/19025 [SPARK-21813][core] Modify TaskMemoryManager.MAXIMUM_PAGE_SIZE_BYTES comments ## What changes were proposed in this pull request? The variable "TaskMemoryManager.MAXIMUM_PAGE_SIZE_BYTES"

[GitHub] spark pull request #19011: [ML][MINOR] Make sharedParams update.

2017-08-22 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19011 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #19011: [ML][MINOR] Make sharedParams update.

2017-08-22 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/19011 Merged into master. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #17373: [SPARK-12664][ML] Expose probability in mlp model

2017-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17373 **[Test build #3895 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3895/testReport)** for PR 17373 at commit

[GitHub] spark issue #17373: [SPARK-12664][ML] Expose probability in mlp model

2017-08-22 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/17373 Thanks! Will merge after rerunning tests --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #19017: SPARK-21804: json_tuple returns null values within repea...

2017-08-22 Thread jmchung
Github user jmchung commented on the issue: https://github.com/apache/spark/pull/19017 @HyukjinKwon That's a good point, thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18953: [SPARK-20682][SQL] Update ORC data source based on Apach...

2017-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18953 **[Test build #81012 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81012/testReport)** for PR 18953 at commit

[GitHub] spark issue #18953: [SPARK-20682][SQL] Update ORC data source based on Apach...

2017-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18953 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18953: [SPARK-20682][SQL] Update ORC data source based on Apach...

2017-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18953 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81012/ Test FAILed. ---

[GitHub] spark issue #18953: [SPARK-20682][SQL] Update ORC data source based on Apach...

2017-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18953 **[Test build #81013 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81013/testReport)** for PR 18953 at commit

[GitHub] spark issue #18953: [SPARK-20682][SQL] Update ORC data source based on Apach...

2017-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18953 **[Test build #81012 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81012/testReport)** for PR 18953 at commit

[GitHub] spark pull request #18962: [SPARK-21714][CORE][YARN] Avoiding re-uploading r...

2017-08-22 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/18962#discussion_r134645792 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -330,19 +332,21 @@ object SparkSubmit extends CommandLineUtils {

[GitHub] spark issue #19023: Add R interface of binarizer

2017-08-22 Thread vistep
Github user vistep commented on the issue: https://github.com/apache/spark/pull/19023 @falaki Hi, do you mean the pipeline models in scala API? I agree with you about this. However, as I can see, there is little feature transformer functions in SparkR, which makes building a

[GitHub] spark issue #18313: [SPARK-21087] [ML] CrossValidator, TrainValidationSplit ...

2017-08-22 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/18313 +1 for merging https://github.com/apache/spark/pull/16774 before proceeding with the other work since it will affect everything else. @MLnick I'd be Ok with adding options for best/all/k

[GitHub] spark pull request #18973: [SPARK-21765] Set isStreaming on leaf nodes for s...

2017-08-22 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18973 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #18961: [SPARK-21746][SQL]there is an java.lang.IllegalAr...

2017-08-22 Thread heary-cao
Github user heary-cao commented on a diff in the pull request: https://github.com/apache/spark/pull/18961#discussion_r134643234 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/predicates.scala --- @@ -31,11 +31,22 @@ object InterpretedPredicate {

[GitHub] spark pull request #18966: [SPARK-21751][SQL] CodeGeneraor.splitExpressions ...

2017-08-22 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/18966#discussion_r134643155 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -582,6 +582,15 @@ object SQLConf { .intConf

[GitHub] spark issue #18972: [SPARK-21720][SQL] Fix 64KB JVM bytecode limit problem w...

2017-08-22 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/18972 ping @cloud-fan --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #18935: [SPARK-9104][CORE] Expose Netty memory metrics in Spark

2017-08-22 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/18935 @squito , as the next following step, I would expose these metrics with MetricsSystem, I'm thinking of exposing shuffle related Netty memory usage. For RPC related memory usage, I'm not fully

[GitHub] spark issue #18973: [SPARK-21765] Set isStreaming on leaf nodes for streamin...

2017-08-22 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/18973 Merging this to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #18961: [SPARK-21746][SQL]there is an java.lang.IllegalAr...

2017-08-22 Thread heary-cao
Github user heary-cao commented on a diff in the pull request: https://github.com/apache/spark/pull/18961#discussion_r134641150 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/predicates.scala --- @@ -31,11 +31,22 @@ object InterpretedPredicate {

[GitHub] spark issue #18961: [SPARK-21746][SQL]there is an java.lang.IllegalArgumentE...

2017-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18961 **[Test build #81011 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81011/testReport)** for PR 18961 at commit

[GitHub] spark pull request #18961: [SPARK-21746][SQL]there is an java.lang.IllegalAr...

2017-08-22 Thread heary-cao
Github user heary-cao commented on a diff in the pull request: https://github.com/apache/spark/pull/18961#discussion_r134639568 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala --- @@ -2029,4 +2029,15 @@ class DataFrameSuite extends QueryTest with

[GitHub] spark issue #18973: [SPARK-21765] Set isStreaming on leaf nodes for streamin...

2017-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18973 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81010/ Test PASSed. ---

[GitHub] spark issue #18973: [SPARK-21765] Set isStreaming on leaf nodes for streamin...

2017-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18973 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18973: [SPARK-21765] Set isStreaming on leaf nodes for streamin...

2017-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18973 **[Test build #81010 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81010/testReport)** for PR 18973 at commit

[GitHub] spark pull request #18492: [SPARK-19326] Speculated task attempts do not get...

2017-08-22 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18492#discussion_r134637603 --- Diff: core/src/test/scala/org/apache/spark/ExecutorAllocationManagerSuite.scala --- @@ -188,6 +188,40 @@ class ExecutorAllocationManagerSuite

[GitHub] spark pull request #18193: [SPARK-15616] [SQL] CatalogRelation should fallba...

2017-08-22 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18193#discussion_r134636110 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveStrategies.scala --- @@ -138,6 +138,54 @@ class DetermineTableStats(session:

[GitHub] spark issue #19012: [SPARK-17742][core] Fail launcher app handle if child pr...

2017-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19012 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81009/ Test PASSed. ---

[GitHub] spark issue #19012: [SPARK-17742][core] Fail launcher app handle if child pr...

2017-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19012 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #19012: [SPARK-17742][core] Fail launcher app handle if child pr...

2017-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19012 **[Test build #81009 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81009/testReport)** for PR 19012 at commit

[GitHub] spark pull request #18193: [SPARK-15616] [SQL] CatalogRelation should fallba...

2017-08-22 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18193#discussion_r134635562 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveStrategies.scala --- @@ -138,6 +138,54 @@ class DetermineTableStats(session:

[GitHub] spark pull request #18193: [SPARK-15616] [SQL] CatalogRelation should fallba...

2017-08-22 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18193#discussion_r134635355 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveStrategies.scala --- @@ -138,6 +138,54 @@ class DetermineTableStats(session:

[GitHub] spark issue #17849: [SPARK-10931][ML][PYSPARK] PySpark Models Copy Param Val...

2017-08-22 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/17849 Merged to master, thanks everyone :) (There is also a follow up JIRA https://issues.apache.org/jira/browse/SPARK-21812 for explicitly defining all of the params in Python). --- If your project is

[GitHub] spark pull request #18492: [SPARK-19326] Speculated task attempts do not get...

2017-08-22 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18492#discussion_r134634721 --- Diff: core/src/test/scala/org/apache/spark/ExecutorAllocationManagerSuite.scala --- @@ -188,6 +188,40 @@ class ExecutorAllocationManagerSuite

[GitHub] spark pull request #17849: [SPARK-10931][ML][PYSPARK] PySpark Models Copy Pa...

2017-08-22 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17849 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

2017-08-22 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18968 SGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark pull request #18971: [SPARK-21764][TESTS] Fix tests failures on Window...

2017-08-22 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18971#discussion_r134633585 --- Diff: core/src/test/scala/org/apache/spark/scheduler/ReplayListenerSuite.scala --- @@ -151,7 +153,10 @@ class ReplayListenerSuite extends

[GitHub] spark pull request #18866: [SPARK-21649][SQL] Support writing data into hive...

2017-08-22 Thread jinxing64
Github user jinxing64 closed the pull request at: https://github.com/apache/spark/pull/18866 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #18973: [SPARK-21765] Set isStreaming on leaf nodes for streamin...

2017-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18973 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81008/ Test PASSed. ---

[GitHub] spark issue #18973: [SPARK-21765] Set isStreaming on leaf nodes for streamin...

2017-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18973 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18866: [SPARK-21649][SQL] Support writing data into hive bucket...

2017-08-22 Thread jinxing64
Github user jinxing64 commented on the issue: https://github.com/apache/spark/pull/18866 @cloud-fan Thanks for reply. Looks like #19001 continues working on this and it's more comprehensive. I will close this pr for now. --- If your project is set up for it, you can reply to

[GitHub] spark issue #18973: [SPARK-21765] Set isStreaming on leaf nodes for streamin...

2017-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18973 **[Test build #81008 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81008/testReport)** for PR 18973 at commit

[GitHub] spark pull request #19021: [SPARK-21603][SQL][FOLLOW-UP] Change the default ...

2017-08-22 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/19021#discussion_r134631466 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -577,10 +577,10 @@ object SQLConf { .doc("The maximum

[GitHub] spark issue #18784: [SPARK-21559][Mesos] remove mesos fine-grained mode

2017-08-22 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/18784 @ArtRand @susanxhuynh pls review. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #18975: [SPARK-4131] Support "Writing data into the filesystem f...

2017-08-22 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18975 Will review it tonight. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #18961: [SPARK-21746][SQL]there is an java.lang.IllegalArgumentE...

2017-08-22 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18961 Will review it tonight. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #18896: [SPARK-21681][ML] fix bug of MLOR do not work correctly ...

2017-08-22 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/18896 @jkbradley OK. (Can this directly merged to 2.2 ?) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request #18896: [SPARK-21681][ML] fix bug of MLOR do not work cor...

2017-08-22 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18896 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #18896: [SPARK-21681][ML] fix bug of MLOR do not work correctly ...

2017-08-22 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/18896 @WeichenXu123 would you mind sending a backport PR for 2.2? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #18896: [SPARK-21681][ML] fix bug of MLOR do not work correctly ...

2017-08-22 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/18896 Merging with master and branch-2.2 Thanks @WeichenXu123 @MLnick @sethah ! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark issue #19023: Add R interface of binarizer

2017-08-22 Thread falaki
Github user falaki commented on the issue: https://github.com/apache/spark/pull/19023 I suggest we look at this problem holistically. Basically what is missing is MLLib pipelines. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request #15435: [SPARK-17139][ML] Add model summary for Multinomi...

2017-08-22 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/15435#discussion_r134628661 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala --- @@ -1324,90 +1350,136 @@ private[ml] class

[GitHub] spark pull request #15435: [SPARK-17139][ML] Add model summary for Multinomi...

2017-08-22 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/15435#discussion_r134627545 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala --- @@ -1324,90 +1354,147 @@ private[ml] class

[GitHub] spark issue #18931: [SPARK-21717][SQL] Decouple consume functions of physica...

2017-08-22 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18931 @maropu Yes, I agreed. This PR changes the code-gen'd codes. It is not going to detect the length of gen'd codes and decide to change codes or not. It's because the generation of codes is a

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

2017-08-22 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18968 Maybe we can still have a case for ListQuery, but it is simpler and mainly for better message? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

2017-08-22 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18968 @dilipbiswal @gatorsmile Regarding with the error message, I do think so. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #19001: [SPARK-19256][SQL] Hive bucketing support

2017-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19001 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81005/ Test PASSed. ---

[GitHub] spark issue #19001: [SPARK-19256][SQL] Hive bucketing support

2017-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19001 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #19001: [SPARK-19256][SQL] Hive bucketing support

2017-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19001 **[Test build #81005 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81005/testReport)** for PR 19001 at commit

[GitHub] spark pull request #18488: [SPARK-21255][SQL][WIP] Fixed NPE when creating e...

2017-08-22 Thread mike0sv
Github user mike0sv commented on a diff in the pull request: https://github.com/apache/spark/pull/18488#discussion_r134622727 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala --- @@ -154,13 +154,13 @@ case class StaticInvoke(

[GitHub] spark issue #18488: [SPARK-21255][SQL][WIP] Fixed NPE when creating encoder ...

2017-08-22 Thread mike0sv
Github user mike0sv commented on the issue: https://github.com/apache/spark/pull/18488 Found this in janino documentation, it explains the need for explicit casting: "Type arguments: Are parsed, but otherwise ignored. The most significant restriction that follows is that you must

  1   2   3   4   5   >