[GitHub] spark issue #16981: [SPARK-19637][SQL] Add to_json in FunctionRegistry

2017-03-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16981 **[Test build #73815 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73815/testReport)** for PR 16981 at commit

[GitHub] spark issue #16910: [SPARK-19575][SQL]Reading from or writing to a hive serd...

2017-03-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16910 **[Test build #73829 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73829/testReport)** for PR 16910 at commit

[GitHub] spark pull request #17081: [SPARK-18726][SQL]resolveRelation for FileFormat ...

2017-03-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17081 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #17081: [SPARK-18726][SQL]resolveRelation for FileFormat DataSou...

2017-03-02 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17081 thanks, merging to master! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #17096: [SPARK-15243][ML][SQL][PYTHON] Add missing support for u...

2017-03-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17096 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73822/ Test PASSed. ---

[GitHub] spark issue #17096: [SPARK-15243][ML][SQL][PYTHON] Add missing support for u...

2017-03-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17096 **[Test build #73822 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73822/testReport)** for PR 17096 at commit

[GitHub] spark issue #17096: [SPARK-15243][ML][SQL][PYTHON] Add missing support for u...

2017-03-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17096 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17147: [Minor][Doc] Fix doc for web UI https configuration

2017-03-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17147 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17147: [Minor][Doc] Fix doc for web UI https configuration

2017-03-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17147 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73826/ Test PASSed. ---

[GitHub] spark issue #17147: [Minor][Doc] Fix doc for web UI https configuration

2017-03-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17147 **[Test build #73826 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73826/testReport)** for PR 17147 at commit

[GitHub] spark issue #16944: [SPARK-19611][SQL] Introduce configurable table schema i...

2017-03-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16944 **[Test build #73828 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73828/testReport)** for PR 16944 at commit

[GitHub] spark issue #17001: [SPARK-19667][SQL]create table with hiveenabled in defau...

2017-03-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17001 **[Test build #73827 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73827/testReport)** for PR 17001 at commit

[GitHub] spark issue #17096: [SPARK-15243][ML][SQL][PYTHON] Add missing support for u...

2017-03-02 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/17096 @holdenk and @viirya, I got rid of the changes in `types.py` and only left that I am pretty sure. There are two kind of changes here that look used in the only local scope. One

[GitHub] spark issue #17136: [SPARK-19783][SQL] Treat shorter/longer lengths of token...

2017-03-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17136 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73816/ Test FAILed. ---

[GitHub] spark issue #17136: [SPARK-19783][SQL] Treat shorter/longer lengths of token...

2017-03-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17136 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17136: [SPARK-19783][SQL] Treat shorter/longer lengths of token...

2017-03-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17136 **[Test build #73816 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73816/testReport)** for PR 17136 at commit

[GitHub] spark issue #17122: [SPARK-19786][SQL] Facilitate loop optimizations in a JI...

2017-03-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17122 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17122: [SPARK-19786][SQL] Facilitate loop optimizations in a JI...

2017-03-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17122 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73813/ Test PASSed. ---

[GitHub] spark issue #17147: [Minor][Doc] Fix doc for web UI https configuration

2017-03-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17147 **[Test build #73826 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73826/testReport)** for PR 17147 at commit

[GitHub] spark issue #17122: [SPARK-19786][SQL] Facilitate loop optimizations in a JI...

2017-03-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17122 **[Test build #73813 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73813/testReport)** for PR 17122 at commit

[GitHub] spark issue #16944: [SPARK-19611][SQL] Introduce configurable table schema i...

2017-03-02 Thread budde
Github user budde commented on the issue: https://github.com/apache/spark/pull/16944 Retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark pull request #17001: [SPARK-19667][SQL]create table with hiveenabled i...

2017-03-02 Thread windpiger
Github user windpiger commented on a diff in the pull request: https://github.com/apache/spark/pull/17001#discussion_r104101806 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveSparkSubmitSuite.scala --- @@ -905,3 +934,91 @@ object SPARK_18989_DESC_TABLE {

[GitHub] spark pull request #17147: [Minor][Doc] Fix doc for web UI https configurati...

2017-03-02 Thread jerryshao
GitHub user jerryshao opened a pull request: https://github.com/apache/spark/pull/17147 [Minor][Doc] Fix doc for web UI https configuration ## What changes were proposed in this pull request? Doc about enabling web UI https is not correct, "spark.ui.https.enabled" is not

[GitHub] spark issue #17145: [SPARK-19805][TEST] Log the row type when query result d...

2017-03-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17145 **[Test build #73825 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73825/testReport)** for PR 17145 at commit

[GitHub] spark pull request #16696: [SPARK-19350] [SQL] Cardinality estimation of Lim...

2017-03-02 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16696#discussion_r104101253 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/StatisticsCollectionSuite.scala --- @@ -116,22 +116,22 @@ class StatisticsCollectionSuite extends

[GitHub] spark issue #17145: [SPARK-19805][TEST] Log the row type when query type dos...

2017-03-02 Thread uncleGen
Github user uncleGen commented on the issue: https://github.com/apache/spark/pull/17145 unrelated failure: ` org.apache.spark.sql.kafka010.KafkaSourceStressForDontFailOnDataLossSuite.stress test for failOnDataLoss=false`. retest this please. --- If your project is set up for it,

[GitHub] spark pull request #16696: [SPARK-19350] [SQL] Cardinality estimation of Lim...

2017-03-02 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16696#discussion_r104101031 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/statsEstimation/StatsEstimationSuite.scala --- @@ -0,0 +1,121 @@ +/* + *

[GitHub] spark issue #17094: [SPARK-19762][ML] Hierarchy for consolidating ML aggrega...

2017-03-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17094 **[Test build #73823 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73823/testReport)** for PR 17094 at commit

[GitHub] spark issue #16696: [SPARK-19350] [SQL] Cardinality estimation of Limit and ...

2017-03-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16696 **[Test build #73824 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73824/testReport)** for PR 16696 at commit

[GitHub] spark issue #17096: [SPARK-15243][ML][SQL][PYTHON] Add missing support for u...

2017-03-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17096 **[Test build #73822 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73822/testReport)** for PR 17096 at commit

[GitHub] spark pull request #16696: [SPARK-19350] [SQL] Cardinality estimation of Lim...

2017-03-02 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16696#discussion_r104100931 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/statsEstimation/StatsConfSuite.scala --- @@ -1,64 +0,0 @@ -/* - * Licensed

[GitHub] spark issue #17135: SPARK-19794 Release HDFS Client after read/write checkpo...

2017-03-02 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/17135 I remember FileSystem will be cached internally by default. Closing it probably will introduce some performance regression. --- If your project is set up for it, you can reply to this email and

[GitHub] spark issue #17094: [SPARK-19762][ML] Hierarchy for consolidating ML aggrega...

2017-03-02 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/17094 Removed WIP, think it's ready now :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17145: [SPARK-19805][TEST] Log the row type when query type dos...

2017-03-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17145 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73817/ Test FAILed. ---

[GitHub] spark issue #17145: [SPARK-19805][TEST] Log the row type when query type dos...

2017-03-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17145 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17145: [SPARK-19805][TEST] Log the row type when query type dos...

2017-03-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17145 **[Test build #73817 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73817/testReport)** for PR 17145 at commit

[GitHub] spark issue #16696: [SPARK-19350] [SQL] Cardinality estimation of Limit and ...

2017-03-02 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/16696 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #17094: [SPARK-19762][ML] Hierarchy for consolidating ML aggrega...

2017-03-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17094 **[Test build #73821 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73821/testReport)** for PR 17094 at commit

[GitHub] spark issue #17094: [SPARK-19762][ML] Hierarchy for consolidating ML aggrega...

2017-03-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17094 **[Test build #73820 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73820/testReport)** for PR 17094 at commit

[GitHub] spark issue #17094: [SPARK-19762][ML] Hierarchy for consolidating ML aggrega...

2017-03-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17094 **[Test build #73819 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73819/testReport)** for PR 17094 at commit

[GitHub] spark issue #15505: [SPARK-18890][CORE] Move task serialization from the Tas...

2017-03-02 Thread witgo
Github user witgo commented on the issue: https://github.com/apache/spark/pull/15505 [SPARK-18890_20170303](https://github.com/witgo/spark/commits/SPARK-18890_20170303) `s code is older but the test case running time is 5.2 s --- If your project is set up for it, you can reply to

[GitHub] spark issue #17096: [SPARK-15243][ML][SQL][PYTHON] Add missing support for u...

2017-03-02 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/17096 Let me check if each is fine for sure. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17096: [SPARK-15243][ML][SQL][PYTHON] Add missing support for u...

2017-03-02 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/17096 @viirya, thank you so much for taking a look and your time. So, basically, the second case it compares str to unicode as below: ```python >>> u"測試" ==

[GitHub] spark pull request #17065: [SPARK-17075][SQL][followup] fix some minor issue...

2017-03-02 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/17065#discussion_r104098256 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/FilterEstimation.scala --- @@ -95,15 +84,16 @@ case class

[GitHub] spark issue #16981: [SPARK-19637][SQL] Add to_json in FunctionRegistry

2017-03-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16981 **[Test build #73818 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73818/testReport)** for PR 16981 at commit

[GitHub] spark issue #16981: [SPARK-19637][SQL] Add to_json in FunctionRegistry

2017-03-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16981 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73809/ Test PASSed. ---

[GitHub] spark issue #16981: [SPARK-19637][SQL] Add to_json in FunctionRegistry

2017-03-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16981 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16981: [SPARK-19637][SQL] Add to_json in FunctionRegistry

2017-03-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16981 **[Test build #73809 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73809/testReport)** for PR 16981 at commit

[GitHub] spark issue #17074: [SPARK-18646][REPL] Set parent classloader as null for E...

2017-03-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17074 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73805/ Test FAILed. ---

[GitHub] spark issue #17074: [SPARK-18646][REPL] Set parent classloader as null for E...

2017-03-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17074 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #14789: [SPARK-17209][YARN] Add the ability to manually u...

2017-03-02 Thread jerryshao
Github user jerryshao closed the pull request at: https://github.com/apache/spark/pull/14789 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #17095: [SPARK-19763][SQL]qualified external datasource t...

2017-03-02 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17095#discussion_r104095925 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLSuite.scala --- @@ -1843,10 +1843,12 @@ class DDLSuite extends QueryTest

[GitHub] spark issue #14731: [SPARK-17159] [streaming]: optimise check for new files ...

2017-03-02 Thread uncleGen
Github user uncleGen commented on the issue: https://github.com/apache/spark/pull/14731 @srowen Waiting for your final OK --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #17117: [SPARK-10780][ML] Support initial model for KMean...

2017-03-02 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/17117#discussion_r104084997 --- Diff: mllib/src/main/scala/org/apache/spark/ml/clustering/KMeans.scala --- @@ -253,7 +255,18 @@ object KMeansModel extends MLReadable[KMeansModel] {

[GitHub] spark pull request #17117: [SPARK-10780][ML] Support initial model for KMean...

2017-03-02 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/17117#discussion_r104084877 --- Diff: mllib/src/main/scala/org/apache/spark/ml/clustering/KMeans.scala --- @@ -123,7 +126,8 @@ class KMeansModel private[ml] ( @Since("2.0.0")

[GitHub] spark pull request #17117: [SPARK-10780][ML] Support initial model for KMean...

2017-03-02 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/17117#discussion_r104095197 --- Diff: mllib/src/test/scala/org/apache/spark/ml/clustering/KMeansSuite.scala --- @@ -182,6 +224,7 @@ object KMeansSuite { "predictionCol" ->

[GitHub] spark pull request #17117: [SPARK-10780][ML] Support initial model for KMean...

2017-03-02 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/17117#discussion_r104091867 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/clustering/KMeans.scala --- @@ -418,6 +418,8 @@ object KMeans { val RANDOM = "random"

[GitHub] spark pull request #17117: [SPARK-10780][ML] Support initial model for KMean...

2017-03-02 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/17117#discussion_r104092158 --- Diff: mllib/src/test/scala/org/apache/spark/ml/clustering/KMeansSuite.scala --- @@ -22,22 +22,28 @@ import scala.util.Random import

[GitHub] spark pull request #17117: [SPARK-10780][ML] Support initial model for KMean...

2017-03-02 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/17117#discussion_r104090529 --- Diff: mllib/src/main/scala/org/apache/spark/ml/clustering/KMeans.scala --- @@ -337,15 +366,61 @@ class KMeans @Since("1.5.0") (

[GitHub] spark pull request #17117: [SPARK-10780][ML] Support initial model for KMean...

2017-03-02 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/17117#discussion_r104094526 --- Diff: mllib/src/test/scala/org/apache/spark/ml/util/DefaultReadWriteTest.scala --- @@ -111,12 +113,20 @@ trait DefaultReadWriteTest extends

[GitHub] spark pull request #17117: [SPARK-10780][ML] Support initial model for KMean...

2017-03-02 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/17117#discussion_r104090273 --- Diff: mllib/src/main/scala/org/apache/spark/ml/clustering/KMeans.scala --- @@ -337,15 +366,61 @@ class KMeans @Since("1.5.0") (

[GitHub] spark pull request #17117: [SPARK-10780][ML] Support initial model for KMean...

2017-03-02 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/17117#discussion_r104092773 --- Diff: mllib/src/test/scala/org/apache/spark/ml/clustering/KMeansSuite.scala --- @@ -152,6 +158,35 @@ class KMeansSuite extends SparkFunSuite with

[GitHub] spark issue #15505: [SPARK-18890][CORE] Move task serialization from the Tas...

2017-03-02 Thread witgo
Github user witgo commented on the issue: https://github.com/apache/spark/pull/15505 Yes, maybe a multithreaded serialization task code can have a better performance, let me close the PR --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request #15505: [SPARK-18890][CORE] Move task serialization from ...

2017-03-02 Thread witgo
Github user witgo closed the pull request at: https://github.com/apache/spark/pull/15505 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #17133: [SPARK-19793] Use clock.getTimeMillis when mark task as ...

2017-03-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17133 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73807/ Test FAILed. ---

[GitHub] spark issue #17133: [SPARK-19793] Use clock.getTimeMillis when mark task as ...

2017-03-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17133 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17133: [SPARK-19793] Use clock.getTimeMillis when mark task as ...

2017-03-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17133 **[Test build #73807 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73807/testReport)** for PR 17133 at commit

[GitHub] spark issue #17067: [SPARK-19602][SQL][TESTS] Add tests for qualified column...

2017-03-02 Thread skambha
Github user skambha commented on the issue: https://github.com/apache/spark/pull/17067 Thanks a lot Xiao. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #16883: [SPARK-17498][ML] StringIndexer enhancement for handling...

2017-03-02 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/16883 @VinceShieh I added some minor comments. This is a nice feature! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request #16883: [SPARK-17498][ML] StringIndexer enhancement for h...

2017-03-02 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/16883#discussion_r104094424 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/StringIndexer.scala --- @@ -163,25 +187,28 @@ class StringIndexerModel ( }

[GitHub] spark pull request #16883: [SPARK-17498][ML] StringIndexer enhancement for h...

2017-03-02 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/16883#discussion_r104093892 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/StringIndexer.scala --- @@ -105,7 +125,11 @@ class StringIndexer @Since("1.4.0") (

[GitHub] spark issue #13320: [SPARK-13184][SQL] Add a datasource-specific option minP...

2017-03-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/13320 @gatorsmile Could you check this and give me comments, too? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request #16883: [SPARK-17498][ML] StringIndexer enhancement for h...

2017-03-02 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/16883#discussion_r104093629 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/StringIndexer.scala --- @@ -163,25 +187,28 @@ class StringIndexerModel ( }

[GitHub] spark issue #17145: [SPARK-19805][TEST] Log the row type when query type dos...

2017-03-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17145 **[Test build #73817 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73817/testReport)** for PR 17145 at commit

[GitHub] spark pull request #16883: [SPARK-17498][ML] StringIndexer enhancement for h...

2017-03-02 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/16883#discussion_r104093452 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/StringIndexer.scala --- @@ -163,25 +190,28 @@ class StringIndexerModel ( }

[GitHub] spark pull request #16883: [SPARK-17498][ML] StringIndexer enhancement for h...

2017-03-02 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/16883#discussion_r104093159 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/StringIndexer.scala --- @@ -105,7 +125,11 @@ class StringIndexer @Since("1.4.0") (

[GitHub] spark pull request #16883: [SPARK-17498][ML] StringIndexer enhancement for h...

2017-03-02 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/16883#discussion_r104093069 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/StringIndexer.scala --- @@ -71,18 +92,17 @@ class StringIndexer @Since("1.4.0") (

[GitHub] spark pull request #16883: [SPARK-17498][ML] StringIndexer enhancement for h...

2017-03-02 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/16883#discussion_r104092772 --- Diff: docs/ml-features.md --- @@ -576,7 +579,22 @@ will be generated: 2 | c| 1.0 -Notice that the row

[GitHub] spark issue #17136: [SPARK-19783][SQL] Treat shorter/longer lengths of token...

2017-03-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17136 **[Test build #73816 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73816/testReport)** for PR 17136 at commit

[GitHub] spark pull request #16883: [SPARK-17498][ML] StringIndexer enhancement for h...

2017-03-02 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/16883#discussion_r104092723 --- Diff: docs/ml-features.md --- @@ -576,7 +579,22 @@ will be generated: 2 | c| 1.0 -Notice that the row

[GitHub] spark issue #16944: [SPARK-19611][SQL] Introduce configurable table schema i...

2017-03-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16944 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #16883: [SPARK-17498][ML] StringIndexer enhancement for h...

2017-03-02 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/16883#discussion_r104092627 --- Diff: docs/ml-features.md --- @@ -542,12 +543,13 @@ column, we should get the following: "a" gets index `0` because it is the most frequent,

[GitHub] spark issue #16944: [SPARK-19611][SQL] Introduce configurable table schema i...

2017-03-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16944 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73808/ Test FAILed. ---

[GitHub] spark issue #15928: [SPARK-18478][SQL] Support codegen'd Hive UDFs

2017-03-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/15928 @rxin yea, I got x1.3-1.4 performance gains in this pr. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #16944: [SPARK-19611][SQL] Introduce configurable table schema i...

2017-03-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16944 **[Test build #73808 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73808/testReport)** for PR 16944 at commit

[GitHub] spark issue #15928: [SPARK-18478][SQL] Support codegen'd Hive UDFs

2017-03-02 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15928 What do you mean? The improvement was small? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17136: [SPARK-19783][SQL] Treat shorter/longer lengths of token...

2017-03-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/17136 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #15928: [SPARK-18478][SQL] Support codegen'd Hive UDFs

2017-03-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/15928 I looked into this though, I got a little luck from this fix. So, I'll close for now. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request #15928: [SPARK-18478][SQL] Support codegen'd Hive UDFs

2017-03-02 Thread maropu
Github user maropu closed the pull request at: https://github.com/apache/spark/pull/15928 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #17140: [SPARK-19796][CORE] Fix serialization of long property v...

2017-03-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17140 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73802/ Test PASSed. ---

[GitHub] spark issue #17140: [SPARK-19796][CORE] Fix serialization of long property v...

2017-03-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17140 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16981: [SPARK-19637][SQL] Add to_json in FunctionRegistry

2017-03-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/16981 @gatorsmile okay, I'll fix the issues you mentioned. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #16981: [SPARK-19637][SQL] Add to_json in FunctionRegistry

2017-03-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16981 **[Test build #73815 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73815/testReport)** for PR 16981 at commit

[GitHub] spark issue #17140: [SPARK-19796][CORE] Fix serialization of long property v...

2017-03-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17140 **[Test build #73802 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73802/testReport)** for PR 17140 at commit

[GitHub] spark pull request #17122: [SPARK-19786][SQL] Facilitate loop optimizations ...

2017-03-02 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/17122#discussion_r104091814 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/WholeStageCodegenExec.scala --- @@ -206,6 +206,18 @@ trait CodegenSupport extends SparkPlan

[GitHub] spark pull request #16981: [SPARK-19637][SQL] Add to_json in FunctionRegistr...

2017-03-02 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/16981#discussion_r104091757 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonUtils.scala --- @@ -55,4 +60,22 @@ object JacksonUtils {

[GitHub] spark pull request #16981: [SPARK-19637][SQL] Add to_json in FunctionRegistr...

2017-03-02 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/16981#discussion_r104091471 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonUtils.scala --- @@ -55,4 +60,22 @@ object JacksonUtils {

[GitHub] spark pull request #16981: [SPARK-19637][SQL] Add to_json in FunctionRegistr...

2017-03-02 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/16981#discussion_r104091422 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala --- @@ -3007,7 +3008,7 @@ object functions { * @since 2.1.0 */

[GitHub] spark issue #17144: [SPARK-19803][TEST] flaky BlockManagerReplicationSuite t...

2017-03-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17144 **[Test build #73814 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73814/testReport)** for PR 17144 at commit

[GitHub] spark pull request #16981: [SPARK-19637][SQL] Add to_json in FunctionRegistr...

2017-03-02 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16981#discussion_r104091265 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/JsonFunctionsSuite.scala --- @@ -174,4 +174,22 @@ class JsonFunctionsSuite extends QueryTest with

  1   2   3   4   5   6   7   >