[GitHub] spark pull request #20851: [SPARK-23727][SQL] Support for pushing down filte...

2018-03-20 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20851#discussion_r175970994 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilterSuite.scala --- @@ -313,6 +315,36 @@ class

[GitHub] spark issue #20433: [SPARK-23264][SQL] Make INTERVAL keyword optional in INT...

2018-03-20 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/20433 ok, I'll do later. thanks. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20433: [SPARK-23264][SQL] Make INTERVAL keyword optional in INT...

2018-03-20 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20433 yea very likely. Do you mind open a new PR to fix it? thanks! --- - To unsubscribe, e-mail:

[GitHub] spark pull request #20786: [SPARK-14681][ML] Provide label/impurity stats fo...

2018-03-20 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/20786#discussion_r175970711 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/Node.scala --- @@ -84,35 +86,73 @@ private[ml] object Node { /** * Create a new

[GitHub] spark pull request #20633: [SPARK-23455][ML] Default Params in ML should be ...

2018-03-20 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/20633#discussion_r175965924 --- Diff: mllib/src/test/scala/org/apache/spark/ml/util/DefaultReadWriteTest.scala --- @@ -169,4 +179,54 @@ class DefaultReadWriteSuite extends

[GitHub] spark pull request #20633: [SPARK-23455][ML] Default Params in ML should be ...

2018-03-20 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/20633#discussion_r175965272 --- Diff: mllib/src/main/scala/org/apache/spark/ml/util/ReadWrite.scala --- @@ -351,27 +359,88 @@ private[ml] object DefaultParamsReader {

[GitHub] spark pull request #20633: [SPARK-23455][ML] Default Params in ML should be ...

2018-03-20 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/20633#discussion_r175961795 --- Diff: mllib/src/main/scala/org/apache/spark/ml/param/params.scala --- @@ -791,7 +791,7 @@ trait Params extends Identifiable with Serializable {

[GitHub] spark pull request #20633: [SPARK-23455][ML] Default Params in ML should be ...

2018-03-20 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/20633#discussion_r175966231 --- Diff: mllib/src/test/scala/org/apache/spark/ml/util/DefaultReadWriteTest.scala --- @@ -169,4 +179,54 @@ class DefaultReadWriteSuite extends

[GitHub] spark pull request #20633: [SPARK-23455][ML] Default Params in ML should be ...

2018-03-20 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/20633#discussion_r175963464 --- Diff: mllib/src/main/scala/org/apache/spark/ml/util/ReadWrite.scala --- @@ -351,27 +359,88 @@ private[ml] object DefaultParamsReader {

[GitHub] spark pull request #20633: [SPARK-23455][ML] Default Params in ML should be ...

2018-03-20 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/20633#discussion_r175962188 --- Diff: mllib/src/main/scala/org/apache/spark/ml/util/ReadWrite.scala --- @@ -296,14 +297,19 @@ private[ml] object DefaultParamsWriter {

[GitHub] spark pull request #20844: [SPARK-23707][SQL] Fresh 'initRange' name to avoi...

2018-03-20 Thread ConeyLiu
Github user ConeyLiu commented on a diff in the pull request: https://github.com/apache/spark/pull/20844#discussion_r175966108 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/basicPhysicalOperators.scala --- @@ -396,9 +396,11 @@ case class RangeExec(range:

[GitHub] spark issue #20827: [SPARK-23666][SQL] Do not display exprIds of Alias in us...

2018-03-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20827 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1655/

[GitHub] spark issue #20827: [SPARK-23666][SQL] Do not display exprIds of Alias in us...

2018-03-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20827 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20827: [SPARK-23666][SQL] Do not display exprIds of Alias in us...

2018-03-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20827 **[Test build #88439 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88439/testReport)** for PR 20827 at commit

[GitHub] spark pull request #20827: [SPARK-23666][SQL] Do not display exprIds of Alia...

2018-03-20 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/20827#discussion_r175964611 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/UDFSuite.scala --- @@ -304,4 +304,14 @@ class UDFSuite extends QueryTest with SharedSQLContext {

[GitHub] spark pull request #20827: [SPARK-23666][SQL] Do not display exprIds of Alia...

2018-03-20 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/20827#discussion_r175963705 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Expression.scala --- @@ -288,6 +288,7 @@ trait NonSQLExpression extends

[GitHub] spark issue #20867: Spark 23759

2018-03-20 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20867 Please update the title. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20433: [SPARK-23264][SQL] Make INTERVAL keyword optional in INT...

2018-03-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20433 **[Test build #88438 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88438/testReport)** for PR 20433 at commit

[GitHub] spark issue #20433: [SPARK-23264][SQL] Make INTERVAL keyword optional in INT...

2018-03-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20433 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1654/

[GitHub] spark issue #20433: [SPARK-23264][SQL] Make INTERVAL keyword optional in INT...

2018-03-20 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/20433 I was thinking we added a temporary rule in `SqlBase.g`: https://github.com/apache/spark/pull/20433/commits/5ee6f897bc71eac24e086f39549ef3a396059b4d But, I have no strong reason to add this in

[GitHub] spark issue #20433: [SPARK-23264][SQL] Make INTERVAL keyword optional in INT...

2018-03-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20433 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20851: [SPARK-23727][SQL] Support for pushing down filte...

2018-03-20 Thread yucai
Github user yucai commented on a diff in the pull request: https://github.com/apache/spark/pull/20851#discussion_r175962888 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilterSuite.scala --- @@ -313,6 +315,36 @@ class

[GitHub] spark issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle...

2018-03-20 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/20345 @wzhfy Thanks for the review and I'll update in days! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #20786: [SPARK-14681][ML] Provide label/impurity stats fo...

2018-03-20 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/20786#discussion_r175957326 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/Node.scala --- @@ -84,35 +86,73 @@ private[ml] object Node { /** * Create a new

[GitHub] spark pull request #20786: [SPARK-14681][ML] Provide label/impurity stats fo...

2018-03-20 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/20786#discussion_r175959353 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/treeModels.scala --- @@ -212,15 +212,13 @@ private[ml] object TreeEnsembleModel { def

[GitHub] spark pull request #20786: [SPARK-14681][ML] Provide label/impurity stats fo...

2018-03-20 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/20786#discussion_r175957767 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/Node.scala --- @@ -135,32 +175,59 @@ class LeafNode private[ml] ( override

[GitHub] spark pull request #20786: [SPARK-14681][ML] Provide label/impurity stats fo...

2018-03-20 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/20786#discussion_r175955388 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/Node.scala --- @@ -84,35 +86,73 @@ private[ml] object Node { /** * Create a new

[GitHub] spark pull request #20786: [SPARK-14681][ML] Provide label/impurity stats fo...

2018-03-20 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/20786#discussion_r175958393 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/Node.scala --- @@ -240,6 +302,69 @@ private object InternalNode { } } +/**

[GitHub] spark pull request #20786: [SPARK-14681][ML] Provide label/impurity stats fo...

2018-03-20 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/20786#discussion_r175957538 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/Node.scala --- @@ -84,35 +86,73 @@ private[ml] object Node { /** * Create a new

[GitHub] spark pull request #20786: [SPARK-14681][ML] Provide label/impurity stats fo...

2018-03-20 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/20786#discussion_r175957188 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/Node.scala --- @@ -84,35 +86,73 @@ private[ml] object Node { /** * Create a new

[GitHub] spark pull request #20786: [SPARK-14681][ML] Provide label/impurity stats fo...

2018-03-20 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/20786#discussion_r175959372 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/treeModels.scala --- @@ -306,17 +304,21 @@ private[ml] object DecisionTreeModelReadWrite {

[GitHub] spark pull request #20786: [SPARK-14681][ML] Provide label/impurity stats fo...

2018-03-20 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/20786#discussion_r175957487 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/Node.scala --- @@ -84,35 +86,73 @@ private[ml] object Node { /** * Create a new

[GitHub] spark pull request #20786: [SPARK-14681][ML] Provide label/impurity stats fo...

2018-03-20 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/20786#discussion_r175957245 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/Node.scala --- @@ -84,35 +86,73 @@ private[ml] object Node { /** * Create a new

[GitHub] spark pull request #20786: [SPARK-14681][ML] Provide label/impurity stats fo...

2018-03-20 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/20786#discussion_r175956585 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/DecisionTreeClassifier.scala --- @@ -276,8 +276,9 @@ object

[GitHub] spark pull request #20786: [SPARK-14681][ML] Provide label/impurity stats fo...

2018-03-20 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/20786#discussion_r175957437 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/Node.scala --- @@ -84,35 +86,73 @@ private[ml] object Node { /** * Create a new

[GitHub] spark issue #20868: [SPARK-23750][SQL] Inner Join Elimination based on Infor...

2018-03-20 Thread ioana-delaney
Github user ioana-delaney commented on the issue: https://github.com/apache/spark/pull/20868 Note to the reviewers: This performance PR contains two commits: (1) dependent DDL changes from SPARK-21784 and (2) the actual rewrite changes. The DDL changes should be reviewed as part of

[GitHub] spark issue #20868: [SPARK-23750][SQL] Inner Join Elimination based on Infor...

2018-03-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20868 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1653/

[GitHub] spark issue #20868: [SPARK-23750][SQL] Inner Join Elimination based on Infor...

2018-03-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20868 **[Test build #88437 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88437/testReport)** for PR 20868 at commit

[GitHub] spark issue #20868: [SPARK-23750][SQL] Inner Join Elimination based on Infor...

2018-03-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20868 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20868: [SPARK-23750][SQL] Inner Join Elimination based o...

2018-03-20 Thread ioana-delaney
GitHub user ioana-delaney opened a pull request: https://github.com/apache/spark/pull/20868 [SPARK-23750][SQL] Inner Join Elimination based on Informational RI constraints ## What changes were proposed in this pull request? This transformation detects RI joins and

[GitHub] spark issue #20777: [SPARK-23615][ML][PYSPARK]Add maxDF Parameter to Python ...

2018-03-20 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/20777 @huaxingao , it's best to keep the same PR if possible to better preserve the discussion history. Could you give it another try to resolve conflicts? ---

[GitHub] spark pull request #20786: [SPARK-14681][ML] Provide label/impurity stats fo...

2018-03-20 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/20786#discussion_r175955065 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/Node.scala --- @@ -84,35 +86,73 @@ private[ml] object Node { /** * Create a new

[GitHub] spark issue #19222: [SPARK-10399][CORE][SQL] Introduce multiple MemoryBlocks...

2018-03-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19222 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88435/ Test PASSed. ---

[GitHub] spark issue #19222: [SPARK-10399][CORE][SQL] Introduce multiple MemoryBlocks...

2018-03-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19222 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19222: [SPARK-10399][CORE][SQL] Introduce multiple MemoryBlocks...

2018-03-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19222 **[Test build #88435 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88435/testReport)** for PR 19222 at commit

[GitHub] spark pull request #20579: [SPARK-23372][SQL] Writing empty struct in parque...

2018-03-20 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/20579#discussion_r175952404 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFileFormat.scala --- @@ -77,7 +77,6 @@ class

[GitHub] spark pull request #20579: [SPARK-23372][SQL] Writing empty struct in parque...

2018-03-20 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/20579#discussion_r175952408 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala --- @@ -719,4 +720,27 @@ object DataSource extends

[GitHub] spark issue #20579: [SPARK-23372][SQL] Writing empty struct in parquet fails...

2018-03-20 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/20579 @cloud-fan ok. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20831: [SPARK-23614][SQL] Fix incorrect reuse exchange when cac...

2018-03-20 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20831 good catch! LGTM except a few comments --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20831: [SPARK-23614][SQL] Fix incorrect reuse exchange w...

2018-03-20 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20831#discussion_r175951728 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/InMemoryTableScanExec.scala --- @@ -169,7 +174,10 @@ case class

[GitHub] spark pull request #20831: [SPARK-23614][SQL] Fix incorrect reuse exchange w...

2018-03-20 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20831#discussion_r175951487 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/InMemoryRelation.scala --- @@ -68,6 +69,15 @@ case class InMemoryRelation(

[GitHub] spark issue #20774: [SPARK-23549][SQL] Cast to timestamp when comparing time...

2018-03-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20774 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20774: [SPARK-23549][SQL] Cast to timestamp when comparing time...

2018-03-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20774 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88434/ Test PASSed. ---

[GitHub] spark issue #20579: [SPARK-23372][SQL] Writing empty struct in parquet fails...

2018-03-20 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20579 LGTM, can we add something to the migration guide? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #20774: [SPARK-23549][SQL] Cast to timestamp when comparing time...

2018-03-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20774 **[Test build #88434 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88434/testReport)** for PR 20774 at commit

[GitHub] spark pull request #20579: [SPARK-23372][SQL] Writing empty struct in parque...

2018-03-20 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20579#discussion_r175950101 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFileFormat.scala --- @@ -77,7 +77,6 @@ class

[GitHub] spark pull request #20579: [SPARK-23372][SQL] Writing empty struct in parque...

2018-03-20 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20579#discussion_r175950061 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala --- @@ -719,4 +720,27 @@ object DataSource extends Logging

[GitHub] spark issue #20867: Spark 23759

2018-03-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20867 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20867: Spark 23759

2018-03-20 Thread felixalbani
GitHub user felixalbani reopened a pull request: https://github.com/apache/spark/pull/20867 Spark 23759 ## What changes were proposed in this pull request? This pull is to fix SPARK-23759 issue Problem was created due connector.setHost(hostName) call was after

[GitHub] spark pull request #20867: Spark 23759

2018-03-20 Thread felixalbani
Github user felixalbani closed the pull request at: https://github.com/apache/spark/pull/20867 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20867: Spark 23759

2018-03-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20867 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20867: Spark 23759

2018-03-20 Thread felixalbani
GitHub user felixalbani opened a pull request: https://github.com/apache/spark/pull/20867 Spark 23759 ## What changes were proposed in this pull request? This pull is to fix SPARK-23759 issue Problem was created due connector.setHost(hostName) call was after

[GitHub] spark issue #20579: [SPARK-23372][SQL] Writing empty struct in parquet fails...

2018-03-20 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/20579 @cloud-fan Thank you. I assumed (wrongly) that we don't want to change the behaviour for an external file based datasource and we wanted to scope our check only to spark built in data sources.

[GitHub] spark issue #20579: [SPARK-23372][SQL] Writing empty struct in parquet fails...

2018-03-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20579 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20579: [SPARK-23372][SQL] Writing empty struct in parquet fails...

2018-03-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20579 **[Test build #88436 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88436/testReport)** for PR 20579 at commit

[GitHub] spark issue #20579: [SPARK-23372][SQL] Writing empty struct in parquet fails...

2018-03-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20579 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1652/

[GitHub] spark issue #19381: [SPARK-10884][ML] Support prediction on single instance ...

2018-03-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19381 **[Test build #4142 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4142/testReport)** for PR 19381 at commit

[GitHub] spark issue #19381: [SPARK-10884][ML] Support prediction on single instance ...

2018-03-20 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/19381 LGTM Merging after fresh tests run Thanks a lot @WeichenXu123 for the PR and others for reviews! --- - To unsubscribe,

[GitHub] spark issue #20774: [SPARK-23549][SQL] Cast to timestamp when comparing time...

2018-03-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20774 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88433/ Test PASSed. ---

[GitHub] spark issue #20774: [SPARK-23549][SQL] Cast to timestamp when comparing time...

2018-03-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20774 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20774: [SPARK-23549][SQL] Cast to timestamp when comparing time...

2018-03-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20774 **[Test build #88433 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88433/testReport)** for PR 20774 at commit

[GitHub] spark pull request #20327: [SPARK-12963][CORE] NM host for driver end points

2018-03-20 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20327#discussion_r175938595 --- Diff: core/src/main/scala/org/apache/spark/ui/WebUI.scala --- @@ -126,7 +126,11 @@ private[spark] abstract class WebUI( def bind(): Unit = {

[GitHub] spark issue #20859: [SPARK-23702][SS] Forbid watermarks on both sides of sta...

2018-03-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20859 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88432/ Test PASSed. ---

[GitHub] spark issue #20859: [SPARK-23702][SS] Forbid watermarks on both sides of sta...

2018-03-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20859 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20859: [SPARK-23702][SS] Forbid watermarks on both sides of sta...

2018-03-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20859 **[Test build #88432 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88432/testReport)** for PR 20859 at commit

[GitHub] spark issue #20798: [SPARK-23645][MINOR][DOCS][PYTHON] Add docs RE `pandas_u...

2018-03-20 Thread mstewart141
Github user mstewart141 commented on the issue: https://github.com/apache/spark/pull/20798 all that makes sense; i will update. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20327: [SPARK-12963][CORE] NM host for driver end points

2018-03-20 Thread gerashegalov
Github user gerashegalov commented on a diff in the pull request: https://github.com/apache/spark/pull/20327#discussion_r175934086 --- Diff: core/src/main/scala/org/apache/spark/ui/WebUI.scala --- @@ -126,7 +126,11 @@ private[spark] abstract class WebUI( def bind(): Unit = {

[GitHub] spark issue #20777: [SPARK-23615][ML][PYSPARK]Add maxDF Parameter to Python ...

2018-03-20 Thread huaxingao
Github user huaxingao commented on the issue: https://github.com/apache/spark/pull/20777 @BryanCutler Do you mind if I close this PR and open a new one? I got problems when I tried to resolve the conflicts. --- -

[GitHub] spark issue #20866: [SPARK-23749][SQL] Avoid Hive.get() to compatible with d...

2018-03-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20866 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20866: [SPARK-23749][SQL] Avoid Hive.get() to compatible with d...

2018-03-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20866 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88428/ Test PASSed. ---

[GitHub] spark issue #20866: [SPARK-23749][SQL] Avoid Hive.get() to compatible with d...

2018-03-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20866 **[Test build #88428 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88428/testReport)** for PR 20866 at commit

[GitHub] spark issue #19222: [SPARK-10399][CORE][SQL] Introduce multiple MemoryBlocks...

2018-03-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19222 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88431/ Test FAILed. ---

[GitHub] spark issue #19222: [SPARK-10399][CORE][SQL] Introduce multiple MemoryBlocks...

2018-03-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19222 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19222: [SPARK-10399][CORE][SQL] Introduce multiple MemoryBlocks...

2018-03-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19222 **[Test build #88431 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88431/testReport)** for PR 19222 at commit

[GitHub] spark issue #20853: [SPARK-23729][CORE] Respect URI fragment when resolving ...

2018-03-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20853 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88429/ Test PASSed. ---

[GitHub] spark issue #20853: [SPARK-23729][CORE] Respect URI fragment when resolving ...

2018-03-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20853 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19876: [ML][SPARK-11171][SPARK-11239] Add PMML export to Spark ...

2018-03-20 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/19876 Will do a last pass myself and merge on Friday if no one else has opinions. --- - To unsubscribe, e-mail:

[GitHub] spark issue #20853: [SPARK-23729][CORE] Respect URI fragment when resolving ...

2018-03-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20853 **[Test build #88429 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88429/testReport)** for PR 20853 at commit

[GitHub] spark issue #20853: [SPARK-23729][CORE] Respect URI fragment when resolving ...

2018-03-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20853 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88430/ Test PASSed. ---

[GitHub] spark issue #20853: [SPARK-23729][CORE] Respect URI fragment when resolving ...

2018-03-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20853 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20853: [SPARK-23729][CORE] Respect URI fragment when resolving ...

2018-03-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20853 **[Test build #88430 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88430/testReport)** for PR 20853 at commit

[GitHub] spark issue #20657: [SPARK-23361][yarn] Allow AM to restart after initial to...

2018-03-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20657 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88427/ Test PASSed. ---

[GitHub] spark issue #20657: [SPARK-23361][yarn] Allow AM to restart after initial to...

2018-03-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20657 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20657: [SPARK-23361][yarn] Allow AM to restart after initial to...

2018-03-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20657 **[Test build #88427 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88427/testReport)** for PR 20657 at commit

[GitHub] spark pull request #20829: [SPARK-23690][ML] Add handleinvalid to VectorAsse...

2018-03-20 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/20829#discussion_r175908732 --- Diff: mllib/src/test/scala/org/apache/spark/ml/feature/VectorAssemblerSuite.scala --- @@ -147,4 +149,72 @@ class VectorAssemblerSuite

[GitHub] spark pull request #20829: [SPARK-23690][ML] Add handleinvalid to VectorAsse...

2018-03-20 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/20829#discussion_r175907866 --- Diff: mllib/src/test/scala/org/apache/spark/ml/feature/VectorAssemblerSuite.scala --- @@ -37,24 +37,26 @@ class VectorAssemblerSuite

[GitHub] spark pull request #20829: [SPARK-23690][ML] Add handleinvalid to VectorAsse...

2018-03-20 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/20829#discussion_r175904796 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/StringIndexer.scala --- @@ -234,7 +234,7 @@ class StringIndexerModel ( val metadata =

[GitHub] spark pull request #20829: [SPARK-23690][ML] Add handleinvalid to VectorAsse...

2018-03-20 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/20829#discussion_r175913440 --- Diff: mllib/src/test/scala/org/apache/spark/ml/feature/VectorAssemblerSuite.scala --- @@ -147,4 +149,72 @@ class VectorAssemblerSuite

[GitHub] spark pull request #20829: [SPARK-23690][ML] Add handleinvalid to VectorAsse...

2018-03-20 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/20829#discussion_r175912462 --- Diff: mllib/src/test/scala/org/apache/spark/ml/feature/VectorAssemblerSuite.scala --- @@ -147,4 +149,72 @@ class VectorAssemblerSuite

[GitHub] spark pull request #20829: [SPARK-23690][ML] Add handleinvalid to VectorAsse...

2018-03-20 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/20829#discussion_r175910846 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/VectorAssembler.scala --- @@ -136,34 +172,88 @@ class VectorAssembler @Since("1.4.0")

<    1   2   3   4   5   >