[GitHub] spark pull request #21070: [SPARK-23972][BUILD][SQL] Update Parquet to 1.10....

2018-05-07 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21070#discussion_r186602846 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedPlainValuesReader.java --- @@ -63,115 +59,157 @@ public

[GitHub] spark issue #21263: [SPARK-24084][ThriftServer] Add job group id for sql sta...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21263 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90350/ Test PASSed. ---

[GitHub] spark issue #21263: [SPARK-24084][ThriftServer] Add job group id for sql sta...

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21263 **[Test build #90350 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90350/testReport)** for PR 21263 at commit

[GitHub] spark issue #21263: [SPARK-24084][ThriftServer] Add job group id for sql sta...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21263 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21129: [SPARK-7132][ML] Add fit with validation set to spark.ml...

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21129 **[Test build #4170 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4170/testReport)** for PR 21129 at commit

[GitHub] spark issue #21264: Branch 2.2

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21264 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21264: Branch 2.2

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21264 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21264: Branch 2.2

2018-05-07 Thread yotingting
GitHub user yotingting opened a pull request: https://github.com/apache/spark/pull/21264 Branch 2.2 ## What changes were proposed in this pull request? when i use yarn client mode, the spark task locked in the collect stage countDf.collect().map(_.getLong(0)).mkString.toLong

[GitHub] spark issue #21209: [SPARK-24141][CORE] Fix bug in CoarseGrainedSchedulerBac...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21209 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90343/ Test PASSed. ---

[GitHub] spark issue #21209: [SPARK-24141][CORE] Fix bug in CoarseGrainedSchedulerBac...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21209 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21209: [SPARK-24141][CORE] Fix bug in CoarseGrainedSchedulerBac...

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21209 **[Test build #90343 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90343/testReport)** for PR 21209 at commit

[GitHub] spark issue #20929: [SPARK-23772][SQL][WIP] Provide an option to ignore colu...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20929 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90345/ Test FAILed. ---

[GitHub] spark issue #20929: [SPARK-23772][SQL][WIP] Provide an option to ignore colu...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20929 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20929: [SPARK-23772][SQL][WIP] Provide an option to ignore colu...

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20929 **[Test build #90345 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90345/testReport)** for PR 20929 at commit

[GitHub] spark pull request #21257: [SPARK-24194] [SQL]HadoopFsRelation cannot overwr...

2018-05-07 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21257#discussion_r186600573 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/InsertIntoHadoopFsRelationCommand.scala --- @@ -207,9 +207,25 @@ case class

[GitHub] spark issue #21263: [SPARK-24084][ThriftServer] Add job group id for sql sta...

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21263 **[Test build #90350 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90350/testReport)** for PR 21263 at commit

[GitHub] spark pull request #21257: [SPARK-24194] [SQL]HadoopFsRelation cannot overwr...

2018-05-07 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21257#discussion_r186600352 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/InsertIntoHadoopFsRelationCommand.scala --- @@ -207,9 +207,25 @@ case class

[GitHub] spark issue #21263: [SPARK-24084][ThriftServer] Add job group id for sql sta...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21263 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21259: [SPARK-24112][SQL] Add `convertMetastoreTableProp...

2018-05-07 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21259#discussion_r186600116 --- Diff: docs/sql-programming-guide.md --- @@ -1812,6 +1812,8 @@ working with timestamps in `pandas_udf`s to get the best performance, see -

[GitHub] spark pull request #21263: [SPARK-24084][ThriftServer] Add job group id for ...

2018-05-07 Thread caneGuy
GitHub user caneGuy opened a pull request: https://github.com/apache/spark/pull/21263 [SPARK-24084][ThriftServer] Add job group id for sql statement through spark-sql ## What changes were proposed in this pull request? Add job group id for spark-sql mode ## How

[GitHub] spark pull request #21257: [SPARK-24194] [SQL]HadoopFsRelation cannot overwr...

2018-05-07 Thread zheh12
Github user zheh12 commented on a diff in the pull request: https://github.com/apache/spark/pull/21257#discussion_r186599760 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/InsertIntoHadoopFsRelationCommand.scala --- @@ -207,9 +207,25 @@ case class

[GitHub] spark issue #21118: SPARK-23325: Use InternalRow when reading with DataSourc...

2018-05-07 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21118 > If we did that, then only ProjectExec and FilterExec would need to support InternalRow, which they already do. This partially brings #10511 back, and we need to plan project and filter

[GitHub] spark issue #21211: [SPARK-24131][PYSPARK][Followup] Add majorMinorVersion A...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21211 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90348/ Test PASSed. ---

[GitHub] spark issue #21211: [SPARK-24131][PYSPARK][Followup] Add majorMinorVersion A...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21211 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21211: [SPARK-24131][PYSPARK][Followup] Add majorMinorVersion A...

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21211 **[Test build #90348 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90348/testReport)** for PR 21211 at commit

[GitHub] spark pull request #21145: [SPARK-24073][SQL]: Rename DataReaderFactory to R...

2018-05-07 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21145#discussion_r186597255 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/reader/ReadTask.java --- @@ -22,20 +22,20 @@ import

[GitHub] spark issue #21259: [SPARK-24112][SQL] Add `convertMetastoreTableProperty` c...

2018-05-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/21259 Could you review this backward compatibility PR, @cloud-fan , @gatorsmile , @mridulm , and @HyukjinKwon ? --- - To

[GitHub] spark issue #21253: [SPARK-24158][SS] Enable no-data batches for streaming j...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21253 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90342/ Test PASSed. ---

[GitHub] spark issue #21253: [SPARK-24158][SS] Enable no-data batches for streaming j...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21253 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21230: [SPARK-24172][SQL] we should not apply operator pushdown...

2018-05-07 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21230 Hi @rdblue , thanks for your new approach! Like you said, the major problem is about statistics. This is unfortunately a problem of Spark's CBO design: the statistics should belong to physical

[GitHub] spark issue #21253: [SPARK-24158][SS] Enable no-data batches for streaming j...

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21253 **[Test build #90342 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90342/testReport)** for PR 21253 at commit

[GitHub] spark issue #21173: [SPARK-23856][SQL] Add an option `queryTimeout` in JDBCO...

2018-05-07 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21173 @gatorsmile ping --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21211: [SPARK-24131][PYSPARK][Followup] Add majorMinorVersion A...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21211 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21211: [SPARK-24131][PYSPARK][Followup] Add majorMinorVersion A...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21211 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3021/

[GitHub] spark pull request #20929: [SPARK-23772][SQL][WIP] Provide an option to igno...

2018-05-07 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/20929#discussion_r186593019 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/types/TypePlaceholder.scala --- @@ -0,0 +1,23 @@ +/* + * Licensed to the Apache Software

[GitHub] spark issue #21199: [SPARK-24127][SS] Continuous text socket source

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21199 **[Test build #90349 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90349/testReport)** for PR 21199 at commit

[GitHub] spark issue #21211: [SPARK-24131][PYSPARK][Followup] Add majorMinorVersion A...

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21211 **[Test build #90348 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90348/testReport)** for PR 21211 at commit

[GitHub] spark issue #21199: [SPARK-24127][SS] Continuous text socket source

2018-05-07 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21199 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21199: [SPARK-24127][SS] Continuous text socket source

2018-05-07 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21199 add to whitelist --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21092: [SPARK-23984][K8S] Initial Python Bindings for Py...

2018-05-07 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/21092#discussion_r186591946 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/KubernetesDriverBuilder.scala --- @@ -29,18 +31,36 @@

[GitHub] spark issue #21211: [SPARK-24131][PYSPARK][Followup] Add majorMinorVersion A...

2018-05-07 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21211 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21092: [SPARK-23984][K8S] Initial Python Bindings for Py...

2018-05-07 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/21092#discussion_r186591608 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/KubernetesDriverBuilder.scala --- @@ -29,18 +31,36 @@

[GitHub] spark pull request #21092: [SPARK-23984][K8S] Initial Python Bindings for Py...

2018-05-07 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/21092#discussion_r186591836 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/KubernetesDriverBuilder.scala --- @@ -29,18 +31,36 @@

[GitHub] spark pull request #21092: [SPARK-23984][K8S] Initial Python Bindings for Py...

2018-05-07 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/21092#discussion_r186591534 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/bindings/PythonDriverFeatureStep.scala --- @@ -0,0 +1,72

[GitHub] spark pull request #20929: [SPARK-23772][SQL][WIP] Provide an option to igno...

2018-05-07 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/20929#discussion_r186589216 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/types/TypePlaceholder.scala --- @@ -0,0 +1,23 @@ +/* + * Licensed to the Apache Software

[GitHub] spark issue #21259: [SPARK-24112][SQL] Add `convertMetastoreTableProperty` c...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21259 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21259: [SPARK-24112][SQL] Add `convertMetastoreTableProperty` c...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21259 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90341/ Test PASSed. ---

[GitHub] spark issue #21259: [SPARK-24112][SQL] Add `convertMetastoreTableProperty` c...

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21259 **[Test build #90341 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90341/testReport)** for PR 21259 at commit

[GitHub] spark pull request #21145: [SPARK-24073][SQL]: Rename DataReaderFactory to R...

2018-05-07 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/21145#discussion_r186587220 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/reader/ReadTask.java --- @@ -22,20 +22,20 @@ import

[GitHub] spark issue #21254: [SPARK-23094][SPARK-23723][SPARK-23724][SQL][FOLLOW-UP] ...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21254 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3020/

[GitHub] spark issue #21254: [SPARK-23094][SPARK-23723][SPARK-23724][SQL][FOLLOW-UP] ...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21254 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21254: [SPARK-23094][SPARK-23723][SPARK-23724][SQL][FOLLOW-UP] ...

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21254 **[Test build #90347 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90347/testReport)** for PR 21254 at commit

[GitHub] spark issue #21118: SPARK-23325: Use InternalRow when reading with DataSourc...

2018-05-07 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/21118 @cloud-fan: This PR is also related to #21262 because that PR updates the conversion from logical to physical plan and handles projections and filtering. We could modify that strategy to always

[GitHub] spark issue #21254: [SPARK-23094][SPARK-23723][SPARK-23724][SQL][FOLLOW-UP] ...

2018-05-07 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21254 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21262: [SPARK-24172][SQL]: Push projection and filters once whe...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21262 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3019/

[GitHub] spark issue #21262: [SPARK-24172][SQL]: Push projection and filters once whe...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21262 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21230: [SPARK-24172][SQL] we should not apply operator pushdown...

2018-05-07 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/21230 @cloud-fan, I opened #21262 that is similar to this, but does pushdown when converting to a physical plan. You might like that as an alternative because it cleans up `DataSourceV2Relation` quite a

[GitHub] spark issue #21262: [SPARK-24172][SQL]: Push projection and filters once whe...

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21262 **[Test build #90346 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90346/testReport)** for PR 21262 at commit

[GitHub] spark issue #21250: [SPARK-23291][SQL][R][BRANCH-2.3] R's substr should not ...

2018-05-07 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21250 Thanks, @cloud-fan, @yanboliang, @felixcheung and @viirya. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #21262: [SPARK-24172][SQL]: Push projection and filters o...

2018-05-07 Thread rdblue
GitHub user rdblue opened a pull request: https://github.com/apache/spark/pull/21262 [SPARK-24172][SQL]: Push projection and filters once when converting to physical plan. ## What changes were proposed in this pull request? This removes `PruneFileSourcePartitions` and

[GitHub] spark issue #21249: [SPARK-23291][R][FOLLOWUP] Update SparkR migration note ...

2018-05-07 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21249 Thanks, @cloud-fan, @yanboliang, @felixcheung and @viirya. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #20929: [SPARK-23772][SQL][WIP] Provide an option to igno...

2018-05-07 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/20929#discussion_r186584474 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/types/TypePlaceholder.scala --- @@ -0,0 +1,23 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #21250: [SPARK-23291][SQL][R][BRANCH-2.3] R's substr shou...

2018-05-07 Thread HyukjinKwon
Github user HyukjinKwon closed the pull request at: https://github.com/apache/spark/pull/21250 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21144: [SPARK-24043][SQL] Interpreted Predicate should initiali...

2018-05-07 Thread bersprockets
Github user bersprockets commented on the issue: https://github.com/apache/spark/pull/21144 Thanks much! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21239: [SPARK-24040][SS] Support single partition aggregates in...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21239 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90340/ Test PASSed. ---

[GitHub] spark issue #21239: [SPARK-24040][SS] Support single partition aggregates in...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21239 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21239: [SPARK-24040][SS] Support single partition aggregates in...

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21239 **[Test build #90340 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90340/testReport)** for PR 21239 at commit

[GitHub] spark issue #20929: [SPARK-23772][SQL][WIP] Provide an option to ignore colu...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20929 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3018/

[GitHub] spark issue #20929: [SPARK-23772][SQL][WIP] Provide an option to ignore colu...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20929 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20929: [SPARK-23772][SQL][WIP] Provide an option to ignore colu...

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20929 **[Test build #90345 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90345/testReport)** for PR 20929 at commit

[GitHub] spark pull request #20929: [SPARK-23772][SQL][WIP] Provide an option to igno...

2018-05-07 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/20929#discussion_r186581349 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -887,6 +887,14 @@ object SQLConf { .booleanConf

[GitHub] spark pull request #20929: [SPARK-23772][SQL][WIP] Provide an option to igno...

2018-05-07 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/20929#discussion_r186581071 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -887,6 +887,14 @@ object SQLConf { .booleanConf

[GitHub] spark issue #20929: [SPARK-23772][SQL][WIP] Provide an option to ignore colu...

2018-05-07 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/20929 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21106: [SPARK-23711][SQL][WIP] Add fallback logic for Un...

2018-05-07 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21106#discussion_r186580919 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/CodegenObjectFactory.scala --- @@ -0,0 +1,80 @@ +/* + * Licensed to

[GitHub] spark issue #21195: [Spark-23975][ML] Add support of array input for all clu...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21195 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21195: [Spark-23975][ML] Add support of array input for all clu...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21195 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90344/ Test PASSed. ---

[GitHub] spark issue #21195: [Spark-23975][ML] Add support of array input for all clu...

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21195 **[Test build #90344 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90344/testReport)** for PR 21195 at commit

[GitHub] spark pull request #21129: [SPARK-7132][ML] Add fit with validation set to s...

2018-05-07 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/21129#discussion_r186572477 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/treeParams.scala --- @@ -497,6 +498,9 @@ private[ml] trait GBTParams extends TreeEnsembleParams

[GitHub] spark pull request #21129: [SPARK-7132][ML] Add fit with validation set to s...

2018-05-07 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/21129#discussion_r186572374 --- Diff: mllib/src/test/scala/org/apache/spark/ml/classification/GBTClassifierSuite.scala --- @@ -365,6 +366,50 @@ class GBTClassifierSuite extends

[GitHub] spark pull request #21129: [SPARK-7132][ML] Add fit with validation set to s...

2018-05-07 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/21129#discussion_r186573136 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/GBTClassifier.scala --- @@ -146,20 +146,40 @@ class GBTClassifier @Since("1.4.0") (

[GitHub] spark pull request #21129: [SPARK-7132][ML] Add fit with validation set to s...

2018-05-07 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/21129#discussion_r186569928 --- Diff: mllib/src/test/scala/org/apache/spark/ml/classification/GBTClassifierSuite.scala --- @@ -365,6 +366,50 @@ class GBTClassifierSuite extends

[GitHub] spark pull request #21129: [SPARK-7132][ML] Add fit with validation set to s...

2018-05-07 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/21129#discussion_r186572853 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/GBTClassifier.scala --- @@ -146,20 +146,40 @@ class GBTClassifier @Since("1.4.0") (

[GitHub] spark issue #21073: [SPARK-23936][SQL] Implement map_concat

2018-05-07 Thread bersprockets
Github user bersprockets commented on the issue: https://github.com/apache/spark/pull/21073 @ueshin Hopefully I have addressed all of your review comments. Also, I have a question about what it means to dedup across maps when Spark allows duplicates in maps

[GitHub] spark pull request #21073: [SPARK-23936][SQL] Implement map_concat

2018-05-07 Thread bersprockets
Github user bersprockets commented on a diff in the pull request: https://github.com/apache/spark/pull/21073#discussion_r186570491 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -116,6 +117,169 @@ case class

[GitHub] spark issue #21258: [SPARK-23933][SQL] Add map_fromarray function

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21258 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21258: [SPARK-23933][SQL] Add map_fromarray function

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21258 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90338/ Test FAILed. ---

[GitHub] spark issue #21258: [SPARK-23933][SQL] Add map_fromarray function

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21258 **[Test build #90338 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90338/testReport)** for PR 21258 at commit

[GitHub] spark issue #21073: [SPARK-23936][SQL] Implement map_concat

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21073 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21073: [SPARK-23936][SQL] Implement map_concat

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21073 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90336/ Test PASSed. ---

[GitHub] spark issue #21129: [SPARK-7132][ML] Add fit with validation set to spark.ml...

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21129 **[Test build #4170 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4170/testReport)** for PR 21129 at commit

[GitHub] spark issue #21073: [SPARK-23936][SQL] Implement map_concat

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21073 **[Test build #90336 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90336/testReport)** for PR 21073 at commit

[GitHub] spark pull request #21251: [SPARK-10878][core] Fix race condition when multi...

2018-05-07 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/21251#discussion_r186567342 --- Diff: core/src/test/scala/org/apache/spark/deploy/SparkSubmitUtilsSuite.scala --- @@ -256,4 +256,20 @@ class SparkSubmitUtilsSuite extends SparkFunSuite

[GitHub] spark pull request #21251: [SPARK-10878][core] Fix race condition when multi...

2018-05-07 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/21251#discussion_r186567456 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -1204,7 +1205,36 @@ private[spark] object SparkSubmitUtils { /**

[GitHub] spark pull request #21251: [SPARK-10878][core] Fix race condition when multi...

2018-05-07 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/21251#discussion_r186566903 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -1204,7 +1205,36 @@ private[spark] object SparkSubmitUtils { /**

[GitHub] spark pull request #21097: [SPARK-14682][ML] Provide evaluateEachIteration m...

2018-05-07 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/21097#discussion_r186565756 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/GBTRegressor.scala --- @@ -269,6 +269,20 @@ class GBTRegressionModel private[ml](

[GitHub] spark issue #21195: [Spark-23975][ML] Add support of array input for all clu...

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21195 **[Test build #90344 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90344/testReport)** for PR 21195 at commit

[GitHub] spark pull request #21195: [Spark-23975][ML] Add support of array input for ...

2018-05-07 Thread ludatabricks
Github user ludatabricks commented on a diff in the pull request: https://github.com/apache/spark/pull/21195#discussion_r186566521 --- Diff: mllib/src/test/scala/org/apache/spark/ml/clustering/LDASuite.scala --- @@ -323,4 +324,21 @@ class LDASuite extends SparkFunSuite with

[GitHub] spark issue #21209: [SPARK-24141][CORE] Fix bug in CoarseGrainedSchedulerBac...

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21209 **[Test build #90343 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90343/testReport)** for PR 21209 at commit

[GitHub] spark issue #21209: [SPARK-24141][CORE] Fix bug in CoarseGrainedSchedulerBac...

2018-05-07 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/21209 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20261: [SPARK-22885][ML][TEST] ML test for StructuredStr...

2018-05-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20261 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

<    1   2   3   4   5   6   7   >