[GitHub] spark pull request #19160: [SPARK-21934][CORE] Expose Shuffle Netty memory u...

2017-09-19 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19160#discussion_r139799940 --- Diff: core/src/main/scala/org/apache/spark/deploy/ExternalShuffleServiceSource.scala --- @@ -19,19 +19,19 @@ package org.apache.spark.deploy

[GitHub] spark issue #19211: [SPARK-18838][core] Add separate listener queues to Live...

2017-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19211 **[Test build #81940 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81940/testReport)** for PR 19211 at commit [`283b733`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #19211: [SPARK-18838][core] Add separate listener queues to Live...

2017-09-19 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/19211 I don't really have a readily available cluster that can easily put that much pressure on the listener bus; I tried with the one I have and monitored the listener bus metrics, and found pretty much w

[GitHub] spark issue #19196: [SPARK-21977] SinglePartition optimizations break certai...

2017-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19196 **[Test build #81946 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81946/testReport)** for PR 19196 at commit [`f34fc8a`](https://github.com/apache/spark/commit/f

[GitHub] spark issue #19196: [SPARK-21977] SinglePartition optimizations break certai...

2017-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19196 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19196: [SPARK-21977] SinglePartition optimizations break certai...

2017-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19196 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81946/ Test FAILed. ---

[GitHub] spark issue #19196: [SPARK-21977] SinglePartition optimizations break certai...

2017-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19196 **[Test build #81946 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81946/testReport)** for PR 19196 at commit [`f34fc8a`](https://github.com/apache/spark/commit/f3

[GitHub] spark issue #19196: [SPARK-21977] SinglePartition optimizations break certai...

2017-09-19 Thread brkyvz
Github user brkyvz commented on the issue: https://github.com/apache/spark/pull/19196 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h..

[GitHub] spark pull request #19272: [Spark-21842][Mesos] Support Kerberos ticket rene...

2017-09-19 Thread susanxhuynh
Github user susanxhuynh commented on a diff in the pull request: https://github.com/apache/spark/pull/19272#discussion_r139772286 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosCredentialRenewer.scala --- @@ -0,0 +1,150 @@ +/*

[GitHub] spark pull request #19272: [Spark-21842][Mesos] Support Kerberos ticket rene...

2017-09-19 Thread susanxhuynh
Github user susanxhuynh commented on a diff in the pull request: https://github.com/apache/spark/pull/19272#discussion_r139726573 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosCredentialRenewer.scala --- @@ -0,0 +1,150 @@ +/*

[GitHub] spark pull request #19272: [Spark-21842][Mesos] Support Kerberos ticket rene...

2017-09-19 Thread susanxhuynh
Github user susanxhuynh commented on a diff in the pull request: https://github.com/apache/spark/pull/19272#discussion_r139779444 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala --- @@ -159,6 +159,13 @@ class CoarseGrainedSchedu

[GitHub] spark issue #19211: [SPARK-18838][core] Add separate listener queues to Live...

2017-09-19 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/19211 lgtm do you know how much better this makes it? Eg., if we had an existing case where things go haywire in dynamic allocation because of this -- we could see if after this change, the only

[GitHub] spark issue #18659: [SPARK-21190][PYSPARK][WIP] Python Vectorized UDFs

2017-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18659 **[Test build #81945 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81945/testReport)** for PR 18659 at commit [`69112a5`](https://github.com/apache/spark/commit/69

[GitHub] spark pull request #19211: [SPARK-18838][core] Add separate listener queues ...

2017-09-19 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19211#discussion_r139787945 --- Diff: core/src/main/scala/org/apache/spark/scheduler/AsyncEventQueue.scala --- @@ -0,0 +1,196 @@ +/* + * Licensed to the Apache Software Foundati

[GitHub] spark issue #19281: [SPARK-21998][SQL] SortMergeJoinExec did not calculate i...

2017-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19281 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19218: [SPARK-21786][SQL] The 'spark.sql.parquet.compression.co...

2017-09-19 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19218 You just need to fix `InsertIntoHiveTable `. It is not related to `Insertintohadoopfsrelationcommand.scala`. In your test cases, you just need to set `spark.sql.hive.convertMetastoreParquet` to `

[GitHub] spark pull request #19281: [SPARK-21998][SQL] SortMergeJoinExec did not calc...

2017-09-19 Thread maryannxue
GitHub user maryannxue opened a pull request: https://github.com/apache/spark/pull/19281 [SPARK-21998][SQL] SortMergeJoinExec did not calculate its outputOrdering correctly during physical planning ## What changes were proposed in this pull request? Right now the calculatio

[GitHub] spark issue #19218: [SPARK-21786][SQL] The 'spark.sql.parquet.compression.co...

2017-09-19 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19218 Table-level metadata has the highest priority. In the table-level metadata, `compression` has a higher priority than `parquet.compression`. If the is not specified, we need to check the session-l

[GitHub] spark pull request #18754: [WIP][SPARK-21552][SQL] Add DecimalType support t...

2017-09-19 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/18754#discussion_r139777828 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/arrow/ArrowWriter.scala --- @@ -224,6 +226,25 @@ private[arrow] class DoubleWriter(val

[GitHub] spark issue #18193: [SPARK-15616] [SQL] CatalogRelation should fallback to H...

2017-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18193 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #18193: [SPARK-15616] [SQL] CatalogRelation should fallback to H...

2017-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18193 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81939/ Test PASSed. ---

[GitHub] spark issue #18193: [SPARK-15616] [SQL] CatalogRelation should fallback to H...

2017-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18193 **[Test build #81939 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81939/testReport)** for PR 18193 at commit [`fd95fb3`](https://github.com/apache/spark/commit/f

[GitHub] spark issue #18193: [SPARK-15616] [SQL] CatalogRelation should fallback to H...

2017-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18193 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #18193: [SPARK-15616] [SQL] CatalogRelation should fallback to H...

2017-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18193 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81937/ Test PASSed. ---

[GitHub] spark issue #18193: [SPARK-15616] [SQL] CatalogRelation should fallback to H...

2017-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18193 **[Test build #81937 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81937/testReport)** for PR 18193 at commit [`fdd63c3`](https://github.com/apache/spark/commit/f

[GitHub] spark issue #19196: [SPARK-21977] SinglePartition optimizations break certai...

2017-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19196 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81942/ Test FAILed. ---

[GitHub] spark issue #19196: [SPARK-21977] SinglePartition optimizations break certai...

2017-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19196 **[Test build #81942 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81942/testReport)** for PR 19196 at commit [`f34fc8a`](https://github.com/apache/spark/commit/f

[GitHub] spark issue #19196: [SPARK-21977] SinglePartition optimizations break certai...

2017-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19196 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #18853: [SPARK-21646][SQL] CommonType for binary comparis...

2017-09-19 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18853#discussion_r139774828 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala --- @@ -352,11 +374,16 @@ object TypeCoercion {

[GitHub] spark issue #19280: [SPARK-21928][CORE] Set classloader on SerializerManager...

2017-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19280 **[Test build #81944 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81944/testReport)** for PR 19280 at commit [`acbaf8b`](https://github.com/apache/spark/commit/ac

[GitHub] spark pull request #19280: [SPARK-21928][CORE] Set classloader on Serializer...

2017-09-19 Thread squito
GitHub user squito opened a pull request: https://github.com/apache/spark/pull/19280 [SPARK-21928][CORE] Set classloader on SerializerManager private kryo ## What changes were proposed in this pull request? We have to make sure thatthat SerializerManager's private instance o

[GitHub] spark issue #19250: [SPARK-12297] Table timezone correction for Timestamps

2017-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19250 **[Test build #81943 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81943/testReport)** for PR 19250 at commit [`515b38b`](https://github.com/apache/spark/commit/51

[GitHub] spark issue #19196: [SPARK-21977] SinglePartition optimizations break certai...

2017-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19196 **[Test build #81942 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81942/testReport)** for PR 19196 at commit [`f34fc8a`](https://github.com/apache/spark/commit/f3

[GitHub] spark issue #19196: [SPARK-21977] SinglePartition optimizations break certai...

2017-09-19 Thread brkyvz
Github user brkyvz commented on the issue: https://github.com/apache/spark/pull/19196 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h..

[GitHub] spark issue #19252: [SPARK-21969][SQL] CommandUtils.updateTableStats should ...

2017-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19252 **[Test build #81941 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81941/testReport)** for PR 19252 at commit [`63f9dc2`](https://github.com/apache/spark/commit/63

[GitHub] spark issue #19020: [SPARK-3181] [ML] Implement huber loss for LinearRegress...

2017-09-19 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/19020 I'll check this out now --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: rev

[GitHub] spark issue #19256: [SPARK-21338][SQL]implement isCascadingTruncateTable() m...

2017-09-19 Thread huaxingao
Github user huaxingao commented on the issue: https://github.com/apache/spark/pull/19256 @gatorsmile Thanks a lot for your help!!! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional com

[GitHub] spark pull request #18754: [WIP][SPARK-21552][SQL] Add DecimalType support t...

2017-09-19 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/18754#discussion_r139762422 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/arrow/ArrowWriter.scala --- @@ -224,6 +226,25 @@ private[arrow] class DoubleWriter(val

[GitHub] spark issue #19256: [SPARK-21338][SQL]implement isCascadingTruncateTable() m...

2017-09-19 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19256 Thanks! Merged to master --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: r

[GitHub] spark issue #17819: [SPARK-20542][ML][SQL] Add an API to Bucketizer that can...

2017-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17819 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81935/ Test FAILed. ---

[GitHub] spark issue #17819: [SPARK-20542][ML][SQL] Add an API to Bucketizer that can...

2017-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17819 **[Test build #81935 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81935/testReport)** for PR 17819 at commit [`92ef9bd`](https://github.com/apache/spark/commit/9

[GitHub] spark issue #17819: [SPARK-20542][ML][SQL] Add an API to Bucketizer that can...

2017-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17819 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19250: [SPARK-12297] Table timezone correction for Timestamps

2017-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19250 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81929/ Test PASSed. ---

[GitHub] spark issue #19250: [SPARK-12297] Table timezone correction for Timestamps

2017-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19250 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19250: [SPARK-12297] Table timezone correction for Timestamps

2017-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19250 **[Test build #81929 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81929/testReport)** for PR 19250 at commit [`2069b65`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #19269: [SPARK-22026][SQL][WIP] data source v2 write path

2017-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19269 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81930/ Test PASSed. ---

[GitHub] spark issue #19269: [SPARK-22026][SQL][WIP] data source v2 write path

2017-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19269 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19269: [SPARK-22026][SQL][WIP] data source v2 write path

2017-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19269 **[Test build #81930 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81930/testReport)** for PR 19269 at commit [`862a679`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #19211: [SPARK-18838][core] Add separate listener queues to Live...

2017-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19211 **[Test build #81940 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81940/testReport)** for PR 19211 at commit [`283b733`](https://github.com/apache/spark/commit/28

[GitHub] spark pull request #19211: [SPARK-18838][core] Add separate listener queues ...

2017-09-19 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19211#discussion_r139750030 --- Diff: core/src/main/scala/org/apache/spark/scheduler/LiveListenerBus.scala --- @@ -39,20 +41,13 @@ import org.apache.spark.util.Utils * has started

[GitHub] spark pull request #19211: [SPARK-18838][core] Add separate listener queues ...

2017-09-19 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19211#discussion_r139749744 --- Diff: streaming/src/test/scala/org/apache/spark/streaming/StreamingContextSuite.scala --- @@ -575,8 +575,6 @@ class StreamingContextSuite extends SparkF

[GitHub] spark pull request #19211: [SPARK-18838][core] Add separate listener queues ...

2017-09-19 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19211#discussion_r139748667 --- Diff: core/src/main/scala/org/apache/spark/scheduler/LiveListenerBus.scala --- @@ -173,80 +159,62 @@ private[spark] class LiveListenerBus(conf: SparkConf

[GitHub] spark pull request #19211: [SPARK-18838][core] Add separate listener queues ...

2017-09-19 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19211#discussion_r139748516 --- Diff: core/src/main/scala/org/apache/spark/scheduler/LiveListenerBus.scala --- @@ -173,80 +159,62 @@ private[spark] class LiveListenerBus(conf: SparkConf

[GitHub] spark issue #18853: [SPARK-21646][SQL] CommonType for binary comparison

2017-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18853 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #18853: [SPARK-21646][SQL] CommonType for binary comparison

2017-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18853 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81934/ Test FAILed. ---

[GitHub] spark issue #18853: [SPARK-21646][SQL] CommonType for binary comparison

2017-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18853 **[Test build #81934 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81934/testReport)** for PR 18853 at commit [`7812018`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #19196: [SPARK-21977] SinglePartition optimizations break certai...

2017-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19196 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19196: [SPARK-21977] SinglePartition optimizations break certai...

2017-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19196 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81938/ Test FAILed. ---

[GitHub] spark issue #19196: [SPARK-21977] SinglePartition optimizations break certai...

2017-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19196 **[Test build #81938 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81938/testReport)** for PR 19196 at commit [`f34fc8a`](https://github.com/apache/spark/commit/f

[GitHub] spark issue #18193: [SPARK-15616] [SQL] CatalogRelation should fallback to H...

2017-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18193 **[Test build #81939 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81939/testReport)** for PR 18193 at commit [`fd95fb3`](https://github.com/apache/spark/commit/fd

[GitHub] spark pull request #19256: [SPARK-21338][SQL]implement isCascadingTruncateTa...

2017-09-19 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19256 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #19276: [SPARK-22049][DOCS] Confusing behavior of from_ut...

2017-09-19 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19276#discussion_r139745184 --- Diff: R/pkg/R/functions.R --- @@ -2226,8 +2226,8 @@ setMethod("from_json", signature(x = "Column", schema = "characterOrstructType")

[GitHub] spark issue #18193: [SPARK-15616] [SQL] CatalogRelation should fallback to H...

2017-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18193 **[Test build #81937 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81937/testReport)** for PR 18193 at commit [`fdd63c3`](https://github.com/apache/spark/commit/fd

[GitHub] spark issue #19196: [SPARK-21977] SinglePartition optimizations break certai...

2017-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19196 **[Test build #81938 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81938/testReport)** for PR 19196 at commit [`f34fc8a`](https://github.com/apache/spark/commit/f3

[GitHub] spark issue #19196: [SPARK-21977] SinglePartition optimizations break certai...

2017-09-19 Thread brkyvz
Github user brkyvz commented on the issue: https://github.com/apache/spark/pull/19196 @tdas Addressed --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@s

[GitHub] spark issue #17819: [SPARK-20542][ML][SQL] Add an API to Bucketizer that can...

2017-09-19 Thread MLnick
Github user MLnick commented on the issue: https://github.com/apache/spark/pull/17819 The issue is the in the trait `setXXX` returns `this.type` which in Java in the concrete class doesn't work, so the `setXXX` methods need to be implemented in the concrete subclass. See the decision

[GitHub] spark pull request #18193: [SPARK-15616] [SQL] CatalogRelation should fallba...

2017-09-19 Thread lianhuiwang
Github user lianhuiwang commented on a diff in the pull request: https://github.com/apache/spark/pull/18193#discussion_r139740180 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveStrategies.scala --- @@ -139,6 +138,54 @@ class DetermineTableStats(session: SparkSessio

[GitHub] spark issue #18636: added support word2vec training with additional data

2017-09-19 Thread MLnick
Github user MLnick commented on the issue: https://github.com/apache/spark/pull/18636 I'm sorry but I still don't understand the intention here. You can already train on a Wikipedia dump (or any other dataset) by passing that dataset as the input DataFrame to Word2Vec. If you

[GitHub] spark issue #19106: [SPARK-21770][ML] ProbabilisticClassificationModel fix c...

2017-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19106 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81933/ Test PASSed. ---

[GitHub] spark issue #19106: [SPARK-21770][ML] ProbabilisticClassificationModel fix c...

2017-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19106 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19106: [SPARK-21770][ML] ProbabilisticClassificationModel fix c...

2017-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19106 **[Test build #81933 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81933/testReport)** for PR 19106 at commit [`e7fe2fa`](https://github.com/apache/spark/commit/e

[GitHub] spark issue #19270: [SPARK-21809] : Change Stage Page to use datatables to s...

2017-09-19 Thread pgandhi999
Github user pgandhi999 commented on the issue: https://github.com/apache/spark/pull/19270 The error logs for test build #81683 state that method this(Long,Int,Int,Long,Long,Long,Long,Long,Long)Unit in class org.apache.spark.status.api.v1.ExecutorStageSummary does not have a correspon

[GitHub] spark issue #19270: [SPARK-21809] : Change Stage Page to use datatables to s...

2017-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19270 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19270: [SPARK-21809] : Change Stage Page to use datatables to s...

2017-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19270 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81936/ Test FAILed. ---

[GitHub] spark issue #19270: [SPARK-21809] : Change Stage Page to use datatables to s...

2017-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19270 **[Test build #81936 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81936/testReport)** for PR 19270 at commit [`c588953`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #19250: [SPARK-12297] Table timezone correction for Timestamps

2017-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19250 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81931/ Test FAILed. ---

[GitHub] spark issue #19250: [SPARK-12297] Table timezone correction for Timestamps

2017-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19250 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19250: [SPARK-12297] Table timezone correction for Timestamps

2017-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19250 **[Test build #81931 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81931/testReport)** for PR 19250 at commit [`74a9905`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #19270: [SPARK-21809] : Change Stage Page to use datatables to s...

2017-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19270 **[Test build #81936 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81936/testReport)** for PR 19270 at commit [`c588953`](https://github.com/apache/spark/commit/c5

[GitHub] spark issue #19270: [SPARK-21809] : Change Stage Page to use datatables to s...

2017-09-19 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/19270 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spa

[GitHub] spark pull request #17819: [SPARK-20542][ML][SQL] Add an API to Bucketizer t...

2017-09-19 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/17819#discussion_r139729964 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -2102,6 +2102,53 @@ class Dataset[T] private[sql]( } /** +

[GitHub] spark issue #17819: [SPARK-20542][ML][SQL] Add an API to Bucketizer that can...

2017-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17819 **[Test build #81935 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81935/testReport)** for PR 17819 at commit [`92ef9bd`](https://github.com/apache/spark/commit/92

[GitHub] spark issue #19106: [SPARK-21770][ML] ProbabilisticClassificationModel fix c...

2017-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19106 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19106: [SPARK-21770][ML] ProbabilisticClassificationModel fix c...

2017-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19106 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81932/ Test PASSed. ---

[GitHub] spark issue #19106: [SPARK-21770][ML] ProbabilisticClassificationModel fix c...

2017-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19106 **[Test build #81932 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81932/testReport)** for PR 19106 at commit [`e7fe2fa`](https://github.com/apache/spark/commit/e

[GitHub] spark issue #18636: added support word2vec training with additional data

2017-09-19 Thread LeoIV
Github user LeoIV commented on the issue: https://github.com/apache/spark/pull/18636 At the moment, it is not possible to improve a models accuracy by incorporating additional data. I think this should be supported since it can increase a classifiers performance significantly. With th

[GitHub] spark issue #18636: added support word2vec training with additional data

2017-09-19 Thread MLnick
Github user MLnick commented on the issue: https://github.com/apache/spark/pull/18636 Hi there - I don't see the value here of adding a few words in a String array to the training. You're effectively adding a second (non-distributed, therefore limited in size) corpus to the training.

[GitHub] spark issue #18853: [SPARK-21646][SQL] CommonType for binary comparison

2017-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18853 **[Test build #81934 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81934/testReport)** for PR 18853 at commit [`7812018`](https://github.com/apache/spark/commit/78

[GitHub] spark pull request #18853: [SPARK-21646][SQL] CommonType for binary comparis...

2017-09-19 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/18853#discussion_r139719600 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala --- @@ -352,11 +374,16 @@ object TypeCoercion {

[GitHub] spark issue #19278: [SPARK-22060][ML] Fix CrossValidator/TrainValidationSpli...

2017-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19278 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81927/ Test PASSed. ---

[GitHub] spark issue #19278: [SPARK-22060][ML] Fix CrossValidator/TrainValidationSpli...

2017-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19278 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19278: [SPARK-22060][ML] Fix CrossValidator/TrainValidationSpli...

2017-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19278 **[Test build #81927 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81927/testReport)** for PR 19278 at commit [`042b3d5`](https://github.com/apache/spark/commit/0

[GitHub] spark pull request #19204: [SPARK-21981][PYTHON][ML] Added Python interface ...

2017-09-19 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/19204#discussion_r139718610 --- Diff: python/pyspark/ml/evaluation.py --- @@ -328,6 +329,77 @@ def setParams(self, predictionCol="prediction", labelCol="label", kwargs

[GitHub] spark issue #19106: [SPARK-21770][ML] ProbabilisticClassificationModel fix c...

2017-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19106 **[Test build #81933 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81933/testReport)** for PR 19106 at commit [`e7fe2fa`](https://github.com/apache/spark/commit/e7

[GitHub] spark issue #19106: [SPARK-21770][ML] ProbabilisticClassificationModel fix c...

2017-09-19 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/19106 Jenkins, test this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mai

[GitHub] spark pull request #15544: [SPARK-17997] [SQL] Add an aggregation function f...

2017-09-19 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/15544#discussion_r139713569 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/aggregate/ApproxCountDistinctForIntervalsSuite.scala --- @@ -0,0 +1,206 @

[GitHub] spark issue #19156: [SPARK-19634][SQL][ML][FOLLOW-UP] Improve interface of d...

2017-09-19 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/19156 @WeichenXu123 Sorry for late response, really busy in these days. I will take a look in a few days. Thanks for your patience. --- ---

[GitHub] spark pull request #15544: [SPARK-17997] [SQL] Add an aggregation function f...

2017-09-19 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/15544#discussion_r139713018 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/aggregate/ApproxCountDistinctForIntervalsSuite.scala --- @@ -0,0 +1,206 @

[GitHub] spark issue #12066: [SPARK-7424] [ML] ML ClassificationModel should add meta...

2017-09-19 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/12066 @yanboliang Are you still working on this ? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additiona

[GitHub] spark pull request #15544: [SPARK-17997] [SQL] Add an aggregation function f...

2017-09-19 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/15544#discussion_r139710680 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/aggregate/ApproxCountDistinctForIntervalsSuite.scala --- @@ -0,0 +1,206 @

<    1   2   3   4   5   6   >