[GitHub] spark pull request #21169: [SPARK-23715][SQL] the input of to/from_utc_times...

2018-04-26 Thread bersprockets
Github user bersprockets commented on a diff in the pull request: https://github.com/apache/spark/pull/21169#discussion_r184561518 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala --- @@ -296,10 +296,27 @@ object DateTimeUtils {

[GitHub] spark issue #21088: [SPARK-24003][CORE] Add support to provide spark.executo...

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21088 **[Test build #89909 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89909/testReport)** for PR 21088 at commit [`932b7d1`](https://github.com/apache/spark/commit/93

[GitHub] spark issue #21088: [SPARK-24003][CORE] Add support to provide spark.executo...

2018-04-26 Thread devaraj-kavali
Github user devaraj-kavali commented on the issue: https://github.com/apache/spark/pull/21088 For standalone cluster(`DriverRunner.scala:182`), there is driverId which we can use here and for Mesos cluster(`MesosRestServer.scala:107`), there is submission ID available but there is no

[GitHub] spark issue #21088: [SPARK-24003][CORE] Add support to provide spark.executo...

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21088 **[Test build #89908 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89908/testReport)** for PR 21088 at commit [`27cc29c`](https://github.com/apache/spark/commit/27

[GitHub] spark pull request #17086: [SPARK-24101][ML][MLLIB] ML Evaluators should use...

2018-04-26 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/17086#discussion_r184566012 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/evaluation/MulticlassMetricsSuite.scala --- @@ -95,4 +95,95 @@ class MulticlassMetricsSuite exte

[GitHub] spark issue #21174: [SPARK-24085] Query returns UnsupportedOperationExceptio...

2018-04-26 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/21174 @maropu Thanks for your response. ORC has CONVERT_METASTORE_ORC set to false as default. So its not converted to a file based datasource. If we set this to true then we would see the same issue

[GitHub] spark issue #21174: [SPARK-24085] Query returns UnsupportedOperationExceptio...

2018-04-26 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21174 Why the orc works fine? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: review

[GitHub] spark issue #18717: [SPARK-21510] [SQL] Add isMaterialized() and eager persi...

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18717 **[Test build #89907 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89907/testReport)** for PR 18717 at commit [`31bf797`](https://github.com/apache/spark/commit/31

[GitHub] spark pull request #21151: [SPARK-24083][YARN] Log stacktrace for uncaught e...

2018-04-26 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21151 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21151: [SPARK-24083][YARN] Log stacktrace for uncaught exceptio...

2018-04-26 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/21151 Merging to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h..

[GitHub] spark issue #21174: [SPARK-24085] Query returns UnsupportedOperationExceptio...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21174 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2707/ Tes

[GitHub] spark issue #21174: [SPARK-24085] Query returns UnsupportedOperationExceptio...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21174 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20522: [SPARK-23355][SQL] convertMetastore should not ignore ta...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20522 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89901/ Test PASSed. ---

[GitHub] spark issue #20522: [SPARK-23355][SQL] convertMetastore should not ignore ta...

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20522 **[Test build #89901 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89901/testReport)** for PR 20522 at commit [`06a9a45`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #20522: [SPARK-23355][SQL] convertMetastore should not ignore ta...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20522 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21174: [SPARK-24085] Query returns UnsupportedOperationExceptio...

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21174 **[Test build #89906 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89906/testReport)** for PR 21174 at commit [`38c7692`](https://github.com/apache/spark/commit/38

[GitHub] spark pull request #21174: [SPARK-24085] Query returns UnsupportedOperationE...

2018-04-26 Thread dilipbiswal
GitHub user dilipbiswal opened a pull request: https://github.com/apache/spark/pull/21174 [SPARK-24085] Query returns UnsupportedOperationException when scalar subquery is present in partitioning expression ## What changes were proposed in this pull request? In this case, the pa

[GitHub] spark issue #21068: [SPARK-16630][YARN] Blacklist a node if executors won't ...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21068 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21068: [SPARK-16630][YARN] Blacklist a node if executors won't ...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21068 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89898/ Test FAILed. ---

[GitHub] spark issue #21068: [SPARK-16630][YARN] Blacklist a node if executors won't ...

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21068 **[Test build #89898 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89898/testReport)** for PR 21068 at commit [`4df2311`](https://github.com/apache/spark/commit/4

[GitHub] spark pull request #21107: [SPARK-24044][PYTHON] Explicitly print out skippe...

2018-04-26 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21107 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21107: [SPARK-24044][PYTHON] Explicitly print out skipped tests...

2018-04-26 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/21107 merged to master, thanks @HyukjinKwon ! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional com

[GitHub] spark pull request #21088: [SPARK-24003][CORE] Add support to provide spark....

2018-04-26 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/21088#discussion_r184543396 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala --- @@ -892,7 +892,8 @@ private[spark] class Client( // I

[GitHub] spark pull request #21088: [SPARK-24003][CORE] Add support to provide spark....

2018-04-26 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/21088#discussion_r184543613 --- Diff: core/src/main/scala/org/apache/spark/deploy/worker/ExecutorRunner.scala --- @@ -142,7 +142,11 @@ private[deploy] class ExecutorRunner( priva

[GitHub] spark issue #21173: [SPARK-23856][SQL] Add an option `queryTimeout` in JDBCO...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21173 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #21088: [SPARK-24003][CORE] Add support to provide spark....

2018-04-26 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/21088#discussion_r184542727 --- Diff: docs/configuration.md --- @@ -328,6 +328,11 @@ Apart from these, the following properties are also available, and may be useful Note that

[GitHub] spark issue #21173: [SPARK-23856][SQL] Add an option `queryTimeout` in JDBCO...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21173 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2706/ Tes

[GitHub] spark pull request #21088: [SPARK-24003][CORE] Add support to provide spark....

2018-04-26 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/21088#discussion_r184542781 --- Diff: docs/configuration.md --- @@ -328,6 +328,11 @@ Apart from these, the following properties are also available, and may be useful Note that

[GitHub] spark issue #21173: [SPARK-23856][SQL] Add an option `queryTimeout` in JDBCO...

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21173 **[Test build #89905 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89905/testReport)** for PR 21173 at commit [`f134548`](https://github.com/apache/spark/commit/f1

[GitHub] spark pull request #21173: [SPARK-23856][SQL] Add an option `queryTimeout` i...

2018-04-26 Thread maropu
GitHub user maropu opened a pull request: https://github.com/apache/spark/pull/21173 [SPARK-23856][SQL] Add an option `queryTimeout` in JDBCOptions ## What changes were proposed in this pull request? This pr added an option `queryTimeout` for the number of seconds the the drive

[GitHub] spark issue #21088: [SPARK-24003][CORE] Add support to provide spark.executo...

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21088 **[Test build #89904 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89904/testReport)** for PR 21088 at commit [`1758577`](https://github.com/apache/spark/commit/17

[GitHub] spark issue #21171: [SPARK-24104] SQLAppStatusListener overwrites metrics on...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21171 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89897/ Test PASSed. ---

[GitHub] spark issue #21171: [SPARK-24104] SQLAppStatusListener overwrites metrics on...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21171 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21171: [SPARK-24104] SQLAppStatusListener overwrites metrics on...

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21171 **[Test build #89897 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89897/testReport)** for PR 21171 at commit [`69e09cc`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #21159: [SPARK-24057][PYTHON]put the real data type in the Asser...

2018-04-26 Thread huaxingao
Github user huaxingao commented on the issue: https://github.com/apache/spark/pull/21159 Thanks @BryanCutler @HyukjinKwon --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark issue #21068: [SPARK-16630][YARN] Blacklist a node if executors won't ...

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21068 **[Test build #89903 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89903/testReport)** for PR 21068 at commit [`17bbbee`](https://github.com/apache/spark/commit/17

[GitHub] spark issue #20761: [SPARK-20327][CORE][YARN] Add CLI support for YARN custo...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20761 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21159: [SPARK-24057][PYTHON]put the real data type in the Asser...

2018-04-26 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/21159 merged to master, thanks @huaxingao ! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #21159: [SPARK-24057][PYTHON]put the real data type in th...

2018-04-26 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21159 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21095: [SPARK-23529][K8s] Support mounting hostPath volumes

2018-04-26 Thread liyinan926
Github user liyinan926 commented on the issue: https://github.com/apache/spark/pull/21095 @madanadit sorry I didn't get a chance to look into this. I will take a detailed look once @foxish's comments on testing are addressed. ---

[GitHub] spark issue #21141: [SPARK-23853][PYSPARK][TEST] Run Hive-related PySpark te...

2018-04-26 Thread bersprockets
Github user bersprockets commented on the issue: https://github.com/apache/spark/pull/21141 @holdenk Yes, see [this jira](https://issues.apache.org/jira/browse/SPARK-23776). If you build with sbt, you need to also run 'build/sbt sql/test:compile' to get udfs from the test files. --

[GitHub] spark issue #21145: [SPARK-24073][SQL]: Rename DataReaderFactory to ReadTask...

2018-04-26 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/21145 @arunmahadevan, the problem is that the current naming is misleading. This is not a factory (it only produces one specific reader) and it does not have the same lifecycle as the write-side factory. U

[GitHub] spark pull request #21136: [SPARK-24061][SS]Add TypedFilter support for cont...

2018-04-26 Thread arunmahadevan
Github user arunmahadevan commented on a diff in the pull request: https://github.com/apache/spark/pull/21136#discussion_r184528290 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/UnsupportedOperationsSuite.scala --- @@ -771,6 +778,16 @@ class Unsuppor

[GitHub] spark issue #21095: [SPARK-23529][K8s] Support mounting hostPath volumes

2018-04-26 Thread foxish
Github user foxish commented on the issue: https://github.com/apache/spark/pull/21095 @madanadit, sorry about the long turnaround time here. I made a cursory pass over it - and it looks good. Before getting deeper into the code, I think we need more exhaustive unit testing for non-hos

[GitHub] spark issue #21095: [SPARK-23529][K8s] Support mounting hostPath volumes

2018-04-26 Thread madanadit
Github user madanadit commented on the issue: https://github.com/apache/spark/pull/21095 Hey @liyinan926, do you think you'll have time to take a look this week? --- - To unsubscribe, e-mail: reviews-unsubscr...@spa

[GitHub] spark issue #21170: [SPARK-22732][SS][FOLLOW-UP] Fix memoryV2.scala toString...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21170 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89895/ Test PASSed. ---

[GitHub] spark issue #21170: [SPARK-22732][SS][FOLLOW-UP] Fix memoryV2.scala toString...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21170 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21170: [SPARK-22732][SS][FOLLOW-UP] Fix memoryV2.scala toString...

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21170 **[Test build #89895 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89895/testReport)** for PR 21170 at commit [`11b9b90`](https://github.com/apache/spark/commit/1

[GitHub] spark issue #21172: [SPARK-23120][PYSPARK][ML][WIP] Add basic PMML export su...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21172 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21172: [SPARK-23120][PYSPARK][ML][WIP] Add basic PMML export su...

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21172 **[Test build #89902 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89902/testReport)** for PR 21172 at commit [`64d6db8`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #21172: [SPARK-23120][PYSPARK][ML][WIP] Add basic PMML export su...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21172 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89902/ Test FAILed. ---

[GitHub] spark issue #21157: [SPARK-22674][PYTHON] Removed the namedtuple pickling pa...

2018-04-26 Thread superbobry
Github user superbobry commented on the issue: https://github.com/apache/spark/pull/21157 One improvement we can make is change the patch to bypass namedtuples which are importable. This would resolve the issues with namedtuples coming from third-party libraries. I can open a new PR d

[GitHub] spark issue #20522: [SPARK-23355][SQL] convertMetastore should not ignore ta...

2018-04-26 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20522 @gatorsmile . Sorry for late updating. I updated this PR by narrowing the scope of configuration key names specifically for ORC and Parquet. The test coverage is reading and writing non-partit

[GitHub] spark issue #21172: [SPARK-23120][PYSPARK][ML][WIP] Add basic PMML export su...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21172 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2705/ Tes

[GitHub] spark issue #21172: [SPARK-23120][PYSPARK][ML][WIP] Add basic PMML export su...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21172 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21169: [SPARK-23715][SQL] the input of to/from_utc_timestamp ca...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21169 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89893/ Test PASSed. ---

[GitHub] spark issue #21169: [SPARK-23715][SQL] the input of to/from_utc_timestamp ca...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21169 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21169: [SPARK-23715][SQL] the input of to/from_utc_timestamp ca...

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21169 **[Test build #89893 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89893/testReport)** for PR 21169 at commit [`7c1dcc3`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #21172: [SPARK-23120][PYSPARK][ML][WIP] Add basic PMML export su...

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21172 **[Test build #89902 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89902/testReport)** for PR 21172 at commit [`64d6db8`](https://github.com/apache/spark/commit/64

[GitHub] spark issue #20522: [SPARK-23355][SQL] convertMetastore should not ignore ta...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20522 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20522: [SPARK-23355][SQL] convertMetastore should not ignore ta...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20522 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2704/ Tes

[GitHub] spark issue #21141: [SPARK-23853][PYSPARK][TEST] Run Hive-related PySpark te...

2018-04-26 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/21141 Also weirdly when I run this locally without hive built I get some UDF registration exceptions (could be unrelated) - do you get that? --- -

[GitHub] spark issue #20522: [SPARK-23355][SQL] convertMetastore should not ignore ta...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20522 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2703/ Tes

[GitHub] spark issue #20522: [SPARK-23355][SQL] convertMetastore should not ignore ta...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20522 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20522: [SPARK-23355][SQL] convertMetastore should not ignore ta...

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20522 **[Test build #89901 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89901/testReport)** for PR 20522 at commit [`06a9a45`](https://github.com/apache/spark/commit/06

[GitHub] spark issue #21172: [SPARK-23120][PYSPARK][ML][WIP] Add basic PMML export su...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21172 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89900/ Test FAILed. ---

[GitHub] spark issue #21172: [SPARK-23120][PYSPARK][ML][WIP] Add basic PMML export su...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21172 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21172: [SPARK-23120][PYSPARK][ML][WIP] Add basic PMML export su...

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21172 **[Test build #89900 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89900/testReport)** for PR 21172 at commit [`9b14a1c`](https://github.com/apache/spark/commit/9

[GitHub] spark issue #20280: [SPARK-22232][PYTHON][SQL] Fixed Row pickling to include...

2018-04-26 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/20280 I'm worried that people might have two rows with different meanings but the same type and their application will start producing garbage #s. I think a lot of people go from RDDs of Rows to DFs in Py

[GitHub] spark pull request #21168: added check to ensure main method is found [SPARK...

2018-04-26 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/21168#discussion_r184506540 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala --- @@ -675,9 +675,14 @@ private[spark] class Applicati

[GitHub] spark pull request #21168: added check to ensure main method is found [SPARK...

2018-04-26 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/21168#discussion_r184506499 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala --- @@ -675,9 +675,14 @@ private[spark] class Applicati

[GitHub] spark issue #21172: [SPARK-23120][PYSPARK][ML][WIP] Add basic PMML export su...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21172 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21172: [SPARK-23120][PYSPARK][ML][WIP] Add basic PMML export su...

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21172 **[Test build #89899 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89899/testReport)** for PR 21172 at commit [`8cbef82`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #21172: [SPARK-23120][PYSPARK][ML][WIP] Add basic PMML export su...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21172 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89899/ Test PASSed. ---

[GitHub] spark issue #21172: [SPARK-23120][PYSPARK][ML][WIP] Add basic PMML export su...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21172 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21172: [SPARK-23120][PYSPARK][ML][WIP] Add basic PMML export su...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21172 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2702/ Tes

[GitHub] spark issue #21172: [SPARK-23120][PYSPARK][ML][WIP] Add basic PMML export su...

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21172 **[Test build #89900 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89900/testReport)** for PR 21172 at commit [`9b14a1c`](https://github.com/apache/spark/commit/9b

[GitHub] spark pull request #21141: [SPARK-23853][PYSPARK][TEST] Run Hive-related PyS...

2018-04-26 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/21141#discussion_r184504258 --- Diff: python/pyspark/sql/tests.py --- @@ -3021,6 +3021,17 @@ def test_sort_with_nulls_order(self): class HiveSparkSubmitTests(SparkSubmitTests

[GitHub] spark pull request #21141: [SPARK-23853][PYSPARK][TEST] Run Hive-related PyS...

2018-04-26 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/21141#discussion_r184503219 --- Diff: python/pyspark/sql/tests.py --- @@ -3021,6 +3021,17 @@ def test_sort_with_nulls_order(self): class HiveSparkSubmitTests(SparkSubmitTests

[GitHub] spark issue #21172: [SPARK-23120][PYSPARK][ML][WIP] Add basic PMML export su...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21172 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21172: [SPARK-23120][PYSPARK][ML][WIP] Add basic PMML export su...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21172 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2701/ Tes

[GitHub] spark issue #21172: [SPARK-23120][PYSPARK][ML][WIP] Add basic PMML export su...

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21172 **[Test build #89899 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89899/testReport)** for PR 21172 at commit [`8cbef82`](https://github.com/apache/spark/commit/8c

[GitHub] spark pull request #21172: [SPARK-23120][PYSPARK][ML][WIP] Add basic PMML ex...

2018-04-26 Thread holdenk
GitHub user holdenk opened a pull request: https://github.com/apache/spark/pull/21172 [SPARK-23120][PYSPARK][ML][WIP] Add basic PMML export support to PySpark ## What changes were proposed in this pull request? Adds basic PMML export support for Spark ML stages to PySpark as

[GitHub] spark issue #21068: [SPARK-16630][YARN] Blacklist a node if executors won't ...

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21068 **[Test build #89898 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89898/testReport)** for PR 21068 at commit [`4df2311`](https://github.com/apache/spark/commit/4d

[GitHub] spark issue #21171: [SPARK-24104] SQLAppStatusListener overwrites metrics on...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21171 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21171: [SPARK-24104] SQLAppStatusListener overwrites metrics on...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21171 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2700/ Tes

[GitHub] spark issue #21171: [SPARK-24104] SQLAppStatusListener overwrites metrics on...

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21171 **[Test build #89897 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89897/testReport)** for PR 21171 at commit [`69e09cc`](https://github.com/apache/spark/commit/69

[GitHub] spark issue #21145: [SPARK-24073][SQL]: Rename DataReaderFactory to ReadTask...

2018-04-26 Thread arunmahadevan
Github user arunmahadevan commented on the issue: https://github.com/apache/spark/pull/21145 IMO, its better to keep it the current way. `DataReaderFactory` implies that its something that produces `DataReader` which it does, whereas `ReadTask` is gives a notion that it does

[GitHub] spark issue #21171: [SPARK-24104] SQLAppStatusListener overwrites metrics on...

2018-04-26 Thread juliuszsompolski
Github user juliuszsompolski commented on the issue: https://github.com/apache/spark/pull/21171 cc @gengliangwang @vanzin --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark pull request #21171: [SPARK-24104] SQLAppStatusListener overwrites met...

2018-04-26 Thread juliuszsompolski
GitHub user juliuszsompolski opened a pull request: https://github.com/apache/spark/pull/21171 [SPARK-24104] SQLAppStatusListener overwrites metrics onDriverAccumUpdates instead of updating them ## What changes were proposed in this pull request? Event `SparkListenerDriverA

[GitHub] spark issue #21121: [SPARK-24042][SQL] Collection function: zip_with_index

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21121 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89888/ Test PASSed. ---

[GitHub] spark issue #21121: [SPARK-24042][SQL] Collection function: zip_with_index

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21121 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21121: [SPARK-24042][SQL] Collection function: zip_with_index

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21121 **[Test build #89888 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89888/testReport)** for PR 21121 at commit [`bcd52bd`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #21068: [SPARK-16630][YARN] Blacklist a node if executors won't ...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21068 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89889/ Test FAILed. ---

[GitHub] spark issue #21068: [SPARK-16630][YARN] Blacklist a node if executors won't ...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21068 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #21122: [SPARK-24017] [SQL] Refactor ExternalCatalog to b...

2018-04-26 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21122#discussion_r184475841 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/ExternalCatalog.scala --- @@ -31,10 +30,16 @@ import org.apache.spark.util.L

[GitHub] spark issue #21068: [SPARK-16630][YARN] Blacklist a node if executors won't ...

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21068 **[Test build #89889 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89889/testReport)** for PR 21068 at commit [`0ba8510`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #21014: [SPARK-23941][Mesos] Mesos task failed on specific spark...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21014 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21014: [SPARK-23941][Mesos] Mesos task failed on specific spark...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21014 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89896/ Test PASSed. ---

[GitHub] spark issue #21014: [SPARK-23941][Mesos] Mesos task failed on specific spark...

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21014 **[Test build #89896 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89896/testReport)** for PR 21014 at commit [`0038a23`](https://github.com/apache/spark/commit/0

<    1   2   3   4   5   >