[GitHub] spark issue #19218: [SPARK-21786][SQL] The 'spark.sql.parquet.compression.co...

2017-12-22 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19218 What are multiple expressions? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #19218: [SPARK-21786][SQL] The 'spark.sql.parquet.compression.co...

2017-12-22 Thread fjh100456
Github user fjh100456 commented on the issue: https://github.com/apache/spark/pull/19218 @gatorsmile I'd test manully. When table-level compression not configured, it always take the session level compression, and ignore the existing file compression. Seems like a bug, however,

[GitHub] spark pull request #19984: [SPARK-22789] Map-only continuous processing exec...

2017-12-22 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19984 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19984: [SPARK-22789] Map-only continuous processing execution

2017-12-22 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/19984 Thanks! Merging to master! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19984: [SPARK-22789] Map-only continuous processing execution

2017-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19984 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19984: [SPARK-22789] Map-only continuous processing execution

2017-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19984 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85332/ Test PASSed. ---

[GitHub] spark issue #19984: [SPARK-22789] Map-only continuous processing execution

2017-12-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19984 **[Test build #85332 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85332/testReport)** for PR 19984 at commit

[GitHub] spark pull request #20034: [SPARK-22846][SQL] Fix table owner is null when c...

2017-12-22 Thread BruceXu1991
Github user BruceXu1991 commented on a diff in the pull request: https://github.com/apache/spark/pull/20034#discussion_r158577749 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala --- @@ -186,7 +186,7 @@ private[hive] class HiveClientImpl(

[GitHub] spark issue #20059: [SPARK-22648][K8s] Add documentation covering init conta...

2017-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20059 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85331/ Test PASSed. ---

[GitHub] spark issue #20059: [SPARK-22648][K8s] Add documentation covering init conta...

2017-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20059 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20059: [SPARK-22648][K8s] Add documentation covering init conta...

2017-12-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20059 **[Test build #85331 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85331/testReport)** for PR 20059 at commit

[GitHub] spark issue #20061: [SPARK-22890][TEST] Basic tests for DateTimeOperations

2017-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20061 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20061: [SPARK-22890][TEST] Basic tests for DateTimeOperations

2017-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20061 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85330/ Test PASSed. ---

[GitHub] spark issue #20061: [SPARK-22890][TEST] Basic tests for DateTimeOperations

2017-12-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20061 **[Test build #85330 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85330/testReport)** for PR 20061 at commit

[GitHub] spark issue #19977: [SPARK-22771][SQL] Concatenate binary inputs into a bina...

2017-12-22 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/19977 might have...I'll check the performance. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20004: [Spark-22818][SQL] csv escape of quote escape

2017-12-22 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20004#discussion_r158576836 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVOptions.scala --- @@ -148,6 +149,9 @@ class CSVOptions(

[GitHub] spark issue #20004: [Spark-22818][SQL] csv escape of quote escape

2017-12-22 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20004 Thank you for your investigation! Basically, because the option `escape` is set to `\`, the default value of `charToEscapeQuoteEscaping` is actually `\` in effect. Could you

[GitHub] spark issue #20057: [SPARK-22880][SQL] Add cascadeTruncate option to JDBC da...

2017-12-22 Thread danielvdende
Github user danielvdende commented on the issue: https://github.com/apache/spark/pull/20057 @gatorsmile could you explain why you have doubts about the feature? Thanks! --- - To unsubscribe, e-mail:

[GitHub] spark issue #19977: [SPARK-22771][SQL] Concatenate binary inputs into a bina...

2017-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19977 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85329/ Test PASSed. ---

[GitHub] spark issue #19977: [SPARK-22771][SQL] Concatenate binary inputs into a bina...

2017-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19977 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19977: [SPARK-22771][SQL] Concatenate binary inputs into a bina...

2017-12-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19977 **[Test build #85329 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85329/testReport)** for PR 19977 at commit

[GitHub] spark issue #20023: [SPARK-22036][SQL] Decimal multiplication with high prec...

2017-12-22 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20023 ``` db2 => create table decimals_test(id int, a decimal(31,18), b decimal(31,18)) DB2I The SQL command completed successfully. db2 => insert into decimals_test values (1, 2.33,

[GitHub] spark issue #17619: [SPARK-19755][Mesos] Blacklist is always active for Meso...

2017-12-22 Thread timout
Github user timout commented on the issue: https://github.com/apache/spark/pull/17619 That does exactly what is supposed to do. And you absolutely right it related to executors. I am sorry if it is not clear from my previous explanations. Let us say: Spark Streaming App -

[GitHub] spark issue #19984: [SPARK-22789] Map-only continuous processing execution

2017-12-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19984 **[Test build #85332 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85332/testReport)** for PR 19984 at commit

[GitHub] spark issue #19954: [SPARK-22757][Kubernetes] Enable use of remote dependenc...

2017-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19954 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85327/ Test PASSed. ---

[GitHub] spark issue #19954: [SPARK-22757][Kubernetes] Enable use of remote dependenc...

2017-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19954 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19954: [SPARK-22757][Kubernetes] Enable use of remote dependenc...

2017-12-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19954 **[Test build #85327 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85327/testReport)** for PR 19954 at commit

[GitHub] spark issue #20011: [SPARK-20654][core] Add config to limit disk usage of th...

2017-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20011 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20011: [SPARK-20654][core] Add config to limit disk usage of th...

2017-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20011 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85326/ Test PASSed. ---

[GitHub] spark issue #20011: [SPARK-20654][core] Add config to limit disk usage of th...

2017-12-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20011 **[Test build #85326 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85326/testReport)** for PR 20011 at commit

[GitHub] spark issue #20057: [SPARK-22880][SQL] Add cascadeTruncate option to JDBC da...

2017-12-22 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20057 Thank you for your contribution! I am doubting the value of this feature. If you are interested in the JDBC-related work, could you take

[GitHub] spark issue #20059: [SPARK-22648][K8s] Add documentation covering init conta...

2017-12-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20059 **[Test build #85331 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85331/testReport)** for PR 20059 at commit

[GitHub] spark issue #19977: [SPARK-22771][SQL] Concatenate binary inputs into a bina...

2017-12-22 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19977 Yes; otherwise, it will introduce a performance regression, right? --- - To unsubscribe, e-mail:

[GitHub] spark pull request #19218: [SPARK-21786][SQL] The 'spark.sql.parquet.compres...

2017-12-22 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19218#discussion_r158575144 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetOptions.scala --- @@ -42,8 +43,15 @@ private[parquet] class

[GitHub] spark pull request #20059: [SPARK-22648][K8s] Add documentation covering ini...

2017-12-22 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/20059#discussion_r158575137 --- Diff: docs/running-on-kubernetes.md --- @@ -528,51 +579,90 @@ specific to Spark on Kubernetes. -

[GitHub] spark pull request #20059: [SPARK-22648][K8s] Add documentation covering ini...

2017-12-22 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/20059#discussion_r158575135 --- Diff: docs/running-on-kubernetes.md --- @@ -120,6 +120,57 @@ by their appropriate remote URIs. Also, application dependencies can be pre-moun

[GitHub] spark issue #20061: [SPARK-22890][TEST] Basic tests for DateTimeOperations

2017-12-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20061 **[Test build #85330 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85330/testReport)** for PR 20061 at commit

[GitHub] spark pull request #20061: [SPARK-22890][TEST] Basic tests for DateTimeOpera...

2017-12-22 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/20061 [SPARK-22890][TEST] Basic tests for DateTimeOperations ## What changes were proposed in this pull request? Test Coverage for `DateTimeOperations`, this is a Sub-tasks for

[GitHub] spark pull request #19218: [SPARK-21786][SQL] The 'spark.sql.parquet.compres...

2017-12-22 Thread fjh100456
Github user fjh100456 commented on a diff in the pull request: https://github.com/apache/spark/pull/19218#discussion_r158574986 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetOptions.scala --- @@ -42,8 +43,15 @@ private[parquet] class

[GitHub] spark issue #20059: [SPARK-22648][K8s] Add documentation covering init conta...

2017-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20059 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20059: [SPARK-22648][K8s] Add documentation covering init conta...

2017-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20059 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85325/ Test PASSed. ---

[GitHub] spark issue #20059: [SPARK-22648][K8s] Add documentation covering init conta...

2017-12-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20059 **[Test build #85325 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85325/testReport)** for PR 20059 at commit

[GitHub] spark issue #20059: [SPARK-22648][K8s] Add documentation covering init conta...

2017-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20059 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85323/ Test PASSed. ---

[GitHub] spark issue #20059: [SPARK-22648][K8s] Add documentation covering init conta...

2017-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20059 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20059: [SPARK-22648][K8s] Add documentation covering init conta...

2017-12-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20059 **[Test build #85323 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85323/testReport)** for PR 20059 at commit

[GitHub] spark issue #19954: [SPARK-22757][Kubernetes] Enable use of remote dependenc...

2017-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19954 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19954: [SPARK-22757][Kubernetes] Enable use of remote dependenc...

2017-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19954 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85324/ Test PASSed. ---

[GitHub] spark issue #19954: [SPARK-22757][Kubernetes] Enable use of remote dependenc...

2017-12-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19954 **[Test build #85324 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85324/testReport)** for PR 19954 at commit

[GitHub] spark pull request #20039: [SPARK-22850][core] Ensure queued events are deli...

2017-12-22 Thread Ngone51
Github user Ngone51 commented on a diff in the pull request: https://github.com/apache/spark/pull/20039#discussion_r158574199 --- Diff: core/src/main/scala/org/apache/spark/scheduler/LiveListenerBus.scala --- @@ -125,13 +128,39 @@ private[spark] class LiveListenerBus(conf:

[GitHub] spark pull request #20039: [SPARK-22850][core] Ensure queued events are deli...

2017-12-22 Thread Ngone51
Github user Ngone51 commented on a diff in the pull request: https://github.com/apache/spark/pull/20039#discussion_r158573853 --- Diff: core/src/main/scala/org/apache/spark/scheduler/LiveListenerBus.scala --- @@ -149,7 +158,11 @@ private[spark] class LiveListenerBus(conf:

[GitHub] spark issue #19982: [SPARK-22787] [TEST] [SQL] Add a TPC-H query suite

2017-12-22 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/19982 @gatorsmile Any progress on this? https://github.com/apache/spark/pull/19982#discussion_r157119941 After I thought your comment, I came up with collecting metrics for each rule like;

[GitHub] spark issue #20011: [SPARK-20654][core] Add config to limit disk usage of th...

2017-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20011 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85320/ Test PASSed. ---

[GitHub] spark issue #20011: [SPARK-20654][core] Add config to limit disk usage of th...

2017-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20011 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20011: [SPARK-20654][core] Add config to limit disk usage of th...

2017-12-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20011 **[Test build #85320 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85320/testReport)** for PR 20011 at commit

[GitHub] spark pull request #20039: [SPARK-22850][core] Ensure queued events are deli...

2017-12-22 Thread Ngone51
Github user Ngone51 commented on a diff in the pull request: https://github.com/apache/spark/pull/20039#discussion_r158573360 --- Diff: core/src/main/scala/org/apache/spark/scheduler/LiveListenerBus.scala --- @@ -149,7 +158,11 @@ private[spark] class LiveListenerBus(conf:

[GitHub] spark issue #20060: [SPARK-22889][SPARKR] Set overwrite=T when install Spark...

2017-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20060 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85328/ Test PASSed. ---

[GitHub] spark issue #20060: [SPARK-22889][SPARKR] Set overwrite=T when install Spark...

2017-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20060 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20060: [SPARK-22889][SPARKR] Set overwrite=T when install Spark...

2017-12-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20060 **[Test build #85328 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85328/testReport)** for PR 20060 at commit

[GitHub] spark issue #20059: [SPARK-22648][K8s] Add documentation covering init conta...

2017-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20059 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20059: [SPARK-22648][K8s] Add documentation covering init conta...

2017-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20059 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85321/ Test FAILed. ---

[GitHub] spark issue #20059: [SPARK-22648][K8s] Add documentation covering init conta...

2017-12-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20059 **[Test build #85321 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85321/testReport)** for PR 20059 at commit

[GitHub] spark issue #19929: [SPARK-22629][PYTHON] Add deterministic flag to pyspark ...

2017-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19929 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85322/ Test PASSed. ---

[GitHub] spark issue #19929: [SPARK-22629][PYTHON] Add deterministic flag to pyspark ...

2017-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19929 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19929: [SPARK-22629][PYTHON] Add deterministic flag to pyspark ...

2017-12-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19929 **[Test build #85322 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85322/testReport)** for PR 19929 at commit

[GitHub] spark issue #19977: [SPARK-22771][SQL] Concatenate binary inputs into a bina...

2017-12-22 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/19977 I found the optimizer rule can't combine nested concat like; ``` scala>: psate val df = sql(""" SELECT ((col1 || col2) || (col3 || col4)) col FROM ( SELECT

[GitHub] spark issue #19977: [SPARK-22771][SQL] Concatenate binary inputs into a bina...

2017-12-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19977 **[Test build #85329 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85329/testReport)** for PR 19977 at commit

[GitHub] spark pull request #20039: [SPARK-22850][core] Ensure queued events are deli...

2017-12-22 Thread Ngone51
Github user Ngone51 commented on a diff in the pull request: https://github.com/apache/spark/pull/20039#discussion_r158572288 --- Diff: core/src/main/scala/org/apache/spark/scheduler/LiveListenerBus.scala --- @@ -149,7 +158,11 @@ private[spark] class LiveListenerBus(conf:

[GitHub] spark pull request #19977: [SPARK-22771][SQL] Concatenate binary inputs into...

2017-12-22 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/19977#discussion_r158572201 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala --- @@ -2171,7 +2171,8 @@ object functions { def base64(e: Column): Column =

[GitHub] spark pull request #20053: [SPARK-22873] [CORE] Init lastReportTimestamp wit...

2017-12-22 Thread Ngone51
Github user Ngone51 commented on a diff in the pull request: https://github.com/apache/spark/pull/20053#discussion_r158572161 --- Diff: core/src/main/scala/org/apache/spark/scheduler/AsyncEventQueue.scala --- @@ -112,6 +112,7 @@ private class AsyncEventQueue(val name: String,

[GitHub] spark issue #20060: [SPARK-22889][SPARKR] Set overwrite=T when install Spark...

2017-12-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20060 **[Test build #85328 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85328/testReport)** for PR 20060 at commit

[GitHub] spark issue #20060: [SPARK-22889][SPARKR] Set overwrite=T when install Spark...

2017-12-22 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/20060 cc @felixcheung --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20060: [SPARK-22889][SPARKR] Set overwrite=T when instal...

2017-12-22 Thread shivaram
GitHub user shivaram opened a pull request: https://github.com/apache/spark/pull/20060 [SPARK-22889][SPARKR] Set overwrite=T when install SparkR in tests ## What changes were proposed in this pull request? Since all CRAN checks go through the same machine, if there is an

[GitHub] spark pull request #19218: [SPARK-21786][SQL] The 'spark.sql.parquet.compres...

2017-12-22 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19218#discussion_r158571702 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetOptions.scala --- @@ -42,8 +43,15 @@ private[parquet] class

[GitHub] spark pull request #19218: [SPARK-21786][SQL] The 'spark.sql.parquet.compres...

2017-12-22 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19218#discussion_r158571657 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/HiveOptions.scala --- @@ -102,4 +111,18 @@ object HiveOptions {

[GitHub] spark pull request #19218: [SPARK-21786][SQL] The 'spark.sql.parquet.compres...

2017-12-22 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19218#discussion_r158571649 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/InsertSuite.scala --- @@ -35,7 +39,7 @@ case class TestData(key: Int, value: String)

[GitHub] spark pull request #19929: [SPARK-22629][PYTHON] Add deterministic flag to p...

2017-12-22 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19929#discussion_r158571445 --- Diff: python/pyspark/sql/functions.py --- @@ -2075,9 +2075,10 @@ class PandasUDFType(object): def udf(f=None, returnType=StringType()): ---

[GitHub] spark pull request #19929: [SPARK-22629][PYTHON] Add deterministic flag to p...

2017-12-22 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19929#discussion_r158571371 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/UDFRegistration.scala --- @@ -58,6 +58,7 @@ class UDFRegistration private[sql] (functionRegistry:

[GitHub] spark pull request #19929: [SPARK-22629][PYTHON] Add deterministic flag to p...

2017-12-22 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19929#discussion_r158571355 --- Diff: python/pyspark/sql/udf.py --- @@ -157,5 +158,13 @@ def wrapper(*args): wrapper.func = self.func wrapper.returnType =

[GitHub] spark pull request #19929: [SPARK-22629][PYTHON] Add deterministic flag to p...

2017-12-22 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19929#discussion_r158571320 --- Diff: python/pyspark/sql/tests.py --- @@ -434,6 +434,16 @@ def test_udf_with_array_type(self): self.assertEqual(list(range(3)), l1)

[GitHub] spark issue #19954: [SPARK-22757][Kubernetes] Enable use of remote dependenc...

2017-12-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19954 **[Test build #85327 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85327/testReport)** for PR 19954 at commit

[GitHub] spark pull request #20059: [SPARK-22648][Kubernetes] Update documentation to...

2017-12-22 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20059#discussion_r158570531 --- Diff: docs/running-on-kubernetes.md --- @@ -120,6 +120,57 @@ by their appropriate remote URIs. Also, application dependencies can be pre-moun Those

[GitHub] spark pull request #20059: [SPARK-22648][Kubernetes] Update documentation to...

2017-12-22 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20059#discussion_r158570653 --- Diff: docs/running-on-kubernetes.md --- @@ -528,51 +579,90 @@ specific to Spark on Kubernetes. -

[GitHub] spark issue #20011: [SPARK-20654][core] Add config to limit disk usage of th...

2017-12-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20011 **[Test build #85326 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85326/testReport)** for PR 20011 at commit

[GitHub] spark pull request #20011: [SPARK-20654][core] Add config to limit disk usag...

2017-12-22 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20011#discussion_r158569095 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/HistoryServerDiskManager.scala --- @@ -0,0 +1,310 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #20059: [SPARK-22648][Kubernetes] Update documentation to...

2017-12-22 Thread foxish
Github user foxish commented on a diff in the pull request: https://github.com/apache/spark/pull/20059#discussion_r158568498 --- Diff: docs/running-on-kubernetes.md --- @@ -120,6 +120,23 @@ by their appropriate remote URIs. Also, application dependencies can be pre-moun Those

[GitHub] spark pull request #20011: [SPARK-20654][core] Add config to limit disk usag...

2017-12-22 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/20011#discussion_r158568388 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/HistoryServerDiskManager.scala --- @@ -0,0 +1,310 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #20059: [SPARK-22648][Kubernetes] Update documentation to cover ...

2017-12-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20059 **[Test build #85325 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85325/testReport)** for PR 20059 at commit

[GitHub] spark issue #20059: [SPARK-22648][Kubernetes] Update documentation to cover ...

2017-12-22 Thread liyinan926
Github user liyinan926 commented on the issue: https://github.com/apache/spark/pull/20059 Updated in https://github.com/apache/spark/pull/20059/commits/fbb21121447fe131358042ce05454f88d6fb. --- - To

[GitHub] spark issue #20039: [SPARK-22850][core] Ensure queued events are delivered t...

2017-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20039 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85317/ Test PASSed. ---

[GitHub] spark issue #20039: [SPARK-22850][core] Ensure queued events are delivered t...

2017-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20039 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20059: [SPARK-22648][Kubernetes] Update documentation to...

2017-12-22 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/20059#discussion_r158568089 --- Diff: docs/running-on-kubernetes.md --- @@ -120,6 +120,23 @@ by their appropriate remote URIs. Also, application dependencies can be pre-moun

[GitHub] spark issue #20039: [SPARK-22850][core] Ensure queued events are delivered t...

2017-12-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20039 **[Test build #85317 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85317/testReport)** for PR 20039 at commit

[GitHub] spark pull request #18754: [SPARK-21552][SQL] Add DecimalType support to Arr...

2017-12-22 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/18754#discussion_r158566190 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/arrow/ArrowWriter.scala --- @@ -214,6 +216,22 @@ private[arrow] class

[GitHub] spark pull request #18754: [SPARK-21552][SQL] Add DecimalType support to Arr...

2017-12-22 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/18754#discussion_r158567491 --- Diff: python/pyspark/sql/types.py --- @@ -1617,7 +1617,7 @@ def to_arrow_type(dt): elif type(dt) == DoubleType: arrow_type =

[GitHub] spark issue #20045: [Spark-22360][SQL] Add unit tests for Window Specificati...

2017-12-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20045 **[Test build #4023 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4023/testReport)** for PR 20045 at commit

[GitHub] spark pull request #20059: [SPARK-22648][Kubernetes] Update documentation to...

2017-12-22 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/20059#discussion_r158567222 --- Diff: docs/running-on-kubernetes.md --- @@ -120,6 +120,23 @@ by their appropriate remote URIs. Also, application dependencies can be pre-moun

[GitHub] spark issue #19954: [SPARK-22757][Kubernetes] Enable use of remote dependenc...

2017-12-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19954 **[Test build #85324 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85324/testReport)** for PR 19954 at commit

[GitHub] spark pull request #20059: [SPARK-22648][Kubernetes] Update documentation to...

2017-12-22 Thread foxish
Github user foxish commented on a diff in the pull request: https://github.com/apache/spark/pull/20059#discussion_r158566909 --- Diff: docs/running-on-kubernetes.md --- @@ -120,6 +120,23 @@ by their appropriate remote URIs. Also, application dependencies can be pre-moun Those

[GitHub] spark pull request #20059: [SPARK-22648][Kubernetes] Update documentation to...

2017-12-22 Thread foxish
Github user foxish commented on a diff in the pull request: https://github.com/apache/spark/pull/20059#discussion_r158566870 --- Diff: docs/running-on-kubernetes.md --- @@ -120,6 +120,23 @@ by their appropriate remote URIs. Also, application dependencies can be pre-moun Those

[GitHub] spark issue #20059: [SPARK-22648][Kubernetes] Update documentation to cover ...

2017-12-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20059 **[Test build #85323 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85323/testReport)** for PR 20059 at commit

  1   2   3   4   >