[GitHub] spark issue #21143: [SPARK-24072][SQL] clearly define pushed filters

2018-04-26 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21143 LGTM except above comments. This PR does not change anything else but just clean the implementation of existing Data Source V2 API --- --

[GitHub] spark pull request #21175: [SPARK-24107][CORE] ChunkedByteBuffer.writeFully ...

2018-04-26 Thread manbuyun
Github user manbuyun commented on a diff in the pull request: https://github.com/apache/spark/pull/21175#discussion_r184603606 --- Diff: core/src/test/scala/org/apache/spark/io/ChunkedByteBufferSuite.scala --- @@ -56,6 +56,12 @@ class ChunkedByteBufferSuite extends SparkFunSuite {

[GitHub] spark pull request #21175: [SPARK-24107][CORE] ChunkedByteBuffer.writeFully ...

2018-04-26 Thread manbuyun
Github user manbuyun commented on a diff in the pull request: https://github.com/apache/spark/pull/21175#discussion_r184603588 --- Diff: core/src/test/scala/org/apache/spark/io/ChunkedByteBufferSuite.scala --- @@ -56,6 +56,12 @@ class ChunkedByteBufferSuite extends SparkFunSuite {

[GitHub] spark issue #21141: [SPARK-23853][PYSPARK][TEST] Run Hive-related PySpark te...

2018-04-26 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/21141 Thank you for review, @HyukjinKwon , @holdenk , @bersprockets . I didn't notice SPARK-23776 when I chose SPARK-23853 . I think we can merge those PRs now. @bersprockets . Coul

[GitHub] spark pull request #21175: [SPARK-24107][CORE] ChunkedByteBuffer.writeFully ...

2018-04-26 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21175#discussion_r184602233 --- Diff: core/src/test/scala/org/apache/spark/io/ChunkedByteBufferSuite.scala --- @@ -56,6 +56,12 @@ class ChunkedByteBufferSuite extends SparkFunSuite {

[GitHub] spark pull request #21143: [SPARK-24072][SQL] clearly define pushed filters

2018-04-26 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21143#discussion_r184601974 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala --- @@ -224,24 +226,29 @@ object DataSourceV2

[GitHub] spark pull request #21143: [SPARK-24072][SQL] clearly define pushed filters

2018-04-26 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21143#discussion_r184601574 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala --- @@ -224,24 +226,29 @@ object DataSourceV2

[GitHub] spark pull request #21143: [SPARK-24072][SQL] clearly define pushed filters

2018-04-26 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21143#discussion_r184600798 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala --- @@ -224,24 +226,29 @@ object DataSourceV2

[GitHub] spark pull request #21143: [SPARK-24072][SQL] clearly define pushed filters

2018-04-26 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21143#discussion_r184600589 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala --- @@ -56,7 +56,7 @@ case class DataSourceV2

[GitHub] spark pull request #21143: [SPARK-24072][SQL] clearly define pushed filters

2018-04-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21143#discussion_r184599222 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala --- @@ -56,7 +56,7 @@ case class DataSourceV2R

[GitHub] spark issue #21136: [SPARK-24061][SS]Add TypedFilter support for continuous ...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21136 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21136: [SPARK-24061][SS]Add TypedFilter support for continuous ...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21136 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89912/ Test PASSed. ---

[GitHub] spark issue #21136: [SPARK-24061][SS]Add TypedFilter support for continuous ...

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21136 **[Test build #89912 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89912/testReport)** for PR 21136 at commit [`77ee1c7`](https://github.com/apache/spark/commit/7

[GitHub] spark pull request #21143: [SPARK-24072][SQL] clearly define pushed filters

2018-04-26 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21143#discussion_r184598059 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala --- @@ -56,7 +56,7 @@ case class DataSourceV2

[GitHub] spark pull request #21175: [SPARK-24107][CORE] ChunkedByteBuffer.writeFully ...

2018-04-26 Thread Ngone51
Github user Ngone51 commented on a diff in the pull request: https://github.com/apache/spark/pull/21175#discussion_r184597197 --- Diff: core/src/test/scala/org/apache/spark/io/ChunkedByteBufferSuite.scala --- @@ -56,6 +56,12 @@ class ChunkedByteBufferSuite extends SparkFunSuite {

[GitHub] spark pull request #21175: [SPARK-24107][CORE] ChunkedByteBuffer.writeFully ...

2018-04-26 Thread Ngone51
Github user Ngone51 commented on a diff in the pull request: https://github.com/apache/spark/pull/21175#discussion_r184596199 --- Diff: core/src/test/scala/org/apache/spark/io/ChunkedByteBufferSuite.scala --- @@ -56,6 +56,12 @@ class ChunkedByteBufferSuite extends SparkFunSuite {

[GitHub] spark pull request #21169: [SPARK-23715][SQL] the input of to/from_utc_times...

2018-04-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21169#discussion_r184597402 --- Diff: docs/sql-programming-guide.md --- @@ -1805,12 +1805,13 @@ working with timestamps in `pandas_udf`s to get the best performance, see

[GitHub] spark pull request #21169: [SPARK-23715][SQL] the input of to/from_utc_times...

2018-04-26 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/21169#discussion_r184596334 --- Diff: docs/sql-programming-guide.md --- @@ -1805,12 +1805,13 @@ working with timestamps in `pandas_udf`s to get the best performance, see - Si

[GitHub] spark issue #21174: [SPARK-24085][SQL] Query returns UnsupportedOperationExc...

2018-04-26 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/21174 @maropu So with the fix, if the query predicate contains an scalar subquery expression, then that expression is not considered for partition pruning. For example, if the predicate was , part_ke

[GitHub] spark issue #21165: [Spark-20087][CORE] Attach accumulators / metrics to 'Ta...

2018-04-26 Thread advancedxy
Github user advancedxy commented on the issue: https://github.com/apache/spark/pull/21165 > We should not do these 2 things together, and to me the second one is way simpler to get in and we should do it first. Agreed. For the scope of this pr, let's get killed tasks's accumul

[GitHub] spark pull request #21175: [SPARK-24107][CORE] ChunkedByteBuffer.writeFully ...

2018-04-26 Thread manbuyun
Github user manbuyun commented on a diff in the pull request: https://github.com/apache/spark/pull/21175#discussion_r184594822 --- Diff: core/src/test/scala/org/apache/spark/io/ChunkedByteBufferSuite.scala --- @@ -56,6 +56,12 @@ class ChunkedByteBufferSuite extends SparkFunSuite {

[GitHub] spark issue #21176: [SPARK-24109] Remove class SnappyOutputStreamWrapper

2018-04-26 Thread manbuyun
Github user manbuyun commented on the issue: https://github.com/apache/spark/pull/21176 @maropu Ok. It sounds reasonable to me --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional command

[GitHub] spark pull request #21176: [SPARK-24109] Remove class SnappyOutputStreamWrap...

2018-04-26 Thread manbuyun
Github user manbuyun closed the pull request at: https://github.com/apache/spark/pull/21176 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21165: [Spark-20087][CORE] Attach accumulators / metrics to 'Ta...

2018-04-26 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21165 > For example user may want to record CPU time for every task and get the total CPU time for the application. The problem is, shall we allow end users to collect metrics via accumulators?

[GitHub] spark issue #21088: [SPARK-24003][CORE] Add support to provide spark.executo...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21088 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21088: [SPARK-24003][CORE] Add support to provide spark.executo...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21088 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89909/ Test PASSed. ---

[GitHub] spark issue #21088: [SPARK-24003][CORE] Add support to provide spark.executo...

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21088 **[Test build #89909 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89909/testReport)** for PR 21088 at commit [`932b7d1`](https://github.com/apache/spark/commit/9

[GitHub] spark issue #21169: [SPARK-23715][SQL] the input of to/from_utc_timestamp ca...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21169 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2709/ Tes

[GitHub] spark issue #21169: [SPARK-23715][SQL] the input of to/from_utc_timestamp ca...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21169 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21169: [SPARK-23715][SQL] the input of to/from_utc_timestamp ca...

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21169 **[Test build #89915 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89915/testReport)** for PR 21169 at commit [`d248d4c`](https://github.com/apache/spark/commit/d2

[GitHub] spark pull request #21175: [SPARK-24107][CORE] ChunkedByteBuffer.writeFully ...

2018-04-26 Thread Ngone51
Github user Ngone51 commented on a diff in the pull request: https://github.com/apache/spark/pull/21175#discussion_r184590989 --- Diff: core/src/test/scala/org/apache/spark/io/ChunkedByteBufferSuite.scala --- @@ -56,6 +56,12 @@ class ChunkedByteBufferSuite extends SparkFunSuite {

[GitHub] spark issue #21088: [SPARK-24003][CORE] Add support to provide spark.executo...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21088 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21088: [SPARK-24003][CORE] Add support to provide spark.executo...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21088 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89908/ Test PASSed. ---

[GitHub] spark issue #21174: [SPARK-24085][SQL] Query returns UnsupportedOperationExc...

2018-04-26 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21174 One question; we have no risk to miss any partition pruning by this fix? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.a

[GitHub] spark issue #21088: [SPARK-24003][CORE] Add support to provide spark.executo...

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21088 **[Test build #89908 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89908/testReport)** for PR 21088 at commit [`27cc29c`](https://github.com/apache/spark/commit/2

[GitHub] spark pull request #21169: [SPARK-23715][SQL] the input of to/from_utc_times...

2018-04-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21169#discussion_r184590539 --- Diff: docs/sql-programming-guide.md --- @@ -1805,12 +1805,13 @@ working with timestamps in `pandas_udf`s to get the best performance, see

[GitHub] spark issue #21175: [SPARK-24107] ChunkedByteBuffer.writeFully method has no...

2018-04-26 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21175 Plz add `[CORE]` in the title. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21176: [SPARK-24109] Remove class SnappyOutputStreamWrapper

2018-04-26 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21176 Have you checked the @srowen 's comment? https://github.com/apache/spark/pull/18949#issuecomment-323354674 --- - To unsubscribe,

[GitHub] spark issue #21169: [SPARK-23715][SQL] the input of to/from_utc_timestamp ca...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21169 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2708/ Tes

[GitHub] spark issue #21131: [SPARK-23433][CORE] Late zombie task completions update ...

2018-04-26 Thread Ngone51
Github user Ngone51 commented on the issue: https://github.com/apache/spark/pull/21131 LGTM, and nice UT. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h.

[GitHub] spark issue #21169: [SPARK-23715][SQL] the input of to/from_utc_timestamp ca...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21169 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #21169: [SPARK-23715][SQL] the input of to/from_utc_times...

2018-04-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21169#discussion_r184589060 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala --- @@ -782,6 +782,22 @@ object TypeCoercion {

[GitHub] spark issue #21169: [SPARK-23715][SQL] the input of to/from_utc_timestamp ca...

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21169 **[Test build #89914 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89914/testReport)** for PR 21169 at commit [`cbe37f2`](https://github.com/apache/spark/commit/cb

[GitHub] spark issue #21175: [SPARK-24107] ChunkedByteBuffer.writeFully method has no...

2018-04-26 Thread Ngone51
Github user Ngone51 commented on the issue: https://github.com/apache/spark/pull/21175 @manbuyun you need to add the unit test into `ChunkedByteBufferSuite.scala` and push a new commit. --- - To unsubscribe, e-mail:

[GitHub] spark issue #21176: [SPARK-24109] Remove class SnappyOutputStreamWrapper

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21176 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21176: [SPARK-24109] Remove class SnappyOutputStreamWrapper

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21176 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21176: [SPARK-24109] Remove class SnappyOutputStreamWrap...

2018-04-26 Thread manbuyun
GitHub user manbuyun opened a pull request: https://github.com/apache/spark/pull/21176 [SPARK-24109] Remove class SnappyOutputStreamWrapper JIRA: https://issues.apache.org/jira/browse/SPARK-24109?jql=text%20~%20%22SnappyOutputStreamWrapper%22 Wrapper over `SnappyOutputStrea

[GitHub] spark issue #21165: [Spark-20087][CORE] Attach accumulators / metrics to 'Ta...

2018-04-26 Thread advancedxy
Github user advancedxy commented on the issue: https://github.com/apache/spark/pull/21165 > However, I don't agree user side accumulators should get updates from killed tasks, that changes the semantic of accumulators. And I don't think end-users need to care about killed tasks. Simil

[GitHub] spark issue #20522: [SPARK-23355][SQL] convertMetastore should not ignore ta...

2018-04-26 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20522 Thank you for review and merge, @cloud-fan ! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additio

[GitHub] spark issue #21175: [SPARK-24107] ChunkedByteBuffer.writeFully method has no...

2018-04-26 Thread manbuyun
Github user manbuyun commented on the issue: https://github.com/apache/spark/pull/21175 test("writeFully() does not affect original buffer's position") { val chunkedByteBuffer = new ChunkedByteBuffer(Array(ByteBuffer.allocate(80*1024*1024))) chunkedByteBuffer.writeFul

[GitHub] spark issue #21175: [SPARK-24107] ChunkedByteBuffer.writeFully method has no...

2018-04-26 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21175 Would it be possible to add a unit test? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands

[GitHub] spark issue #21168: [SPARK-23830][YARN] added check to ensure main method is...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21168 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21168: [SPARK-23830][YARN] added check to ensure main method is...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21168 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89913/ Test PASSed. ---

[GitHub] spark issue #21168: [SPARK-23830][YARN] added check to ensure main method is...

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21168 **[Test build #89913 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89913/testReport)** for PR 21168 at commit [`10a6232`](https://github.com/apache/spark/commit/1

[GitHub] spark issue #17086: [SPARK-24101][ML][MLLIB] ML Evaluators should use weight...

2018-04-26 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/17086 overall good, @jkbradley Would you mind take a look ? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #17086: [SPARK-24101][ML][MLLIB] ML Evaluators should use...

2018-04-26 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/17086#discussion_r184584878 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/evaluation/MulticlassMetricsSuite.scala --- @@ -55,44 +60,128 @@ class MulticlassMetricsSuite ex

[GitHub] spark issue #21168: [SPARK-23830][YARN] added check to ensure main method is...

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21168 **[Test build #89913 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89913/testReport)** for PR 21168 at commit [`10a6232`](https://github.com/apache/spark/commit/10

[GitHub] spark issue #21175: [SPARK-24107] ChunkedByteBuffer.writeFully method has no...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21175 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21165: [Spark-20087][CORE] Attach accumulators / metrics to 'Ta...

2018-04-26 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21165 I do agree task killed event should carry metrics update, as it's reasonable to count killed tasks for something like how many bytes were read from files. However, I don't agree user side

[GitHub] spark issue #21175: [SPARK-24107] ChunkedByteBuffer.writeFully method has no...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21175 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21175: [SPARK-24107] ChunkedByteBuffer.writeFully method...

2018-04-26 Thread manbuyun
GitHub user manbuyun opened a pull request: https://github.com/apache/spark/pull/21175 [SPARK-24107] ChunkedByteBuffer.writeFully method has not reset the limit value JIRA Issue: https://issues.apache.org/jira/browse/SPARK-24107?jql=text%20~%20%22ChunkedByteBuffer%22 Chunk

[GitHub] spark issue #21165: [Spark-20087][CORE] Attach accumulators / metrics to 'Ta...

2018-04-26 Thread advancedxy
Github user advancedxy commented on the issue: https://github.com/apache/spark/pull/21165 I add a note for accumulator update. Please comment if more document is needed. --- - To unsubscribe, e-mail: reviews-unsubsc

[GitHub] spark pull request #20522: [SPARK-23355][SQL] convertMetastore should not ig...

2018-04-26 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20522 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20522: [SPARK-23355][SQL] convertMetastore should not ignore ta...

2018-04-26 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20522 LGTM, merging to master! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: re

[GitHub] spark issue #21174: [SPARK-24085] Query returns UnsupportedOperationExceptio...

2018-04-26 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21174 plz add `[SQL]` in the title. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21174: [SPARK-24085] Query returns UnsupportedOperationExceptio...

2018-04-26 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21174 Ah, ok. Thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@s

[GitHub] spark issue #21168: [SPARK-23830][CORE] added check to ensure main method is...

2018-04-26 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/21168 Also this is a yarn, not core, change. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #21136: [SPARK-24061][SS]Add TypedFilter support for continuous ...

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21136 **[Test build #89912 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89912/testReport)** for PR 21136 at commit [`77ee1c7`](https://github.com/apache/spark/commit/77

[GitHub] spark issue #21088: [SPARK-24003][CORE] Add support to provide spark.executo...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21088 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21088: [SPARK-24003][CORE] Add support to provide spark.executo...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21088 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89904/ Test PASSed. ---

[GitHub] spark issue #21088: [SPARK-24003][CORE] Add support to provide spark.executo...

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21088 **[Test build #89904 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89904/testReport)** for PR 21088 at commit [`1758577`](https://github.com/apache/spark/commit/1

[GitHub] spark pull request #21136: [SPARK-24061][SS]Add TypedFilter support for cont...

2018-04-26 Thread yanlin-Lynn
Github user yanlin-Lynn commented on a diff in the pull request: https://github.com/apache/spark/pull/21136#discussion_r184576733 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/UnsupportedOperationsSuite.scala --- @@ -771,6 +778,16 @@ class Unsupporte

[GitHub] spark issue #21174: [SPARK-24085] Query returns UnsupportedOperationExceptio...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21174 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89906/ Test PASSed. ---

[GitHub] spark issue #21174: [SPARK-24085] Query returns UnsupportedOperationExceptio...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21174 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21174: [SPARK-24085] Query returns UnsupportedOperationExceptio...

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21174 **[Test build #89906 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89906/testReport)** for PR 21174 at commit [`38c7692`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #21068: [SPARK-16630][YARN] Blacklist a node if executors won't ...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21068 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89903/ Test PASSed. ---

[GitHub] spark issue #21068: [SPARK-16630][YARN] Blacklist a node if executors won't ...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21068 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21068: [SPARK-16630][YARN] Blacklist a node if executors won't ...

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21068 **[Test build #89903 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89903/testReport)** for PR 21068 at commit [`17bbbee`](https://github.com/apache/spark/commit/1

[GitHub] spark pull request #21068: [SPARK-16630][YARN] Blacklist a node if executors...

2018-04-26 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/21068#discussion_r184575258 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala --- @@ -170,8 +170,7 @@ class CoarseGrainedSchedulerBac

[GitHub] spark issue #18717: [SPARK-21510] [SQL] Add isMaterialized() and eager persi...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18717 Build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-

[GitHub] spark issue #18717: [SPARK-21510] [SQL] Add isMaterialized() and eager persi...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18717 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89907/ Test FAILed. ---

[GitHub] spark issue #18717: [SPARK-21510] [SQL] Add isMaterialized() and eager persi...

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18717 **[Test build #89907 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89907/testReport)** for PR 18717 at commit [`31bf797`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #21168: [SPARK-23830][CORE] added check to ensure main method is...

2018-04-26 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/21168 The change is fail to build, please fix it. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional c

[GitHub] spark issue #21173: [SPARK-23856][SQL] Add an option `queryTimeout` in JDBCO...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21173 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21173: [SPARK-23856][SQL] Add an option `queryTimeout` in JDBCO...

2018-04-26 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21173 @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spar

[GitHub] spark issue #21173: [SPARK-23856][SQL] Add an option `queryTimeout` in JDBCO...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21173 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89905/ Test PASSed. ---

[GitHub] spark issue #21173: [SPARK-23856][SQL] Add an option `queryTimeout` in JDBCO...

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21173 **[Test build #89905 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89905/testReport)** for PR 21173 at commit [`f134548`](https://github.com/apache/spark/commit/f

[GitHub] spark issue #21168: [SPARK-23830][CORE] added check to ensure main method is...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21168 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21168: [SPARK-23830][CORE] added check to ensure main method is...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21168 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89911/ Test FAILed. ---

[GitHub] spark issue #21168: [SPARK-23830][CORE] added check to ensure main method is...

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21168 **[Test build #89911 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89911/testReport)** for PR 21168 at commit [`3d90a5e`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #21168: [SPARK-23830][CORE] added check to ensure main method is...

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21168 **[Test build #89911 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89911/testReport)** for PR 21168 at commit [`3d90a5e`](https://github.com/apache/spark/commit/3d

[GitHub] spark pull request #21169: [SPARK-23715][SQL] the input of to/from_utc_times...

2018-04-26 Thread bersprockets
Github user bersprockets commented on a diff in the pull request: https://github.com/apache/spark/pull/21169#discussion_r184569738 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala --- @@ -782,6 +782,22 @@ object TypeCoercion {

[GitHub] spark issue #21168: added check to ensure main method is found [SPARK-23830]

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21168 **[Test build #89910 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89910/testReport)** for PR 21168 at commit [`571ae40`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #21168: added check to ensure main method is found [SPARK-23830]

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21168 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89910/ Test FAILed. ---

[GitHub] spark issue #21168: added check to ensure main method is found [SPARK-23830]

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21168 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21168: added check to ensure main method is found [SPARK-23830]

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21168 **[Test build #89910 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89910/testReport)** for PR 21168 at commit [`571ae40`](https://github.com/apache/spark/commit/57

[GitHub] spark issue #21157: [SPARK-22674][PYTHON] Removed the namedtuple pickling pa...

2018-04-26 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21157 Please go ahead if there's another approach to avoid to remove but fix it. --- - To unsubscribe, e-mail: reviews-unsubscr...@

[GitHub] spark issue #21168: added check to ensure main method is found [SPARK-23830]

2018-04-26 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21168 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@s

[GitHub] spark issue #21107: [SPARK-24044][PYTHON] Explicitly print out skipped tests...

2018-04-26 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21107 Thank you for reviewing this @bersprockets, @viirya, @BryanCutler, @icexelloss and @felixcheung. --- - To unsubscribe, e-mai

[GitHub] spark pull request #21169: [SPARK-23715][SQL] the input of to/from_utc_times...

2018-04-26 Thread bersprockets
Github user bersprockets commented on a diff in the pull request: https://github.com/apache/spark/pull/21169#discussion_r184565093 --- Diff: sql/core/src/test/resources/sql-tests/inputs/datetime.sql --- @@ -27,3 +27,8 @@ select current_date = current_date(), current_timestamp = cur

  1   2   3   4   5   >