[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-02 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21696 Don't block on me. Just wanted to doubly sure if this is the only way. I am fine. --- - To unsubscribe, e-mail:

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-07-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21596 **[Test build #92554 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92554/testReport)** for PR 21596 at commit

[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-02 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21696 Yea, I got that we it should have been done like this, and wonder if we can avoid this. It sounds more like a bandaid fix mainly cased by decimal. FWIW, in case of timestamp as INT96

[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-02 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21696 you can't get the physical schema information in a higher layer, as it may vary for different files. The table schema can evolve(add/drop column). ---

[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-02 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21696 physical schema information shouldn't usually referred in a higher layer though cc @liancheng. It's kind of something we should avoid .. I got that we need this but wonder if this is the only

[GitHub] spark pull request #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-02 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21696#discussion_r199686314 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFileFormat.scala --- @@ -379,14 +366,29 @@ class

[GitHub] spark issue #21469: [SPARK-24441][SS] Expose total estimated size of states ...

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21469 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21469: [SPARK-24441][SS] Expose total estimated size of states ...

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21469 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92551/ Test PASSed. ---

[GitHub] spark pull request #21668: [SPARK-24690][SQL] Add a new config to control pl...

2018-07-02 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21668#discussion_r199682495 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala --- @@ -375,16 +375,16 @@ case class CatalogStatistics(

[GitHub] spark issue #21469: [SPARK-24441][SS] Expose total estimated size of states ...

2018-07-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21469 **[Test build #92551 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92551/testReport)** for PR 21469 at commit

[GitHub] spark issue #21701: [SPARK-24730][SS] Add policy to choose max as global wat...

2018-07-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21701 **[Test build #92553 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92553/testReport)** for PR 21701 at commit

[GitHub] spark issue #21701: [SPARK-24730][SS] Add policy to choose max as global wat...

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21701 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/631/

[GitHub] spark issue #21701: [SPARK-24730][SS] Add policy to choose max as global wat...

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21701 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21701: [SPARK-24730][SS] Add policy to choose max as glo...

2018-07-02 Thread tdas
GitHub user tdas opened a pull request: https://github.com/apache/spark/pull/21701 [SPARK-24730][SS] Add policy to choose max as global watermark when streaming query has multiple watermarks ## What changes were proposed in this pull request? Currently, when a streaming

[GitHub] spark pull request #21605: [SPARK-24385][SQL] Resolve self-join condition am...

2018-07-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21605 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21605: [SPARK-24385][SQL] Resolve self-join condition ambiguity...

2018-07-02 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21605 thanks, merging to master! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21668: [SPARK-24690][SQL] Add a new config to control plan stat...

2018-07-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21668 yea, ok. I'll reconsider this again. Thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-02 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21696 makes sense to me, since we need the physical schema information to pushdown decimal and timestamp. also cc @rdblue @michal-databricks ---

[GitHub] spark pull request #21073: [SPARK-23936][SQL] Implement map_concat

2018-07-02 Thread bersprockets
Github user bersprockets commented on a diff in the pull request: https://github.com/apache/spark/pull/21073#discussion_r199678852 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala --- @@ -551,6 +551,36 @@ object TypeCoercion {

[GitHub] spark pull request #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-02 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21696#discussion_r199678338 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFileFormat.scala --- @@ -379,14 +366,29 @@ class

[GitHub] spark issue #21668: [SPARK-24690][SQL] Add a new config to control plan stat...

2018-07-02 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21668 yea this is a real problem, but I feel a better solution is to integrate the StarSchemaDetection into CBO. How hard will it be? ---

[GitHub] spark issue #21700: [SPARK-24717][SS] Split out min retain version of state ...

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21700 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21700: [SPARK-24717][SS] Split out min retain version of state ...

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21700 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92550/ Test PASSed. ---

[GitHub] spark issue #21700: [SPARK-24717][SS] Split out min retain version of state ...

2018-07-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21700 **[Test build #92550 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92550/testReport)** for PR 21700 at commit

[GitHub] spark pull request #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-02 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/21696#discussion_r199672805 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala --- @@ -19,166 +19,186 @@ package

[GitHub] spark issue #21690: [SPARK-24713]AppMatser of spark streaming kafka OOM if t...

2018-07-02 Thread yuanboliu
Github user yuanboliu commented on the issue: https://github.com/apache/spark/pull/21690 The first pause is used to stop poll() in the method paranoidPoll The second one is attached because of p.partition(). I'm not sure whether the state of pause will be rewritten after these

[GitHub] spark issue #21618: [SPARK-20408][SQL] Get the glob path in parallel to redu...

2018-07-02 Thread xuanyuanking
Github user xuanyuanking commented on the issue: https://github.com/apache/spark/pull/21618 gental ping @cloud-fan @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21633: [SPARK-24646][CORE] Minor change to spark.yarn.dist.forc...

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21633 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/630/

[GitHub] spark issue #21633: [SPARK-24646][CORE] Minor change to spark.yarn.dist.forc...

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21633 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21633: [SPARK-24646][CORE] Minor change to spark.yarn.dist.forc...

2018-07-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21633 **[Test build #92552 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92552/testReport)** for PR 21633 at commit

[GitHub] spark issue #21546: [WIP][SPARK-23030][SQL][PYTHON] Use Arrow stream format ...

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21546 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21546: [WIP][SPARK-23030][SQL][PYTHON] Use Arrow stream format ...

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21546 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92543/ Test PASSed. ---

[GitHub] spark issue #21546: [WIP][SPARK-23030][SQL][PYTHON] Use Arrow stream format ...

2018-07-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21546 **[Test build #92543 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92543/testReport)** for PR 21546 at commit

[GitHub] spark issue #21459: [SPARK-24420][Build] Upgrade ASM to 6.1 to support JDK9+

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21459 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92540/ Test PASSed. ---

[GitHub] spark issue #21459: [SPARK-24420][Build] Upgrade ASM to 6.1 to support JDK9+

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21459 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21459: [SPARK-24420][Build] Upgrade ASM to 6.1 to support JDK9+

2018-07-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21459 **[Test build #92540 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92540/testReport)** for PR 21459 at commit

[GitHub] spark issue #21601: [SPARK-24610] fix reading small files via wholeTextFiles

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21601 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92545/ Test FAILed. ---

[GitHub] spark issue #21601: [SPARK-24610] fix reading small files via wholeTextFiles

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21601 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21601: [SPARK-24610] fix reading small files via wholeTextFiles

2018-07-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21601 **[Test build #92545 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92545/testReport)** for PR 21601 at commit

[GitHub] spark issue #21469: [SPARK-24441][SS] Expose total estimated size of states ...

2018-07-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21469 **[Test build #92551 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92551/testReport)** for PR 21469 at commit

[GitHub] spark issue #21469: [SPARK-24441][SS] Expose total estimated size of states ...

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21469 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21469: [SPARK-24441][SS] Expose total estimated size of states ...

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21469 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92544/ Test PASSed. ---

[GitHub] spark issue #21469: [SPARK-24441][SS] Expose total estimated size of states ...

2018-07-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21469 **[Test build #92544 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92544/testReport)** for PR 21469 at commit

[GitHub] spark issue #21469: [SPARK-24441][SS] Expose total estimated size of states ...

2018-07-02 Thread HeartSaVioR
Github user HeartSaVioR commented on the issue: https://github.com/apache/spark/pull/21469 retest this, please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21469: [SPARK-24441][SS] Expose total estimated size of states ...

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21469 Build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #21469: [SPARK-24441][SS] Expose total estimated size of states ...

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21469 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92542/ Test FAILed. ---

[GitHub] spark issue #21469: [SPARK-24441][SS] Expose total estimated size of states ...

2018-07-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21469 **[Test build #92542 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92542/testReport)** for PR 21469 at commit

[GitHub] spark issue #21692: [SPARK-24715][Build] Override jline version as 2.14.3 in...

2018-07-02 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/21692 Thanks @srowen for explanation. As I can tell, `-verbose:class` shows jline classes come from `jline-2.12.jar`, though `sbt dependency-tree` shows `jline:jline:0.9.94`. After this overriding, jline

[GitHub] spark issue #21686: [SPARK-24709][SQL] schema_of_json() - schema inference f...

2018-07-02 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/21686 Thanks. Awesome. This matches what I had in mind then. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #21699: [SPARK-24722][SQL] pivot() with Column type argument

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21699 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92541/ Test PASSed. ---

[GitHub] spark issue #21699: [SPARK-24722][SQL] pivot() with Column type argument

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21699 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21699: [SPARK-24722][SQL] pivot() with Column type argument

2018-07-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21699 **[Test build #92541 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92541/testReport)** for PR 21699 at commit

[GitHub] spark issue #21692: [SPARK-24715][Build] Override jline version as 2.14.3 in...

2018-07-02 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/21692 Thank you all! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21596 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92537/ Test PASSed. ---

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21596 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-07-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21596 **[Test build #92537 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92537/testReport)** for PR 21596 at commit

[GitHub] spark issue #21459: [SPARK-24420][Build] Upgrade ASM to 6.1 to support JDK9+

2018-07-02 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/21459 SGTM. On Mon, Jul 2, 2018 at 4:38 PM DB Tsai wrote: > There are three approvals from the committers, and the changes are pretty > trivial to revert if we see any performance

[GitHub] spark issue #21662: [SPARK-24662][SQL][SS] Support limit in structured strea...

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21662 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21662: [SPARK-24662][SQL][SS] Support limit in structured strea...

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21662 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92536/ Test PASSed. ---

[GitHub] spark issue #21662: [SPARK-24662][SQL][SS] Support limit in structured strea...

2018-07-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21662 **[Test build #92536 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92536/testReport)** for PR 21662 at commit

[GitHub] spark issue #21700: SPARK-24717 Split out min retain version of state for me...

2018-07-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21700 **[Test build #92550 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92550/testReport)** for PR 21700 at commit

[GitHub] spark issue #21700: SPARK-24717 Split out min retain version of state for me...

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21700 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21700: SPARK-24717 Split out min retain version of state for me...

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21700 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92549/ Test FAILed. ---

[GitHub] spark issue #21700: SPARK-24717 Split out min retain version of state for me...

2018-07-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21700 **[Test build #92549 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92549/testReport)** for PR 21700 at commit

[GitHub] spark issue #21700: SPARK-24717 Split out min retain version of state for me...

2018-07-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21700 **[Test build #92549 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92549/testReport)** for PR 21700 at commit

[GitHub] spark issue #21459: [SPARK-24420][Build] Upgrade ASM to 6.1 to support JDK9+

2018-07-02 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/21459 There are three approvals from the committers, and the changes are pretty trivial to revert if we see any performance regression which is unlikely. To move thing forward, if there is no further

[GitHub] spark issue #21700: SPARK-24717 Split out min retain version of state for me...

2018-07-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21700 **[Test build #92548 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92548/testReport)** for PR 21700 at commit

[GitHub] spark pull request #21320: [SPARK-4502][SQL] Parquet nested column pruning -...

2018-07-02 Thread mallman
Github user mallman commented on a diff in the pull request: https://github.com/apache/spark/pull/21320#discussion_r199648692 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetRowConverter.scala --- @@ -182,18 +182,20 @@ private[parquet]

[GitHub] spark issue #21700: SPARK-24717 Split out min retain version of state for me...

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21700 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92548/ Test FAILed. ---

[GitHub] spark issue #21700: SPARK-24717 Split out min retain version of state for me...

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21700 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21700: SPARK-24717 Split out min retain version of state for me...

2018-07-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21700 **[Test build #92548 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92548/testReport)** for PR 21700 at commit

[GitHub] spark issue #21700: SPARK-24717 Split out min retain version of state for me...

2018-07-02 Thread HeartSaVioR
Github user HeartSaVioR commented on the issue: https://github.com/apache/spark/pull/21700 Missing new line in EOF for two new Java files. Just addressed. Jenkins, retest this please. --- - To unsubscribe,

[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-02 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/21696 cc @gatorsmile @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21459: [SPARK-24420][Build] Upgrade ASM to 6.1 to support JDK9+

2018-07-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21459 **[Test build #4202 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4202/testReport)** for PR 21459 at commit

[GitHub] spark issue #21700: SPARK-24717 Split out min retain version of state for me...

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21700 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21700: SPARK-24717 Split out min retain version of state for me...

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21700 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92547/ Test FAILed. ---

[GitHub] spark issue #21700: SPARK-24717 Split out min retain version of state for me...

2018-07-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21700 **[Test build #92547 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92547/testReport)** for PR 21700 at commit

[GitHub] spark pull request #21320: [SPARK-4502][SQL] Parquet nested column pruning -...

2018-07-02 Thread mallman
Github user mallman commented on a diff in the pull request: https://github.com/apache/spark/pull/21320#discussion_r199643803 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetReadSupport.scala --- @@ -71,9 +80,22 @@ private[parquet]

[GitHub] spark issue #21700: SPARK-24717 Split out min retain version of state for me...

2018-07-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21700 **[Test build #92547 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92547/testReport)** for PR 21700 at commit

[GitHub] spark issue #21700: SPARK-24717 Split out min retain version of state for me...

2018-07-02 Thread HeartSaVioR
Github user HeartSaVioR commented on the issue: https://github.com/apache/spark/pull/21700 cc. @tdas @zsxwing @jose-torres @jerryshao @arunmahadevan @HyukjinKwon --- - To unsubscribe, e-mail:

[GitHub] spark issue #21700: SPARK-24717 Split out min retain version of state for me...

2018-07-02 Thread HeartSaVioR
Github user HeartSaVioR commented on the issue: https://github.com/apache/spark/pull/21700 retest this, please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21700: SPARK-24717 Split out min retain version of state for me...

2018-07-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21700 **[Test build #92546 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92546/testReport)** for PR 21700 at commit

[GitHub] spark issue #21700: SPARK-24717 Split out min retain version of state for me...

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21700 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92546/ Test FAILed. ---

[GitHub] spark issue #21700: SPARK-24717 Split out min retain version of state for me...

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21700 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21700: SPARK-24717 Split out min retain version of state for me...

2018-07-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21700 **[Test build #92546 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92546/testReport)** for PR 21700 at commit

[GitHub] spark issue #21700: SPARK-24717 Split out min retain version of state for me...

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21700 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21700: SPARK-24717 Split out min retain version of state for me...

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21700 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21700: SPARK-24717 Split out min retain version of state for me...

2018-07-02 Thread HeartSaVioR
Github user HeartSaVioR commented on the issue: https://github.com/apache/spark/pull/21700 Pasting JIRA issue description to explain why this patch is needed: As default version of "spark.sql.streaming.minBatchesToRetain" is set to high (100), which doesn't require strictly

[GitHub] spark pull request #21700: SPARK-24717 Split out min retain version of state...

2018-07-02 Thread HeartSaVioR
GitHub user HeartSaVioR opened a pull request: https://github.com/apache/spark/pull/21700 SPARK-24717 Split out min retain version of state for memory in HDFSBackedStateStoreProvider ## What changes were proposed in this pull request? This patch proposes breaking down

[GitHub] spark issue #21692: [SPARK-24715][Build] Override jline version as 2.14.3 in...

2018-07-02 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/21692 I haven't looked into this particular issue thoroughly, but, I'm aware that SBT and Maven don't actually resolve dependencies in quite the same way. I think the resolve conflicts with different

[GitHub] spark issue #21687: [SPARK-24165][SQL] Fixing the output data type of CaseWh...

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21687 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92535/ Test PASSed. ---

[GitHub] spark issue #21687: [SPARK-24165][SQL] Fixing the output data type of CaseWh...

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21687 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21687: [SPARK-24165][SQL] Fixing the output data type of CaseWh...

2018-07-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21687 **[Test build #92535 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92535/testReport)** for PR 21687 at commit

[GitHub] spark issue #21605: [SPARK-24385][SQL] Resolve self-join condition ambiguity...

2018-07-02 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/21605 kindly ping @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21599: [SPARK-24598][SQL] Overflow on arithmetic operati...

2018-07-02 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/21599#discussion_r199632638 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/arithmetic.scala --- @@ -128,17 +128,31 @@ abstract class

[GitHub] spark pull request #21320: [SPARK-4502][SQL] Parquet nested column pruning -...

2018-07-02 Thread mallman
Github user mallman commented on a diff in the pull request: https://github.com/apache/spark/pull/21320#discussion_r199631341 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetReadSupport.scala --- @@ -47,16 +47,25 @@ import

[GitHub] spark issue #21601: [SPARK-24610] fix reading small files via wholeTextFiles

2018-07-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21601 **[Test build #92545 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92545/testReport)** for PR 21601 at commit

[GitHub] spark issue #21601: [SPARK-24610] fix reading small files via wholeTextFiles

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21601 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21601: [SPARK-24610] fix reading small files via wholeTextFiles

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21601 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/629/

[GitHub] spark issue #21469: [SPARK-24441][SS] Expose total estimated size of states ...

2018-07-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21469 **[Test build #92544 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92544/testReport)** for PR 21469 at commit

  1   2   3   4   5   >