[GitHub] spark pull request #20286: [SPARK-23119][SS] Minor fixes to V2 streaming API...

2018-01-16 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/20286#discussion_r161976455 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/streaming/reader/Offset.java --- @@ -17,12 +17,20 @@ package org.apache.spark.s

[GitHub] spark pull request #20286: [SPARK-23119][SS] Minor fixes to V2 streaming API...

2018-01-16 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/20286#discussion_r161976313 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/streaming/reader/MicroBatchReader.java --- @@ -25,7 +26,11 @@ /** * A mix-in in

[GitHub] spark pull request #20286: [SPARK-23119][SS] Minor fixes to V2 streaming API...

2018-01-16 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/20286#discussion_r161976371 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/streaming/reader/ContinuousReader.java --- @@ -27,11 +28,15 @@ * interface to allow

[GitHub] spark issue #20243: [SPARK-23052][SS] Migrate ConsoleSink to data source V2 ...

2018-01-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20243 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20243: [SPARK-23052][SS] Migrate ConsoleSink to data source V2 ...

2018-01-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20243 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86235/ Test PASSed. ---

[GitHub] spark issue #20243: [SPARK-23052][SS] Migrate ConsoleSink to data source V2 ...

2018-01-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20243 **[Test build #86235 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86235/testReport)** for PR 20243 at commit [`516fd4a`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #20243: [SPARK-23052][SS] Migrate ConsoleSink to data source V2 ...

2018-01-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20243 Build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-

[GitHub] spark issue #20243: [SPARK-23052][SS] Migrate ConsoleSink to data source V2 ...

2018-01-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20243 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86232/ Test PASSed. ---

[GitHub] spark issue #20243: [SPARK-23052][SS] Migrate ConsoleSink to data source V2 ...

2018-01-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20243 **[Test build #86232 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86232/testReport)** for PR 20243 at commit [`c0ec93f`](https://github.com/apache/spark/commit/c

[GitHub] spark pull request #20265: [SPARK-21783][SQL] Turn on ORC filter push-down b...

2018-01-16 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20265#discussion_r161973792 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/FilterPushdownBenchmark.scala --- @@ -0,0 +1,230 @@ +/* + * Licensed to the Apache Soft

[GitHub] spark pull request #20265: [SPARK-21783][SQL] Turn on ORC filter push-down b...

2018-01-16 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20265#discussion_r161973650 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/FilterPushdownBenchmark.scala --- @@ -0,0 +1,230 @@ +/* + * Licensed to the Apache Soft

[GitHub] spark issue #20266: [SPARK-23072][SQL][TEST] Add a Unicode schema test for f...

2018-01-16 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20266 Thank you, @cloud-fan , @gatorsmile , and @mgaido91 ! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org Fo

[GitHub] spark issue #20025: [SPARK-22837][SQL]Session timeout checker does not work ...

2018-01-16 Thread zuotingbing
Github user zuotingbing commented on the issue: https://github.com/apache/spark/pull/20025 @cloud-fan @gatorsmile @liufengdb @felixcheung @srowen @vanzin Is anybody could make further contact and discuss this PR? Thanks! --- -

[GitHub] spark issue #17298: [SPARK-19094][WIP][PySpark] Plumb through logging for IJ...

2018-01-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17298 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86233/ Test FAILed. ---

[GitHub] spark issue #17298: [SPARK-19094][WIP][PySpark] Plumb through logging for IJ...

2018-01-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17298 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #17298: [SPARK-19094][WIP][PySpark] Plumb through logging for IJ...

2018-01-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17298 **[Test build #86233 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86233/testReport)** for PR 17298 at commit [`ae450e8`](https://github.com/apache/spark/commit/a

[GitHub] spark pull request #20288: [SPARK-23122][PYTHON][SQL] Deprecate register* fo...

2018-01-16 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/20288#discussion_r161969687 --- Diff: python/pyspark/sql/context.py --- @@ -147,7 +147,8 @@ def udf(self): :return: :class:`UDFRegistration` """ -

[GitHub] spark issue #20225: [SPARK-23033] Don't use task level retry for continuous ...

2018-01-16 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/20225 Fix merge conflicts. And add [SS} to the title of this PR. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For ad

[GitHub] spark pull request #20243: [SPARK-23052][SS] Migrate ConsoleSink to data sou...

2018-01-16 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/20243#discussion_r161970902 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/sources/StreamingDataSourceV2Suite.scala --- @@ -0,0 +1,249 @@ +/* + * Licensed to th

[GitHub] spark pull request #20243: [SPARK-23052][SS] Migrate ConsoleSink to data sou...

2018-01-16 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/20243#discussion_r161970553 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/sources/StreamingDataSourceV2Suite.scala --- @@ -0,0 +1,249 @@ +/* + * Licensed to th

[GitHub] spark pull request #20288: [SPARK-23122][PYTHON][SQL] Deprecate register* fo...

2018-01-16 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20288#discussion_r161966469 --- Diff: python/pyspark/sql/session.py --- @@ -778,6 +778,146 @@ def __exit__(self, exc_type, exc_val, exc_tb): self.stop() +cla

[GitHub] spark pull request #20288: [SPARK-23122][PYTHON][SQL] Deprecate register* fo...

2018-01-16 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20288#discussion_r161964247 --- Diff: python/pyspark/sql/catalog.py --- @@ -224,92 +225,18 @@ def dropGlobalTempView(self, viewName): """ self._jcatalog.dropGlo

[GitHub] spark pull request #20288: [SPARK-23122][PYTHON][SQL] Deprecate register* fo...

2018-01-16 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20288#discussion_r161964383 --- Diff: python/pyspark/sql/context.py --- @@ -147,7 +147,8 @@ def udf(self): :return: :class:`UDFRegistration` """ -

[GitHub] spark pull request #20288: [SPARK-23122][PYTHON][SQL] Deprecate register* fo...

2018-01-16 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20288#discussion_r161966507 --- Diff: python/pyspark/sql/session.py --- @@ -778,6 +778,146 @@ def __exit__(self, exc_type, exc_val, exc_tb): self.stop() +cla

[GitHub] spark pull request #20243: [SPARK-23052][SS] Migrate ConsoleSink to data sou...

2018-01-16 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/20243#discussion_r161969954 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/streaming/DataStreamWriter.scala --- @@ -280,14 +280,12 @@ final class DataStreamWriter[T] private[sql]

[GitHub] spark issue #20282: [SPARK-23093][SS] Don't change run id when reconfiguring...

2018-01-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20282 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86236/ Test FAILed. ---

[GitHub] spark issue #20282: [SPARK-23093][SS] Don't change run id when reconfiguring...

2018-01-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20282 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20282: [SPARK-23093][SS] Don't change run id when reconfiguring...

2018-01-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20282 **[Test build #86236 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86236/testReport)** for PR 20282 at commit [`70917a5`](https://github.com/apache/spark/commit/7

[GitHub] spark pull request #20243: [SPARK-23052][SS] Migrate ConsoleSink to data sou...

2018-01-16 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/20243#discussion_r161969566 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/streaming/DataStreamWriter.scala --- @@ -280,14 +280,12 @@ final class DataStreamWriter[T] private[sql]

[GitHub] spark pull request #20243: [SPARK-23052][SS] Migrate ConsoleSink to data sou...

2018-01-16 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/20243#discussion_r161968219 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/sources/PackedRowWriterFactory.scala --- @@ -0,0 +1,50 @@ +/* + * Licensed

[GitHub] spark pull request #20243: [SPARK-23052][SS] Migrate ConsoleSink to data sou...

2018-01-16 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/20243#discussion_r161967927 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/sources/PackedRowWriterFactory.scala --- @@ -0,0 +1,50 @@ +/* + * Licensed

[GitHub] spark pull request #20243: [SPARK-23052][SS] Migrate ConsoleSink to data sou...

2018-01-16 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/20243#discussion_r161967938 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/sources/PackedRowWriterFactory.scala --- @@ -0,0 +1,50 @@ +/* + * Licensed

[GitHub] spark pull request #20243: [SPARK-23052][SS] Migrate ConsoleSink to data sou...

2018-01-16 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/20243#discussion_r161967603 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/sources/ConsoleWriter.scala --- @@ -0,0 +1,57 @@ +/* + * Licensed to the Ap

[GitHub] spark issue #20280: [SPARK-22232][PYTHON][SQL] Fixed Row pickling to include...

2018-01-16 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/20280 Thanks @HyukjinKwon and @MrBago for reviewing. After thinking about this some more, I don't think this is the right solution. Like @HyukjinKwon pointed out, the supplied schema names should al

[GitHub] spark pull request #20243: [SPARK-23052][SS] Migrate ConsoleSink to data sou...

2018-01-16 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/20243#discussion_r161966917 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/sources/ConsoleWriter.scala --- @@ -0,0 +1,57 @@ +/* + * Licensed to the Ap

[GitHub] spark pull request #20243: [SPARK-23052][SS] Migrate ConsoleSink to data sou...

2018-01-16 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/20243#discussion_r161966755 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousExecution.scala --- @@ -54,7 +54,7 @@ class ContinuousExecutio

[GitHub] spark pull request #20243: [SPARK-23052][SS] Migrate ConsoleSink to data sou...

2018-01-16 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/20243#discussion_r161966696 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousExecution.scala --- @@ -69,7 +69,7 @@ class ContinuousExecutio

[GitHub] spark issue #20282: [SPARK-23093][SS] Don't change run id when reconfiguring...

2018-01-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20282 **[Test build #86250 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86250/testReport)** for PR 20282 at commit [`70917a5`](https://github.com/apache/spark/commit/70

[GitHub] spark issue #20286: [SPARK-23119][SS] Minor fixes to V2 streaming APIs

2018-01-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20286 **[Test build #86249 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86249/testReport)** for PR 20286 at commit [`f6dfa58`](https://github.com/apache/spark/commit/f6

[GitHub] spark pull request #20266: [SPARK-23072][SQL][TEST] Add a Unicode schema tes...

2018-01-16 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20266 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20286: [SPARK-23119][SS] Minor fixes to V2 streaming APIs

2018-01-16 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/20286 jenkins retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: revi

[GitHub] spark issue #20282: [SPARK-23093][SS] Don't change run id when reconfiguring...

2018-01-16 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/20282 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h.

[GitHub] spark issue #20266: [SPARK-23072][SQL][TEST] Add a Unicode schema test for f...

2018-01-16 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20266 thanks, merging to master/2.3! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-ma

[GitHub] spark pull request #20289: Fix merge between 07ae39d0ec and 1667057851

2018-01-16 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20289 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20288: [SPARK-23122][PYTHON][SQL] Deprecate register* fo...

2018-01-16 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20288#discussion_r161965369 --- Diff: python/pyspark/sql/session.py --- @@ -778,6 +778,146 @@ def __exit__(self, exc_type, exc_val, exc_tb): self.stop() +

[GitHub] spark issue #20289: Fix merge between 07ae39d0ec and 1667057851

2018-01-16 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/20289 LGTM. Merging to master and 2.3. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-ma

[GitHub] spark pull request #20288: [SPARK-23122][PYTHON][SQL] Deprecate register* fo...

2018-01-16 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20288#discussion_r161965278 --- Diff: python/pyspark/sql/context.py --- @@ -624,6 +536,9 @@ def _test(): globs['os'] = os globs['sc'] = sc globs['sqlContext

[GitHub] spark issue #20266: [SPARK-23072][SQL][TEST] Add a Unicode schema test for f...

2018-01-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20266 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86227/ Test PASSed. ---

[GitHub] spark issue #20266: [SPARK-23072][SQL][TEST] Add a Unicode schema test for f...

2018-01-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20266 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20266: [SPARK-23072][SQL][TEST] Add a Unicode schema test for f...

2018-01-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20266 **[Test build #86227 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86227/testReport)** for PR 20266 at commit [`c67809c`](https://github.com/apache/spark/commit/c

[GitHub] spark pull request #20280: [SPARK-22232][PYTHON][SQL] Fixed Row pickling to ...

2018-01-16 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/20280#discussion_r161964785 --- Diff: python/pyspark/sql/tests.py --- @@ -2306,18 +2306,20 @@ def test_toDF_with_schema_string(self): self.assertEqual(df.schema.simpleS

[GitHub] spark pull request #20288: [SPARK-23122][PYTHON][SQL] Deprecate register* fo...

2018-01-16 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20288#discussion_r161964738 --- Diff: python/pyspark/sql/session.py --- @@ -778,6 +778,146 @@ def __exit__(self, exc_type, exc_val, exc_tb): self.stop() +

[GitHub] spark issue #20288: [SPARK-23122][PYTHON][SQL] Deprecate register* for UDFs ...

2018-01-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20288 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86247/ Test PASSed. ---

[GitHub] spark issue #20288: [SPARK-23122][PYTHON][SQL] Deprecate register* for UDFs ...

2018-01-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20288 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20288: [SPARK-23122][PYTHON][SQL] Deprecate register* for UDFs ...

2018-01-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20288 **[Test build #86247 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86247/testReport)** for PR 20288 at commit [`08438ee`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #13143: [SPARK-15359] [Mesos] Mesos dispatcher should handle DRI...

2018-01-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13143 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20265: [SPARK-21783][SQL] Turn on ORC filter push-down b...

2018-01-16 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20265#discussion_r161964178 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/FilterPushdownBenchmark.scala --- @@ -0,0 +1,230 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #20265: [SPARK-21783][SQL] Turn on ORC filter push-down b...

2018-01-16 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20265#discussion_r161963925 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/FilterPushdownBenchmark.scala --- @@ -0,0 +1,230 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #20290: Testing ssuchter prb branch8

2018-01-16 Thread ssuchter
Github user ssuchter closed the pull request at: https://github.com/apache/spark/pull/20290 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20265: [SPARK-21783][SQL] Turn on ORC filter push-down by defau...

2018-01-16 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20265 LGTM except one comment. Let's worry about row group/stripe size later, since both parquet and orc use default settings, I think it's still fair. ---

[GitHub] spark issue #20287: [SPARK-23121][WEB-UI] When the Spark Streaming app is ru...

2018-01-16 Thread guoxiaolongzte
Github user guoxiaolongzte commented on the issue: https://github.com/apache/spark/pull/20287 @smurakozi Help review the code, this bug results from your added functionality. --- - To unsubscribe, e-mail: revie

[GitHub] spark issue #20290: Testing ssuchter prb branch8

2018-01-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20290 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20290: Testing ssuchter prb branch8

2018-01-16 Thread ssuchter
GitHub user ssuchter opened a pull request: https://github.com/apache/spark/pull/20290 Testing ssuchter prb branch8 ## What changes were proposed in this pull request? (Please fill in changes proposed in this fix) ## How was this patch tested? (Please expla

[GitHub] spark pull request #20265: [SPARK-21783][SQL] Turn on ORC filter push-down b...

2018-01-16 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20265#discussion_r161963288 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/FilterPushdownBenchmark.scala --- @@ -0,0 +1,230 @@ +/* + * Licensed to the Apache Software

[GitHub] spark issue #17280: [SPARK-19939] [ML] Add support for association rules in ...

2018-01-16 Thread hhbyyh
Github user hhbyyh commented on the issue: https://github.com/apache/spark/pull/17280 Thanks for taking a look @MLnick --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-ma

[GitHub] spark pull request #17280: [SPARK-19939] [ML] Add support for association ru...

2018-01-16 Thread hhbyyh
Github user hhbyyh commented on a diff in the pull request: https://github.com/apache/spark/pull/17280#discussion_r161962624 --- Diff: mllib/src/main/scala/org/apache/spark/ml/fpm/FPGrowth.scala --- @@ -319,9 +323,11 @@ object FPGrowthModel extends MLReadable[FPGrowthModel] {

[GitHub] spark issue #20201: [SPARK-22389][SQL] data source v2 partitioning reporting...

2018-01-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20201 **[Test build #86248 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86248/testReport)** for PR 20201 at commit [`713140a`](https://github.com/apache/spark/commit/71

[GitHub] spark issue #20201: [SPARK-22389][SQL] data source v2 partitioning reporting...

2018-01-16 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20201 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-

[GitHub] spark issue #20287: [SPARK-23121][WEB-UI] When the Spark Streaming app is ru...

2018-01-16 Thread guoxiaolongzte
Github user guoxiaolongzte commented on the issue: https://github.com/apache/spark/pull/20287 Well, then you can tell me how specific changes? I do not have a good idea right now. The problem is that the page crashes, it should be a fatal bug. --- ---

[GitHub] spark issue #20288: [SPARK-23122][PYTHON][SQL] Deprecate register* for UDFs ...

2018-01-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20288 **[Test build #86247 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86247/testReport)** for PR 20288 at commit [`08438ee`](https://github.com/apache/spark/commit/08

[GitHub] spark issue #20223: [SPARK-23020][core] Fix races in launcher code, test.

2018-01-16 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20223 We didn't run the yarn test with this PR due to https://issues.apache.org/jira/browse/SPARK-10300 This indicates that it might be a bad idea to skip the yarn test if we don't change yarn

[GitHub] spark issue #20216: [SPARK-23024][WEB-UI]Spark ui about the contents of the ...

2018-01-16 Thread guoxiaolongzte
Github user guoxiaolongzte commented on the issue: https://github.com/apache/spark/pull/20216 test this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: revie

[GitHub] spark pull request #20259: [SPARK-23066][WEB-UI] Master Page increase master...

2018-01-16 Thread guoxiaolongzte
Github user guoxiaolongzte commented on a diff in the pull request: https://github.com/apache/spark/pull/20259#discussion_r161959792 --- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala --- @@ -179,6 +181,7 @@ private[deploy] class Master( } p

[GitHub] spark issue #20281: [SPARK-23089][STS] Recreate session log directory if it ...

2018-01-16 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/20281 LGTM. So looks like the fix is exactly the same as Hive. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #20277: [SPARK-23090][SQL] polish ColumnVector

2018-01-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20277 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20277: [SPARK-23090][SQL] polish ColumnVector

2018-01-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20277 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86222/ Test PASSed. ---

[GitHub] spark issue #20277: [SPARK-23090][SQL] polish ColumnVector

2018-01-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20277 **[Test build #86222 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86222/testReport)** for PR 20277 at commit [`3cba91b`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #20184: [SPARK-22987][Core] UnsafeExternalSorter cases OOM when ...

2018-01-16 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/20184 >I think that a lazy buffer allocation can not thoroughly solve this problem because UnsafeSorterSpillReader has BufferedFileInputStream witch will allocate off heap memory. Can you pleas

[GitHub] spark issue #20257: [SPARK-23048][ML] Add OneHotEncoderEstimator document an...

2018-01-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20257 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20257: [SPARK-23048][ML] Add OneHotEncoderEstimator document an...

2018-01-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20257 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86244/ Test PASSed. ---

[GitHub] spark issue #20257: [SPARK-23048][ML] Add OneHotEncoderEstimator document an...

2018-01-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20257 **[Test build #86244 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86244/testReport)** for PR 20257 at commit [`e57d9ee`](https://github.com/apache/spark/commit/e

[GitHub] spark issue #20289: Fix merge between 07ae39d0ec and 1667057851

2018-01-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20289 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86239/ Test PASSed. ---

[GitHub] spark issue #20285: [SPARK-22735][ML][DOC] Added VectorSizeHint docs and exa...

2018-01-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20285 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20285: [SPARK-22735][ML][DOC] Added VectorSizeHint docs and exa...

2018-01-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20285 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86221/ Test PASSed. ---

[GitHub] spark issue #20289: Fix merge between 07ae39d0ec and 1667057851

2018-01-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20289 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20289: Fix merge between 07ae39d0ec and 1667057851

2018-01-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20289 **[Test build #86239 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86239/testReport)** for PR 20289 at commit [`baee3c2`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #20285: [SPARK-22735][ML][DOC] Added VectorSizeHint docs and exa...

2018-01-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20285 **[Test build #86221 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86221/testReport)** for PR 20285 at commit [`85d0db0`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #20257: [SPARK-23048][ML] Add OneHotEncoderEstimator document an...

2018-01-16 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/20257 @MLnick @WeichenXu123 Your comments are addressed. Please check this again. Thanks. --- - To unsubscribe, e-mail: reviews-unsubsc

[GitHub] spark issue #20277: [SPARK-23090][SQL] polish ColumnVector

2018-01-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20277 **[Test build #86246 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86246/testReport)** for PR 20277 at commit [`f3f9d5e`](https://github.com/apache/spark/commit/f3

[GitHub] spark issue #20277: [SPARK-23090][SQL] polish ColumnVector

2018-01-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20277 **[Test build #86245 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86245/testReport)** for PR 20277 at commit [`77e8c4b`](https://github.com/apache/spark/commit/77

[GitHub] spark pull request #20277: [SPARK-23090][SQL] polish ColumnVector

2018-01-16 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20277#discussion_r161957896 --- Diff: sql/core/src/main/java/org/apache/spark/sql/vectorized/ArrowColumnVector.java --- @@ -33,18 +33,6 @@ private final ArrowVectorAccessor a

[GitHub] spark issue #20257: [SPARK-23048][ML] Add OneHotEncoderEstimator document an...

2018-01-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20257 **[Test build #86244 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86244/testReport)** for PR 20257 at commit [`e57d9ee`](https://github.com/apache/spark/commit/e5

[GitHub] spark issue #20277: [SPARK-23090][SQL] polish ColumnVector

2018-01-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20277 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86240/ Test FAILed. ---

[GitHub] spark issue #20277: [SPARK-23090][SQL] polish ColumnVector

2018-01-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20277 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20277: [SPARK-23090][SQL] polish ColumnVector

2018-01-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20277 **[Test build #86240 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86240/testReport)** for PR 20277 at commit [`08d06a7`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #19247: [Spark-21996][SQL] read files with space in name for str...

2018-01-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19247 **[Test build #86243 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86243/testReport)** for PR 19247 at commit [`10106b3`](https://github.com/apache/spark/commit/10

[GitHub] spark issue #20277: [SPARK-23090][SQL] polish ColumnVector

2018-01-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20277 **[Test build #86242 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86242/testReport)** for PR 20277 at commit [`bc6d0af`](https://github.com/apache/spark/commit/bc

[GitHub] spark issue #20277: [SPARK-23090][SQL] polish ColumnVector

2018-01-16 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20277 @hvanhovell good idea. I ran the `ColumnarBatchBenchmark` and found `getArray` has a regression as it's not final anymore. I've reverted the interface stuff and `ColumnVector` is still abstract cl

[GitHub] spark issue #19247: [Spark-21996][SQL] read files with space in name for str...

2018-01-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19247 **[Test build #86241 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86241/testReport)** for PR 19247 at commit [`04c2b14`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #19247: [Spark-21996][SQL] read files with space in name for str...

2018-01-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19247 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86241/ Test FAILed. ---

  1   2   3   4   5   6   7   >