[GitHub] spark pull request #20437: [SPARK-23270][Streaming][WEB-UI]FileInputDStream ...

2018-01-30 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/20437#discussion_r164973745 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/dstream/FileInputDStream.scala --- @@ -157,7 +157,7 @@ class FileInputDStream[K, V, F <:

[GitHub] spark pull request #20400: [SPARK-23084][PYTHON]Add unboundedPreceding(), un...

2018-01-30 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20400#discussion_r164973111 --- Diff: python/pyspark/sql/window.py --- @@ -212,16 +218,20 @@ def rangeBetween(self, start, end): values directly.

[GitHub] spark pull request #20437: [SPARK-23270][Streaming][WEB-UI]FileInputDStream ...

2018-01-30 Thread guoxiaolongzte
Github user guoxiaolongzte commented on a diff in the pull request: https://github.com/apache/spark/pull/20437#discussion_r164973156 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/dstream/FileInputDStream.scala --- @@ -157,7 +157,7 @@ class FileInputDStream[K, V, F

[GitHub] spark issue #20450: [SPARK-23280][SQL] add map type support to ColumnVector

2018-01-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20450 **[Test build #86866 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86866/testReport)** for PR 20450 at commit

[GitHub] spark issue #20450: [SPARK-23280][SQL] add map type support to ColumnVector

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20450 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/421/

[GitHub] spark issue #20450: [SPARK-23280][SQL] add map type support to ColumnVector

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20450 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20450: [SPARK-23280][SQL] add map type support to ColumnVector

2018-01-30 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20450 cc @hvanhovell @viirya @ueshin @kiszk @gatorsmile @dongjoon-hyun --- - To unsubscribe, e-mail:

[GitHub] spark pull request #20450: [SPARK-23280][SQL] add map type support to Column...

2018-01-30 Thread cloud-fan
GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/20450 [SPARK-23280][SQL] add map type support to ColumnVector ## What changes were proposed in this pull request? Fill the last missing piece of `ColumnVector`: the map type support.

[GitHub] spark pull request #20400: [SPARK-23084][PYTHON]Add unboundedPreceding(), un...

2018-01-30 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20400#discussion_r164972094 --- Diff: python/pyspark/sql/window.py --- @@ -212,16 +218,20 @@ def rangeBetween(self, start, end): values directly.

[GitHub] spark pull request #20400: [SPARK-23084][PYTHON]Add unboundedPreceding(), un...

2018-01-30 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20400#discussion_r164971807 --- Diff: python/pyspark/sql/window.py --- @@ -124,16 +126,20 @@ def rangeBetween(start, end): values directly. :param

[GitHub] spark pull request #20438: [SPARK-23272][SQL] add calendar interval type sup...

2018-01-30 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20438#discussion_r164971509 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/MutableColumnarRow.java --- @@ -146,9 +146,7 @@ public UTF8String

[GitHub] spark issue #17886: [SPARK-13983][SQL] Fix HiveThriftServer2 can not get "--...

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17886 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #17886: [SPARK-13983][SQL] Fix HiveThriftServer2 can not get "--...

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17886 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/420/

[GitHub] spark pull request #20438: [SPARK-23272][SQL] add calendar interval type sup...

2018-01-30 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20438 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20400: [SPARK-23084][PYTHON]Add unboundedPreceding(), un...

2018-01-30 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20400#discussion_r164968965 --- Diff: python/pyspark/sql/functions.py --- @@ -809,6 +809,45 @@ def ntile(n): return Column(sc._jvm.functions.ntile(int(n)))

[GitHub] spark issue #20438: [SPARK-23272][SQL] add calendar interval type support to...

2018-01-30 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20438 thanks, merging to master/2.3! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #20437: [SPARK-23270][Streaming][WEB-UI]FileInputDStream ...

2018-01-30 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/20437#discussion_r164968292 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/dstream/FileInputDStream.scala --- @@ -157,7 +157,7 @@ class FileInputDStream[K, V, F <:

[GitHub] spark pull request #20448: [SPARK-23203][SQL] make DataSourceV2Relation immu...

2018-01-30 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20448#discussion_r164966631 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala --- @@ -17,16 +17,57 @@ package

[GitHub] spark issue #20373: [SPARK-23159][PYTHON] Update cloudpickle to v0.4.2 plus ...

2018-01-30 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20373 Ah, yup. I am okay with including it. Wanted to double check. Seems a small clean bug fix. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #20400: [SPARK-23084][PYTHON]Add unboundedPreceding(), un...

2018-01-30 Thread huaxingao
Github user huaxingao commented on a diff in the pull request: https://github.com/apache/spark/pull/20400#discussion_r164966938 --- Diff: python/pyspark/sql/window.py --- @@ -124,16 +126,20 @@ def rangeBetween(start, end): values directly. :param

[GitHub] spark issue #20442: [SPARK-23265][SQL]Update multi-column error handling log...

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20442 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20442: [SPARK-23265][SQL]Update multi-column error handling log...

2018-01-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20442 **[Test build #86864 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86864/testReport)** for PR 20442 at commit

[GitHub] spark issue #20442: [SPARK-23265][SQL]Update multi-column error handling log...

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20442 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86864/ Test PASSed. ---

[GitHub] spark issue #20373: [SPARK-23159][PYTHON] Update cloudpickle to v0.4.2 plus ...

2018-01-30 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/20373 @HyukjinKwon Yes, I meant to port it too. I agree with matching it to v0.4.2 as same as we can, so I don't think we should include formatting one but bug fixes. ---

[GitHub] spark issue #20448: [SPARK-23203][SQL] make DataSourceV2Relation immutable

2018-01-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20448 **[Test build #86865 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86865/testReport)** for PR 20448 at commit

[GitHub] spark issue #20448: [SPARK-23203][SQL] make DataSourceV2Relation immutable

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20448 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/419/

[GitHub] spark pull request #20400: [SPARK-23084][PYTHON]Add unboundedPreceding(), un...

2018-01-30 Thread huaxingao
Github user huaxingao commented on a diff in the pull request: https://github.com/apache/spark/pull/20400#discussion_r164965129 --- Diff: python/pyspark/sql/functions.py --- @@ -809,6 +809,45 @@ def ntile(n): return Column(sc._jvm.functions.ntile(int(n)))

[GitHub] spark issue #20448: [SPARK-23203][SQL] make DataSourceV2Relation immutable

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20448 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20400: [SPARK-23084][PYTHON]Add unboundedPreceding(), un...

2018-01-30 Thread huaxingao
Github user huaxingao commented on a diff in the pull request: https://github.com/apache/spark/pull/20400#discussion_r164965040 --- Diff: python/pyspark/sql/functions.py --- @@ -809,6 +809,45 @@ def ntile(n): return Column(sc._jvm.functions.ntile(int(n)))

[GitHub] spark issue #20373: [SPARK-23159][PYTHON] Update cloudpickle to v0.4.2 plus ...

2018-01-30 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20373 Hm .. I was thinking we should focus on matching it to v0.4.2 as same as we can. Few bug links I and @ueshin found looked regressions comparing to our copy so I suggested to include them here

[GitHub] spark issue #20445: [SPARK-23092][SQL] Migrate MemoryStream to DataSourceV2 ...

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20445 Build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #20445: [SPARK-23092][SQL] Migrate MemoryStream to DataSourceV2 ...

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20445 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86855/ Test FAILed. ---

[GitHub] spark issue #20445: [SPARK-23092][SQL] Migrate MemoryStream to DataSourceV2 ...

2018-01-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20445 **[Test build #86855 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86855/testReport)** for PR 20445 at commit

[GitHub] spark issue #20373: [SPARK-23159][PYTHON] Update cloudpickle to v0.4.2 plus ...

2018-01-30 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20373 @ueshin, is https://github.com/cloudpipe/cloudpickle/pull/140 a regression comparing to our cloudcpikle copy, or do you suggest to port it too as it's a simple and clean bug fix? Seems the same

[GitHub] spark issue #20449: [SPARK-23040][CORE]: Returns interruptible iterator for ...

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20449 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20449: [SPARK-23040][CORE]: Returns interruptible iterator for ...

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20449 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20449: [SPARK-23040][CORE]: Returns interruptible iterat...

2018-01-30 Thread advancedxy
GitHub user advancedxy opened a pull request: https://github.com/apache/spark/pull/20449 [SPARK-23040][CORE]: Returns interruptible iterator for shuffle reader ## What changes were proposed in this pull request? Before this commit, a non-interruptible iterator is returned

[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...

2018-01-30 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/20404 @felixcheung what is your opinion on this, do we really need to handle this case? --- - To unsubscribe, e-mail:

[GitHub] spark issue #20422: [SPARK-23253][Core][Shuffle]Only write shuffle temporary...

2018-01-30 Thread yaooqinn
Github user yaooqinn commented on the issue: https://github.com/apache/spark/pull/20422 @squito add a test for index file. plz check it again, thanks. --- - To unsubscribe, e-mail:

[GitHub] spark issue #17332: [SPARK-10764][ML] Add optional caching to Pipelines

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17332 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20373: [SPARK-23159][PYTHON] Update cloudpickle to v0.4.2 plus ...

2018-01-30 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/20373 >Wait .. @BryanCutler, did you port the formatting one here ..? I was thinking we should match it to v0.4.2 as same as possible to reduce the diff. Yes I added that here as well because

[GitHub] spark issue #20415: [SPARK-23247][SQL]combines Unsafe operations and statist...

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20415 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20415: [SPARK-23247][SQL]combines Unsafe operations and statist...

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20415 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86860/ Test PASSed. ---

[GitHub] spark issue #20415: [SPARK-23247][SQL]combines Unsafe operations and statist...

2018-01-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20415 **[Test build #86860 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86860/testReport)** for PR 20415 at commit

[GitHub] spark pull request #20447: [SPARK-23279][SS] Avoid triggering distributed jo...

2018-01-30 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20447 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20422: [SPARK-23253][Core][Shuffle]Only write shuffle temporary...

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20422 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86858/ Test PASSed. ---

[GitHub] spark issue #20422: [SPARK-23253][Core][Shuffle]Only write shuffle temporary...

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20422 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20447: [SPARK-23279][SS] Avoid triggering distributed job for C...

2018-01-30 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/20447 Merging to master and 2.3. Thanks for the review! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #20422: [SPARK-23253][Core][Shuffle]Only write shuffle temporary...

2018-01-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20422 **[Test build #86858 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86858/testReport)** for PR 20422 at commit

[GitHub] spark issue #20442: [SPARK-23265][SQL]Update multi-column error handling log...

2018-01-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20442 **[Test build #86864 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86864/testReport)** for PR 20442 at commit

[GitHub] spark issue #20442: [SPARK-23265][SQL]Update multi-column error handling log...

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20442 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20442: [SPARK-23265][SQL]Update multi-column error handling log...

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20442 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/418/

[GitHub] spark pull request #20448: [SPARK-23203][SQL] make DataSourceV2Relation immu...

2018-01-30 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20448#discussion_r164958102 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala --- @@ -17,16 +17,57 @@ package

[GitHub] spark issue #20447: [SPARK-23279][SS] Avoid triggering distributed job for C...

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20447 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20447: [SPARK-23279][SS] Avoid triggering distributed job for C...

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20447 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86859/ Test PASSed. ---

[GitHub] spark issue #20447: [SPARK-23279][SS] Avoid triggering distributed job for C...

2018-01-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20447 **[Test build #86859 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86859/testReport)** for PR 20447 at commit

[GitHub] spark issue #20373: [SPARK-23159][PYTHON] Update cloudpickle to v0.4.2 plus ...

2018-01-30 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/20373 What about https://github.com/cloudpipe/cloudpickle/pull/140? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #20419: [SPARK-23032][SQL][FOLLOW-UP]Add codegenStageId in comme...

2018-01-30 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20419 Then, we should add a test case to ensure it will not be broken. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #20442: [SPARK-23265][SQL]Update multi-column error handl...

2018-01-30 Thread huaxingao
Github user huaxingao commented on a diff in the pull request: https://github.com/apache/spark/pull/20442#discussion_r164956149 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/QuantileDiscretizer.scala --- @@ -167,25 +167,31 @@ final class QuantileDiscretizer

[GitHub] spark pull request #20448: [SPARK-23203][SQL] make DataSourceV2Relation immu...

2018-01-30 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20448#discussion_r164956171 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala --- @@ -17,16 +17,57 @@ package

[GitHub] spark pull request #20442: [SPARK-23265][SQL]Update multi-column error handl...

2018-01-30 Thread huaxingao
Github user huaxingao commented on a diff in the pull request: https://github.com/apache/spark/pull/20442#discussion_r164955798 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/QuantileDiscretizer.scala --- @@ -167,25 +167,31 @@ final class QuantileDiscretizer

[GitHub] spark pull request #20448: [SPARK-23203][SQL] make DataSourceV2Relation immu...

2018-01-30 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20448#discussion_r164955518 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala --- @@ -17,16 +17,57 @@ package

[GitHub] spark pull request #20448: [SPARK-23203][SQL] make DataSourceV2Relation immu...

2018-01-30 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20448#discussion_r164955297 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala --- @@ -17,16 +17,57 @@ package

[GitHub] spark pull request #20448: [SPARK-23203][SQL] make DataSourceV2Relation immu...

2018-01-30 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20448#discussion_r164954926 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala --- @@ -17,16 +17,57 @@ package

[GitHub] spark issue #20279: [SPARK-23092][SQL] Migrate MemoryStream to DataSourceV2 ...

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20279 Build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #20279: [SPARK-23092][SQL] Migrate MemoryStream to DataSourceV2 ...

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20279 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86853/ Test FAILed. ---

[GitHub] spark issue #20448: [SPARK-23203][SQL] make DataSourceV2Relation immutable

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20448 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86863/ Test FAILed. ---

[GitHub] spark issue #20279: [SPARK-23092][SQL] Migrate MemoryStream to DataSourceV2 ...

2018-01-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20279 **[Test build #86853 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86853/testReport)** for PR 20279 at commit

[GitHub] spark issue #20448: [SPARK-23203][SQL] make DataSourceV2Relation immutable

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20448 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20448: [SPARK-23203][SQL] make DataSourceV2Relation immutable

2018-01-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20448 **[Test build #86863 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86863/testReport)** for PR 20448 at commit

[GitHub] spark pull request #20448: [SPARK-23203][SQL] make DataSourceV2Relation immu...

2018-01-30 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20448#discussion_r164954221 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala --- @@ -17,16 +17,57 @@ package

[GitHub] spark pull request #20448: [SPARK-23203][SQL] make DataSourceV2Relation immu...

2018-01-30 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20448#discussion_r164953600 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2QueryPlan.scala --- @@ -19,50 +19,31 @@ package

[GitHub] spark pull request #20448: [SPARK-23203][SQL] make DataSourceV2Relation immu...

2018-01-30 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20448#discussion_r164954392 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala --- @@ -17,16 +17,57 @@ package

[GitHub] spark issue #20448: [SPARK-23203][SQL] make DataSourceV2Relation immutable

2018-01-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20448 **[Test build #86863 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86863/testReport)** for PR 20448 at commit

[GitHub] spark issue #20448: [SPARK-23203][SQL] make DataSourceV2Relation immutable

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20448 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20448: [SPARK-23203][SQL] make DataSourceV2Relation immutable

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20448 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/417/

[GitHub] spark issue #20279: [SPARK-23092][SQL] Migrate MemoryStream to DataSourceV2 ...

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20279 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86854/ Test FAILed. ---

[GitHub] spark issue #20279: [SPARK-23092][SQL] Migrate MemoryStream to DataSourceV2 ...

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20279 Build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #20279: [SPARK-23092][SQL] Migrate MemoryStream to DataSourceV2 ...

2018-01-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20279 **[Test build #86854 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86854/testReport)** for PR 20279 at commit

[GitHub] spark pull request #20435: [SPARK-23268][SQL]Reorganize packages in data sou...

2018-01-30 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/20435#discussion_r164953225 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSourceOffset.scala --- @@ -20,14 +20,16 @@ package

[GitHub] spark pull request #20435: [SPARK-23268][SQL]Reorganize packages in data sou...

2018-01-30 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/20435#discussion_r164953380 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/sources/memoryV2.scala --- @@ -30,9 +30,8 @@ import

[GitHub] spark issue #20448: [SPARK-23203][SQL] make DataSourceV2Relation immutable

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20448 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86862/ Test FAILed. ---

[GitHub] spark issue #20448: [SPARK-23203][SQL] make DataSourceV2Relation immutable

2018-01-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20448 **[Test build #86862 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86862/testReport)** for PR 20448 at commit

[GitHub] spark issue #20448: [SPARK-23203][SQL] make DataSourceV2Relation immutable

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20448 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20448: [SPARK-23203][SQL] make DataSourceV2Relation immutable

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20448 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20448: [SPARK-23203][SQL] make DataSourceV2Relation immutable

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20448 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86861/ Test FAILed. ---

[GitHub] spark issue #20448: [SPARK-23203][SQL] make DataSourceV2Relation immutable

2018-01-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20448 **[Test build #86861 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86861/testReport)** for PR 20448 at commit

[GitHub] spark pull request #20448: [SPARK-23203][SQL] make DataSourceV2Relation immu...

2018-01-30 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20448#discussion_r164952844 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2QueryPlan.scala --- @@ -19,50 +19,31 @@ package

[GitHub] spark issue #20448: [SPARK-23203][SQL] make DataSourceV2Relation immutable

2018-01-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20448 **[Test build #86862 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86862/testReport)** for PR 20448 at commit

[GitHub] spark issue #20448: [SPARK-23203][SQL] make DataSourceV2Relation immutable

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20448 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20448: [SPARK-23203][SQL] make DataSourceV2Relation immutable

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20448 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/416/

[GitHub] spark issue #20448: [SPARK-23203][SQL] make DataSourceV2Relation immutable

2018-01-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20448 **[Test build #86861 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86861/testReport)** for PR 20448 at commit

[GitHub] spark issue #20448: [SPARK-23203][SQL] make DataSourceV2Relation immutable

2018-01-30 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20448 cc @rdblue @tdas @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #20448: [SPARK-23203][SQL] make DataSourceV2Relation immu...

2018-01-30 Thread cloud-fan
GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/20448 [SPARK-23203][SQL] make DataSourceV2Relation immutable ## What changes were proposed in this pull request? This is inspired by https://github.com/apache/spark/pull/20387, but only focus

[GitHub] spark issue #20419: [SPARK-23032][SQL][FOLLOW-UP]Add codegenStageId in comme...

2018-01-30 Thread rednaxelafx
Github user rednaxelafx commented on the issue: https://github.com/apache/spark/pull/20419 @gatorsmile Not directly. The `CodeAndComment` case class is just a "container", it doesn't handle what gets into the `body` field. When we force embed a comment, it'll leave a comment as a

[GitHub] spark issue #20419: [SPARK-23032][SQL][FOLLOW-UP]Add codegenStageId in comme...

2018-01-30 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20419 @rednaxelafx Does the following [codes](https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala#L1286-L1294)

[GitHub] spark pull request #20419: [SPARK-23032][SQL][FOLLOW-UP]Add codegenStageId i...

2018-01-30 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20419#discussion_r164951129 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -1227,12 +1227,13 @@ class

[GitHub] spark issue #20400: [SPARK-23084][PYTHON]Add unboundedPreceding(), unbounded...

2018-01-30 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20400 LGTM too except the three comments above. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20400: [SPARK-23084][PYTHON]Add unboundedPreceding(), un...

2018-01-30 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20400#discussion_r164950417 --- Diff: python/pyspark/sql/functions.py --- @@ -809,6 +809,45 @@ def ntile(n): return Column(sc._jvm.functions.ntile(int(n)))

[GitHub] spark pull request #20400: [SPARK-23084][PYTHON]Add unboundedPreceding(), un...

2018-01-30 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20400#discussion_r164950531 --- Diff: python/pyspark/sql/functions.py --- @@ -809,6 +809,45 @@ def ntile(n): return Column(sc._jvm.functions.ntile(int(n)))

  1   2   3   4   5   6   7   >