[GitHub] spark pull request #20119: [SPARK-21475][Core]Revert "[SPARK-21475][CORE] Us...

Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/20119#discussion_r159372233 --- Diff: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/ShuffleIndexInformation.java --- @@ -39,7 +39,7 @@ public

[GitHub] spark issue #16578: [SPARK-4502][SQL] Parquet nested column pruning

Github user DaimonPl commented on the issue: https://github.com/apache/spark/pull/16578 New year, new review? ;) --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20057: [SPARK-22880][SQL] Add cascadeTruncate option to ...

Github user danielvdende commented on a diff in the pull request: https://github.com/apache/spark/pull/20057#discussion_r159371171 --- Diff: docs/sql-programming-guide.md --- @@ -1339,6 +1339,13 @@ the following case-insensitive options: This is a JDBC writer related

[GitHub] spark pull request #20119: [SPARK-21475][Core]Revert "[SPARK-21475][CORE] Us...

Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/20119#discussion_r159371080 --- Diff: core/src/main/scala/org/apache/spark/shuffle/IndexShuffleBlockResolver.scala --- @@ -198,7 +196,7 @@ private[spark] class

[GitHub] spark pull request #20119: [SPARK-21475][Core]Revert "[SPARK-21475][CORE] Us...

Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/20119#discussion_r159370941 --- Diff: common/network-common/src/main/java/org/apache/spark/network/buffer/FileSegmentManagedBuffer.java --- @@ -133,7 +132,7 @@ public Object

[GitHub] spark pull request #20119: [SPARK-21475][Core]Revert "[SPARK-21475][CORE] Us...

Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/20119#discussion_r159370876 --- Diff: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/OneForOneBlockFetcher.java --- @@ -165,7 +165,7 @@ private void

[GitHub] spark pull request #20057: [SPARK-22880][SQL] Add cascadeTruncate option to ...

Github user danielvdende commented on a diff in the pull request: https://github.com/apache/spark/pull/20057#discussion_r159370871 --- Diff: docs/sql-programming-guide.md --- @@ -1339,6 +1339,13 @@ the following case-insensitive options: This is a JDBC writer related

[GitHub] spark pull request #20119: [SPARK-21475][Core]Revert "[SPARK-21475][CORE] Us...

Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/20119#discussion_r159370554 --- Diff: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/OneForOneBlockFetcher.java --- @@ -165,7 +165,7 @@ private void

[GitHub] spark pull request #20119: [SPARK-21475][Core]Revert "[SPARK-21475][CORE] Us...

Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/20119#discussion_r159370490 --- Diff: core/src/main/scala/org/apache/spark/shuffle/IndexShuffleBlockResolver.scala --- @@ -198,7 +196,7 @@ private[spark] class

[GitHub] spark pull request #20097: [SPARK-22912] v2 data source support in MicroBatc...

Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/20097#discussion_r159365598 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/sources/RateStreamSourceV2.scala --- @@ -32,16 +32,17 @@ import

[GitHub] spark pull request #20097: [SPARK-22912] v2 data source support in MicroBatc...

Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/20097#discussion_r159365501 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/ProgressReporter.scala --- @@ -226,7 +226,7 @@ trait ProgressReporter extends

[GitHub] spark pull request #20097: [SPARK-22912] v2 data source support in MicroBatc...

Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/20097#discussion_r159365350 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala --- @@ -357,7 +397,16 @@ class MicroBatchExecution(

[GitHub] spark pull request #20097: [SPARK-22912] v2 data source support in MicroBatc...

Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/20097#discussion_r159102835 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala --- @@ -35,6 +35,12 @@ case class

[GitHub] spark pull request #20097: [SPARK-22912] v2 data source support in MicroBatc...

Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/20097#discussion_r159114315 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala --- @@ -236,14 +257,31 @@ class MicroBatchExecution(

[GitHub] spark pull request #20097: [SPARK-22912] v2 data source support in MicroBatc...

Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/20097#discussion_r159364715 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala --- @@ -367,21 +416,20 @@ class MicroBatchExecution(

[GitHub] spark pull request #20097: [SPARK-22912] v2 data source support in MicroBatc...

Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/20097#discussion_r159112097 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/streaming/DataStreamReader.scala --- @@ -167,6 +167,24 @@ final class DataStreamReader

[GitHub] spark pull request #20097: [SPARK-22912] v2 data source support in MicroBatc...

Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/20097#discussion_r159364509 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala --- @@ -317,6 +355,8 @@ class MicroBatchExecution(

[GitHub] spark pull request #20097: [SPARK-22912] v2 data source support in MicroBatc...

Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/20097#discussion_r159111979 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/streaming/DataStreamReader.scala --- @@ -167,6 +167,24 @@ final class DataStreamReader

[GitHub] spark pull request #20097: [SPARK-22912] v2 data source support in MicroBatc...

Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/20097#discussion_r159112055 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/streaming/DataStreamReader.scala --- @@ -167,6 +167,24 @@ final class DataStreamReader

[GitHub] spark pull request #20097: [SPARK-22912] v2 data source support in MicroBatc...

Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/20097#discussion_r159114536 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala --- @@ -367,21 +416,20 @@ class MicroBatchExecution(

[GitHub] spark pull request #20097: [SPARK-22912] v2 data source support in MicroBatc...

Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/20097#discussion_r159110696 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala --- @@ -68,8 +76,20 @@ class MicroBatchExecution(

[GitHub] spark pull request #20097: [SPARK-22912] v2 data source support in MicroBatc...

Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/20097#discussion_r159356689 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/sources/RateStreamSourceV2.scala --- @@ -32,16 +32,17 @@ import

[GitHub] spark pull request #20097: [SPARK-22912] v2 data source support in MicroBatc...

Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/20097#discussion_r159356698 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/sources/RateStreamSourceV2.scala --- @@ -32,16 +32,17 @@ import

[GitHub] spark pull request #20097: [SPARK-22912] v2 data source support in MicroBatc...

Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/20097#discussion_r159114244 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala --- @@ -192,7 +212,8 @@ class MicroBatchExecution(

[GitHub] spark pull request #20097: [SPARK-22912] v2 data source support in MicroBatc...

Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/20097#discussion_r159356713 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/sources/RateStreamSourceV2.scala --- @@ -111,7 +112,7 @@ class

[GitHub] spark pull request #20097: [SPARK-22912] v2 data source support in MicroBatc...

Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/20097#discussion_r159364969 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala --- @@ -236,14 +257,31 @@ class MicroBatchExecution(

[GitHub] spark issue #20133: [SPARK-22934] [SQL] Make optional clauses order insensit...

Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20133 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20133: [SPARK-22934] [SQL] Make optional clauses order i...

Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20133#discussion_r159364965 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala --- @@ -1180,7 +1202,7 @@ class SparkSqlAstBuilder(conf: SQLConf)

[GitHub] spark pull request #20133: [SPARK-22934] [SQL] Make optional clauses order i...

Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20133#discussion_r159364941 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala --- @@ -1366,6 +1388,15 @@ class SparkSqlAstBuilder(conf: SQLConf)

[GitHub] spark pull request #20133: [SPARK-22934] [SQL] Make optional clauses order i...

Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20133#discussion_r159364923 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala --- @@ -1104,28 +1118,36 @@ class SparkSqlAstBuilder(conf: SQLConf)

[GitHub] spark pull request #20133: [SPARK-22934] [SQL] Make optional clauses order i...

Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20133#discussion_r159364557 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala --- @@ -383,23 +383,34 @@ class SparkSqlAstBuilder(conf: SQLConf)

[GitHub] spark pull request #20119: [SPARK-21475][Core]Revert "[SPARK-21475][CORE] Us...

Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/20119#discussion_r159364034 --- Diff: common/network-common/src/main/java/org/apache/spark/network/buffer/FileSegmentManagedBuffer.java --- @@ -94,9 +93,9 @@ public ByteBuffer

[GitHub] spark issue #18714: [SPARK-20236][SQL] dynamic partition overwrite

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18714 **[Test build #85621 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85621/testReport)** for PR 18714 at commit

[GitHub] spark issue #20116: [SPARK-20960][SQL] make ColumnVector public

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20116 **[Test build #85620 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85620/testReport)** for PR 20116 at commit

[GitHub] spark pull request #20136: [SPARK-22938] Assert that SQLConf.get is accessed...

Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20136#discussion_r159363498 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -70,7 +72,7 @@ object SQLConf { * Default config. Only

[GitHub] spark pull request #20116: [SPARK-20960][SQL] make ColumnVector public

Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20116#discussion_r159363145 --- Diff: sql/core/src/main/java/org/apache/spark/sql/vectorized/ColumnVector.java --- @@ -14,32 +14,38 @@ * See the License for the specific

[GitHub] spark issue #20137: [SPARK-22939] [PySpark] Support Spark UDF in registerFun...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20137 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85619/ Test FAILed. ---

[GitHub] spark issue #20137: [SPARK-22939] [PySpark] Support Spark UDF in registerFun...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20137 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20137: [SPARK-22939] [PySpark] Support Spark UDF in registerFun...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20137 **[Test build #85619 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85619/testReport)** for PR 20137 at commit

[GitHub] spark issue #20137: [SPARK-22939] [PySpark] Support Spark UDF in registerFun...

Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20137 With this diff: ```diff --- a/python/pyspark/sql/udf.py +++ b/python/pyspark/sql/udf.py @@ -173,4 +173,4 @@ class UserDefinedFunction(object): .. versionadded::

[GitHub] spark issue #20137: [SPARK-22939] [PySpark] Support Spark UDF in registerFun...

Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20137 With this diff: ```diff diff --git a/python/pyspark/sql/udf.py b/python/pyspark/sql/udf.py index 54b5a8656e1..24de9839e90 100644 --- a/python/pyspark/sql/udf.py +++

[GitHub] spark issue #20137: [SPARK-22939] [PySpark] Support Spark UDF in registerFun...

Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20137 Take your time. I will not be online in the next two hours. --- - To unsubscribe, e-mail:

[GitHub] spark issue #20137: [SPARK-22939] [PySpark] Support Spark UDF in registerFun...

Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20137 Let me test it and be back soon. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #20137: [SPARK-22939] [PySpark] Support Spark UDF in registerFun...

Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20137 Can you run the command? ``` help(udf(lambda: 1, "integer").asNondeterministic()) ``` --- - To unsubscribe,

[GitHub] spark issue #20137: [SPARK-22939] [PySpark] Support Spark UDF in registerFun...

Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20137 but if we do ```diff +wrapper.asNondeterministic = lambda: self.asNondeterministic()._wrapped() ``` I think it will still show a proper pydoc .. ---

[GitHub] spark issue #20137: [SPARK-22939] [PySpark] Support Spark UDF in registerFun...

Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20137 I am not against anything, but the outputs of the following two are inconsistent. It looks confusing to end users. ``` help(udf(lambda: 1, "integer").asNondeterministic())

[GitHub] spark issue #20137: [SPARK-22939] [PySpark] Support Spark UDF in registerFun...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20137 **[Test build #85619 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85619/testReport)** for PR 20137 at commit

[GitHub] spark issue #20137: [SPARK-22939] [PySpark] Support Spark UDF in registerFun...

Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20137 Ur, @gatorsmile, then, we will return a wrapped function from `UserDefinedFunction(). asNondeterministic`. Mind if I ask to elaborate why? I thought `UserDefinedFunction` should still return

[GitHub] spark issue #20068: [SPARK-17916][SQL] Fix empty string being parsed as null...

Github user aa8y commented on the issue: https://github.com/apache/spark/pull/20068 I'll work on it in the next week or two. That would involve a PR to the Univocity CSV parser. --- - To unsubscribe, e-mail:

[GitHub] spark issue #20137: [SPARK-22939] [PySpark] Support Spark UDF in registerFun...

Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20137 @HyukjinKwon We need to fix `asNondeterministic ` ``` def asNondeterministic(self): """ Updates UserDefinedFunction to nondeterministic. ..

[GitHub] spark issue #20068: [SPARK-17916][SQL] Fix empty string being parsed as null...

Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20068 Can we make the tests pass BTW, @aa8y? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20013: [SPARK-20657][core] Speed up rendering of the stages pag...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20013 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85614/ Test PASSed. ---

[GitHub] spark issue #20013: [SPARK-20657][core] Speed up rendering of the stages pag...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20013 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20013: [SPARK-20657][core] Speed up rendering of the stages pag...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20013 **[Test build #85614 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85614/testReport)** for PR 20013 at commit

[GitHub] spark issue #18801: SPARK-10878 Fix race condition when multiple clients res...

Github user kiszk commented on the issue: https://github.com/apache/spark/pull/18801 gentle ping @Victsm --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #18801: SPARK-10878 Fix race condition when multiple clients res...

Github user kiszk commented on the issue: https://github.com/apache/spark/pull/18801 +1 for UUID --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #18714: [SPARK-20236][SQL] dynamic partition overwrite

Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18714 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20137: [SPARK-22939] [PySpark] Support Spark UDF in registerFun...

Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20137 Hey @gatorsmile, I was just looking into this now. How about we have `_unwrapped` for wrapped function and then we return wrapped function from wrapped function and `UserDefinedFunction` from

[GitHub] spark pull request #20135: [SPARK-22937][SQL] SQL elt output binary for bina...

Github user gczsjdy commented on a diff in the pull request: https://github.com/apache/spark/pull/20135#discussion_r159353652 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -271,33 +271,45 @@ case class

[GitHub] spark issue #20137: [SPARK-22939] [PySpark] Support Spark UDF in registerFun...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20137 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85618/ Test PASSed. ---

[GitHub] spark issue #20137: [SPARK-22939] [PySpark] Support Spark UDF in registerFun...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20137 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20137: [SPARK-22939] [PySpark] Support Spark UDF in registerFun...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20137 **[Test build #85618 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85618/testReport)** for PR 20137 at commit

[GitHub] spark pull request #18714: [SPARK-20236][SQL] dynamic partition overwrite

Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18714#discussion_r159352979 --- Diff: core/src/main/scala/org/apache/spark/internal/io/HadoopMapReduceCommitProtocol.scala --- @@ -85,11 +104,16 @@ class

[GitHub] spark pull request #18714: [SPARK-20236][SQL] dynamic partition overwrite

Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18714#discussion_r159352867 --- Diff: core/src/main/scala/org/apache/spark/internal/io/HadoopMapReduceCommitProtocol.scala --- @@ -39,8 +39,19 @@ import

[GitHub] spark issue #11994: [SPARK-14151] Expose metrics Source and Sink interface

Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/11994 Sorry for late response, I was off last two weeks. Currently I don't have a better solution for this, @CodingCat let me think about your suggestion, thanks a lot :). ---

[GitHub] spark issue #20137: [SPARK-22939] [PySpark] Support Spark UDF in registerFun...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20137 **[Test build #85618 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85618/testReport)** for PR 20137 at commit

[GitHub] spark issue #20136: [SPARK-22938] Assert that SQLConf.get is accessed only o...

Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20136 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20119: [SPARK-21475][Core]Revert "[SPARK-21475][CORE] Use NIO's...

Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/20119 Sorry I haven't checked the details, let me take a look at it. The changes I made was trying to fix memory issue for shuffle (especially external shuffle service), this issue was occurred in our

[GitHub] spark issue #20133: [SPARK-22934] [SQL] Make optional clauses order insensit...

Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20133 cc @cloud-fan @hvanhovell --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #18906: [SPARK-21692][PYSPARK][SQL] Add nullability suppo...

Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18906#discussion_r159350021 --- Diff: python/pyspark/sql/udf.py --- @@ -17,6 +17,7 @@ """ User-defined function related classes and functions """ +import copy

[GitHub] spark pull request #18906: [SPARK-21692][PYSPARK][SQL] Add nullability suppo...

Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18906#discussion_r159349970 --- Diff: python/pyspark/sql/udf.py --- @@ -174,3 +178,12 @@ def asNondeterministic(self): """ self._deterministic = False

[GitHub] spark pull request #18906: [SPARK-21692][PYSPARK][SQL] Add nullability suppo...

Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18906#discussion_r159349251 --- Diff: python/pyspark/sql/udf.py --- @@ -174,3 +178,12 @@ def asNondeterministic(self): """ self._deterministic = False

[GitHub] spark pull request #18906: [SPARK-21692][PYSPARK][SQL] Add nullability suppo...

Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18906#discussion_r159349953 --- Diff: python/pyspark/sql/udf.py --- @@ -174,3 +178,12 @@ def asNondeterministic(self): """ self._deterministic = False

[GitHub] spark issue #20137: [SPARK-22939] [PySpark] Support Spark UDF in registerFun...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20137 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20137: [SPARK-22939] [PySpark] Support Spark UDF in registerFun...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20137 **[Test build #85617 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85617/testReport)** for PR 20137 at commit

[GitHub] spark issue #20137: [SPARK-22939] [PySpark] Support Spark UDF in registerFun...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20137 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85617/ Test FAILed. ---

[GitHub] spark issue #20137: [SPARK-22939] [PySpark] Support Spark UDF in registerFun...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20137 **[Test build #85617 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85617/testReport)** for PR 20137 at commit

[GitHub] spark issue #19872: [SPARK-22274][PYTHON][SQL] User-defined aggregation func...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19872 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19872: [SPARK-22274][PYTHON][SQL] User-defined aggregation func...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19872 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85613/ Test PASSed. ---

[GitHub] spark issue #19872: [SPARK-22274][PYTHON][SQL] User-defined aggregation func...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19872 **[Test build #85613 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85613/testReport)** for PR 19872 at commit

[GitHub] spark issue #18906: [SPARK-21692][PYSPARK][SQL] Add nullability support to P...

Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18906 I think we expect Python native function for `spark.udf.register` for now. Seems another issue. Returning `UserDefinedFunction` seems another issue too as it won't show up the pydoc

[GitHub] spark issue #20137: [SPARK-22939] [PySpark] Support Spark UDF in registerFun...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20137 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85616/ Test FAILed. ---

[GitHub] spark issue #20137: [SPARK-22939] [PySpark] Support Spark UDF in registerFun...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20137 **[Test build #85616 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85616/testReport)** for PR 20137 at commit

[GitHub] spark issue #20137: [SPARK-22939] [PySpark] Support Spark UDF in registerFun...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20137 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19885: [SPARK-22587] Spark job fails if fs.defaultFS and applic...

Github user merlintang commented on the issue: https://github.com/apache/spark/pull/19885 My local test is ok. I would set up a system test and update this soon. sorry about this delay. On Tue, Jan 2, 2018 at 3:42 PM, Marcelo Vanzin wrote:

[GitHub] spark issue #20137: [SPARK-22939] [PySpark] Support Spark UDF in registerFun...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20137 **[Test build #85616 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85616/testReport)** for PR 20137 at commit

[GitHub] spark pull request #20131: [MINOR] Fix a bunch of typos

Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20131#discussion_r159343623 --- Diff: common/network-common/src/main/java/org/apache/spark/network/protocol/MessageWithHeader.java --- @@ -100,7 +100,7 @@ public long

[GitHub] spark issue #20124: [WIP][SPARK-22126][ML] Fix model-specific optimization s...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20124 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85615/ Test FAILed. ---

[GitHub] spark issue #20124: [WIP][SPARK-22126][ML] Fix model-specific optimization s...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20124 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20124: [WIP][SPARK-22126][ML] Fix model-specific optimization s...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20124 **[Test build #85615 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85615/testReport)** for PR 20124 at commit

[GitHub] spark issue #20124: [WIP][SPARK-22126][ML] Fix model-specific optimization s...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20124 **[Test build #85615 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85615/testReport)** for PR 20124 at commit

[GitHub] spark issue #20045: [Spark-22360][SQL][TEST] Add unit tests for Window Speci...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20045 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20045: [Spark-22360][SQL][TEST] Add unit tests for Window Speci...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20045 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85612/ Test PASSed. ---

[GitHub] spark issue #20045: [Spark-22360][SQL][TEST] Add unit tests for Window Speci...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20045 **[Test build #85612 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85612/testReport)** for PR 20045 at commit

[GitHub] spark issue #19885: [SPARK-22587] Spark job fails if fs.defaultFS and applic...

Github user vanzin commented on the issue: https://github.com/apache/spark/pull/19885 Any updates? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #16735: [SPARK-19228][SQL] Introduce tryParseDate method to proc...

Github user vanzin commented on the issue: https://github.com/apache/spark/pull/16735 @sergey-rubtsov this PR seems pretty broken... probably needs to be closed and a new one opened. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #20098: [SPARK-22914][DEPLOY] Register history.ui.port

Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20098#discussion_r159338937 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/ApplicationMasterUtilSuite.scala --- @@ -0,0 +1,44 @@ +/* + * Licensed

[GitHub] spark pull request #20098: [SPARK-22914][DEPLOY] Register history.ui.port

Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20098#discussion_r159338996 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/ApplicationMasterUtilSuite.scala --- @@ -0,0 +1,44 @@ +/* + * Licensed

[GitHub] spark pull request #20098: [SPARK-22914][DEPLOY] Register history.ui.port

Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20098#discussion_r159338897 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/ApplicationMasterUtilSuite.scala --- @@ -0,0 +1,44 @@ +/* + * Licensed

[GitHub] spark pull request #20098: [SPARK-22914][DEPLOY] Register history.ui.port

Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20098#discussion_r159338562 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala --- @@ -836,6 +833,20 @@ object ApplicationMaster

  1   2   3   >