[GitHub] spark pull request #21316: [SPARK-20538][SQL] Wrap Dataset.reduce with withN...

2018-05-13 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/21316#discussion_r187838099 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -1607,7 +1607,9 @@ class Dataset[T] private[sql]( */ @Experimental

[GitHub] spark pull request #21299: [SPARK-24250][SQL] support accessing SQLConf insi...

2018-05-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21299#discussion_r187841313 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SQLExecution.scala --- @@ -90,13 +92,33 @@ object SQLExecution { * thread from

[GitHub] spark pull request #21296: [SPARK-24244][SQL] Passing only required columns ...

2018-05-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21296#discussion_r187832513 --- Diff: docs/sql-programming-guide.md --- @@ -1814,6 +1814,7 @@ working with timestamps in `pandas_udf`s to get the best performance, see - In

[GitHub] spark pull request #21299: [SPARK-24250][SQL] support accessing SQLConf insi...

2018-05-13 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/21299#discussion_r187833760 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SQLExecution.scala --- @@ -90,13 +92,33 @@ object SQLExecution { * thread from

[GitHub] spark issue #21092: [SPARK-23984][K8S] Initial Python Bindings for PySpark o...

2018-05-13 Thread ifilonenko
Github user ifilonenko commented on the issue: https://github.com/apache/spark/pull/21092 jenkins, retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #21318: [minor] Update docs for functions.scala to make it clear...

2018-05-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21318 **[Test build #90567 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90567/testReport)** for PR 21318 at commit

[GitHub] spark issue #21318: [minor] Update docs for functions.scala to make it clear...

2018-05-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21318 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3187/

[GitHub] spark issue #21318: [minor] Update docs for functions.scala to make it clear...

2018-05-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21318 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21318: [minor] Update docs for functions.scala to make it clear...

2018-05-13 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/21318 Hm the failure doesn't look like it's caused by this PR. Do you guys know what's going on? --- - To unsubscribe, e-mail:

[GitHub] spark pull request #20800: [SPARK-23627][SQL] Provide isEmpty in Dataset

2018-05-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20800#discussion_r187831971 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -511,6 +511,14 @@ class Dataset[T] private[sql]( */ def

[GitHub] spark issue #21092: [SPARK-23984][K8S] Initial Python Bindings for PySpark o...

2018-05-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21092 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21092: [SPARK-23984][K8S] Initial Python Bindings for PySpark o...

2018-05-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21092 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3186/

[GitHub] spark issue #21092: [SPARK-23984][K8S] Initial Python Bindings for PySpark o...

2018-05-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21092 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/3091/ ---

[GitHub] spark issue #21318: [minor] Update docs for functions.scala to make it clear...

2018-05-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21318 **[Test build #90567 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90567/testReport)** for PR 21318 at commit

[GitHub] spark pull request #21246: [SPARK-23901][SQL] Add masking functions

2018-05-13 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21246#discussion_r187839605 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/MaskExpressionsSuite.scala --- @@ -0,0 +1,236 @@ +/* + * Licensed

[GitHub] spark pull request #21246: [SPARK-23901][SQL] Add masking functions

2018-05-13 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21246#discussion_r187836985 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/maskExpressions.scala --- @@ -0,0 +1,569 @@ +/* + * Licensed to the

[GitHub] spark issue #21318: [minor] Update docs for functions.scala to make it clear...

2018-05-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21318 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90567/ Test FAILed. ---

[GitHub] spark pull request #21246: [SPARK-23901][SQL] Add masking functions

2018-05-13 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21246#discussion_r187837123 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/maskExpressions.scala --- @@ -0,0 +1,569 @@ +/* + * Licensed to the

[GitHub] spark pull request #21246: [SPARK-23901][SQL] Add masking functions

2018-05-13 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21246#discussion_r187835032 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/maskExpressions.scala --- @@ -0,0 +1,569 @@ +/* + * Licensed to the

[GitHub] spark pull request #21246: [SPARK-23901][SQL] Add masking functions

2018-05-13 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21246#discussion_r187835126 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/MaskExpressionsUtils.java --- @@ -0,0 +1,80 @@ +/* + * Licensed to

[GitHub] spark pull request #21246: [SPARK-23901][SQL] Add masking functions

2018-05-13 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21246#discussion_r187835331 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/MaskExpressionsUtils.java --- @@ -0,0 +1,80 @@ +/* + * Licensed to

[GitHub] spark pull request #21246: [SPARK-23901][SQL] Add masking functions

2018-05-13 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21246#discussion_r187835834 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/maskExpressions.scala --- @@ -0,0 +1,569 @@ +/* + * Licensed to the

[GitHub] spark pull request #21246: [SPARK-23901][SQL] Add masking functions

2018-05-13 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21246#discussion_r187839186 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/MaskExpressionsSuite.scala --- @@ -0,0 +1,236 @@ +/* + * Licensed

[GitHub] spark issue #21318: [minor] Update docs for functions.scala to make it clear...

2018-05-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21318 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21318: [minor] Update docs for functions.scala to make i...

2018-05-13 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21318#discussion_r187842050 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala --- @@ -39,7 +39,21 @@ import org.apache.spark.util.Utils /**

[GitHub] spark pull request #21316: [SPARK-20538][SQL] Wrap Dataset.reduce with withN...

2018-05-13 Thread sohama4
Github user sohama4 commented on a diff in the pull request: https://github.com/apache/spark/pull/21316#discussion_r187832727 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -1607,7 +1607,9 @@ class Dataset[T] private[sql]( */

[GitHub] spark issue #21092: [SPARK-23984][K8S] Initial Python Bindings for PySpark o...

2018-05-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21092 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/3091/ ---

[GitHub] spark pull request #21318: [minor] Update docs for functions.scala to make i...

2018-05-13 Thread rxin
GitHub user rxin opened a pull request: https://github.com/apache/spark/pull/21318 [minor] Update docs for functions.scala to make it clear not all the built-in functions are defined there The title summarizes the change. You can merge this pull request into a Git repository by

[GitHub] spark issue #21318: [minor] Update docs for functions.scala to make it clear...

2018-05-13 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/21318 cc @gatorsmile @HyukjinKwon --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19789: [SPARK-22562][Streaming] CachedKafkaConsumer unsafe evic...

2018-05-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19789 **[Test build #90551 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90551/testReport)** for PR 19789 at commit

[GitHub] spark issue #21299: [SPARK-24250][SQL] support accessing SQLConf inside task...

2018-05-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21299 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21299: [SPARK-24250][SQL] support accessing SQLConf inside task...

2018-05-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21299 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90550/ Test FAILed. ---

[GitHub] spark issue #21299: [SPARK-24250][SQL] support accessing SQLConf inside task...

2018-05-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21299 **[Test build #90550 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90550/testReport)** for PR 21299 at commit

[GitHub] spark issue #19789: [SPARK-22562][Streaming] CachedKafkaConsumer unsafe evic...

2018-05-13 Thread daroo
Github user daroo commented on the issue: https://github.com/apache/spark/pull/19789 Hi @HyukjinKwon It seems that @zsxwing is not really interested in this PR. Should I cancel it and close https://issues.apache.org/jira/browse/SPARK-22562 ? BTW. Why AppVeyor CI

[GitHub] spark pull request #21307: [SPARK-24186][R][SQL]change reverse and concat to...

2018-05-13 Thread huaxingao
Github user huaxingao commented on a diff in the pull request: https://github.com/apache/spark/pull/21307#discussion_r187802112 --- Diff: R/pkg/R/functions.R --- @@ -2055,20 +2058,10 @@ setMethod("countDistinct", #' @details #' \code{concat}: Concatenates multiple

[GitHub] spark issue #19789: [SPARK-22562][Streaming] CachedKafkaConsumer unsafe evic...

2018-05-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19789 **[Test build #90551 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90551/testReport)** for PR 19789 at commit

[GitHub] spark issue #19789: [SPARK-22562][Streaming] CachedKafkaConsumer unsafe evic...

2018-05-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19789 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90551/ Test PASSed. ---

[GitHub] spark issue #19789: [SPARK-22562][Streaming] CachedKafkaConsumer unsafe evic...

2018-05-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19789 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21213: [SPARK-24120] Show `Jobs` page when `jobId` is missing

2018-05-13 Thread gengliangwang
Github user gengliangwang commented on the issue: https://github.com/apache/spark/pull/21213 @jongyoul @ajbozarth Personally I prefer to the current way, showing the error message:

[GitHub] spark issue #21312: [SPARK-24259][SQL] ArrayWriter for Arrow produces wrong ...

2018-05-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21312 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21312: [SPARK-24259][SQL] ArrayWriter for Arrow produces wrong ...

2018-05-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21312 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90547/ Test PASSed. ---

[GitHub] spark issue #21213: [SPARK-24120] Show `Jobs` page when `jobId` is missing

2018-05-13 Thread jongyoul
Github user jongyoul commented on the issue: https://github.com/apache/spark/pull/21213 @gengliangwang Thanks. But changing URI format might make any incompatibility with other versions, thus we need to consider other aspects more. ---

[GitHub] spark issue #21299: [SPARK-24250][SQL] support accessing SQLConf inside task...

2018-05-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21299 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21299: [SPARK-24250][SQL] support accessing SQLConf inside task...

2018-05-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21299 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90549/ Test FAILed. ---

[GitHub] spark issue #21299: [SPARK-24250][SQL] support accessing SQLConf inside task...

2018-05-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21299 **[Test build #90549 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90549/testReport)** for PR 21299 at commit

[GitHub] spark issue #21299: [SPARK-24250][SQL] support accessing SQLConf inside task...

2018-05-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21299 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21299: [SPARK-24250][SQL] support accessing SQLConf inside task...

2018-05-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21299 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3175/

[GitHub] spark issue #21304: Fix typo in UDF type match error message

2018-05-13 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21304 @wzhfy wanna try to merge this one? Seems safe to merge to both master and branch-2.3. --- - To unsubscribe, e-mail:

[GitHub] spark issue #21312: [SPARK-24259][SQL] ArrayWriter for Arrow produces wrong ...

2018-05-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21312 **[Test build #90547 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90547/testReport)** for PR 21312 at commit

[GitHub] spark issue #21299: [SPARK-24250][SQL] support accessing SQLConf inside task...

2018-05-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21299 **[Test build #90550 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90550/testReport)** for PR 21299 at commit

[GitHub] spark issue #21299: [SPARK-24250][SQL] support accessing SQLConf inside task...

2018-05-13 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/21299 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21301: [SPARK-24228][SQL] Fix Java lint errors

2018-05-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21301 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20761: [SPARK-20327][CORE][YARN] Add CLI support for YARN custo...

2018-05-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20761 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20761: [SPARK-20327][CORE][YARN] Add CLI support for YARN custo...

2018-05-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20761 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90553/ Test PASSed. ---

[GitHub] spark issue #20761: [SPARK-20327][CORE][YARN] Add CLI support for YARN custo...

2018-05-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20761 **[Test build #90553 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90553/testReport)** for PR 20761 at commit

[GitHub] spark pull request #21295: [SPARK-24230][SQL] Fix SpecificParquetRecordReade...

2018-05-13 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/21295#discussion_r187809226 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetQuerySuite.scala --- @@ -879,6 +879,18 @@ class

[GitHub] spark issue #18903: [SPARK-21590][SS]Window start time should support negati...

2018-05-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18903 **[Test build #4175 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4175/testReport)** for PR 18903 at commit

[GitHub] spark issue #19293: [SPARK-22079][SQL] Serializer in HiveOutputWriter miss l...

2018-05-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19293 **[Test build #4176 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4176/testReport)** for PR 19293 at commit

[GitHub] spark issue #21296: [SPARK-24244][SQL] Passing only required columns to the ...

2018-05-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21296 **[Test build #90555 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90555/testReport)** for PR 21296 at commit

[GitHub] spark pull request #21092: [SPARK-23984][K8S] Initial Python Bindings for Py...

2018-05-13 Thread ifilonenko
Github user ifilonenko commented on a diff in the pull request: https://github.com/apache/spark/pull/21092#discussion_r187811483 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/BasicExecutorFeatureStep.scala --- @@ -54,7 +54,8 @@

[GitHub] spark issue #21301: [SPARK-24228][SQL] Fix Java lint errors

2018-05-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21301 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3177/

[GitHub] spark pull request #21295: [SPARK-24230][SQL] Fix SpecificParquetRecordReade...

2018-05-13 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/21295#discussion_r187809171 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/SpecificParquetRecordReaderBase.java --- @@ -147,7 +147,8 @@ public

[GitHub] spark issue #21162: shaded guava is not used anywhere, seems guava is not sh...

2018-05-13 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/21162 Please close this PR. That is the rule that actual does shading of Guava, and we most definitely want to keep it. --- - To

[GitHub] spark pull request #21296: [SPARK-24244][SQL] Passing only required columns ...

2018-05-13 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/21296#discussion_r187810542 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala --- @@ -1322,4 +1322,31 @@ class CSVSuite extends

[GitHub] spark issue #21179: Update StreamingKMeans.scala

2018-05-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21179 **[Test build #4174 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4174/testReport)** for PR 21179 at commit

[GitHub] spark pull request #21313: [SPARK-24187][R][SQL]Add array_join function to S...

2018-05-13 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21313#discussion_r187810848 --- Diff: R/pkg/R/functions.R --- @@ -3006,6 +3008,28 @@ setMethod("array_contains", column(jc) }) +#'

[GitHub] spark issue #21301: [SPARK-24228][SQL] Fix Java lint errors

2018-05-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21301 **[Test build #90554 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90554/testReport)** for PR 21301 at commit

[GitHub] spark issue #21301: [SPARK-24228][SQL] Fix Java lint errors

2018-05-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21301 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90554/ Test FAILed. ---

[GitHub] spark issue #21301: [SPARK-24228][SQL] Fix Java lint errors

2018-05-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21301 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20932: [SPARK-23812][SQL] DFS should be removed from unsupporte...

2018-05-13 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/20932 It may happen to work, but I think the idea here is that we do not want to guarantee it will going forward. I think this should be closed ---

[GitHub] spark issue #20919: Feature/apply func to rdd

2018-05-13 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/20919 This should be closed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21313: [SPARK-24187][R][SQL]Add array_join function to S...

2018-05-13 Thread huaxingao
GitHub user huaxingao opened a pull request: https://github.com/apache/spark/pull/21313 [SPARK-24187][R][SQL]Add array_join function to SparkR ## What changes were proposed in this pull request? This PR adds array_join function to SparkR ## How was this patch

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-13 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r187808495 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeHelper.scala --- @@ -0,0 +1,128 @@ +/* + * Licensed to the

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-13 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r187808501 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeHelper.scala --- @@ -0,0 +1,128 @@ +/* + * Licensed to the

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-13 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r187808492 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeHelper.scala --- @@ -0,0 +1,128 @@ +/* + * Licensed to the

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-13 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r187808457 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala --- @@ -86,6 +86,13 @@ private[spark] class Client(

[GitHub] spark issue #20761: [SPARK-20327][CORE][YARN] Add CLI support for YARN custo...

2018-05-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20761 **[Test build #90553 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90553/testReport)** for PR 20761 at commit

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-13 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r187808470 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeHelper.scala --- @@ -0,0 +1,128 @@ +/* + * Licensed to the

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-13 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r187808488 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeHelper.scala --- @@ -0,0 +1,128 @@ +/* + * Licensed to the

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-13 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r187808479 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeHelper.scala --- @@ -0,0 +1,128 @@ +/* + * Licensed to the

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-13 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r187808482 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeHelper.scala --- @@ -0,0 +1,128 @@ +/* + * Licensed to the

[GitHub] spark pull request #21301: [SPARK-24228][SQL] Fix Java lint errors

2018-05-13 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21301#discussion_r187808776 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/reader/partitioning/Distribution.java --- @@ -22,7 +22,8 @@ /** * An

[GitHub] spark issue #21301: [SPARK-24228][SQL] Fix Java lint errors

2018-05-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21301 **[Test build #90554 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90554/testReport)** for PR 21301 at commit

[GitHub] spark issue #21179: Update StreamingKMeans.scala

2018-05-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21179 **[Test build #4174 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4174/testReport)** for PR 21179 at commit

[GitHub] spark issue #21162: shaded guava is not used anywhere, seems guava is not sh...

2018-05-13 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/21162 CC @vanzin but it's more complex than that as far as I know. It is still shaded. You need to read https://spark.apache.org/contributing.html ---

[GitHub] spark issue #20967: [SPARK-23857][Mesos] remove keytab check in mesos cluste...

2018-05-13 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/20967 Is this one still active? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20677: Event time can't be greater then processing time. 12:21,...

2018-05-13 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/20677 @deil87 if you're willing to just edit the PNG as best you can you can proceed that way. I don't think @tdas is active anymore ---

[GitHub] spark issue #20274: [SPARK-20120][SQL][FOLLOW-UP] Better way to support spar...

2018-05-13 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/20274 This seems fine, but, needs to be updated to merge again. I also don't see what it has to do with kubernetes jars though. I suspect there's a log4j classpath issue in play here, but this maybe

[GitHub] spark pull request #21301: [SPARK-24228][SQL] Fix Java lint errors

2018-05-13 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/21301#discussion_r187804661 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/reader/partitioning/Distribution.java --- @@ -22,7 +22,8 @@ /** * An

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-13 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r187808552 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeValidator.scala --- @@ -0,0 +1,162 @@ +/* + * Licensed to

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-13 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r187808559 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/ClientSuite.scala --- @@ -35,17 +36,23 @@ import

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-05-13 Thread szyszy
Github user szyszy commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r187808565 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/ClientSuite.scala --- @@ -199,6 +206,91 @@ class ClientSuite extends

[GitHub] spark issue #18903: [SPARK-21590][SS]Window start time should support negati...

2018-05-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18903 **[Test build #4175 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4175/testReport)** for PR 18903 at commit

[GitHub] spark issue #17843: [Streaming] groupByKey should also disable map side comb...

2018-05-13 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/17843 There's no real explanation here of why; I think this should be closed --- - To unsubscribe, e-mail:

[GitHub] spark issue #13477: [SPARK-15739][GraphX] Expose aggregateMessagesWithActive...

2018-05-13 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/13477 I doubt we'd make this a new public API in this pretty old component; I think this should be closed --- - To unsubscribe,

[GitHub] spark issue #19293: [SPARK-22079][SQL] Serializer in HiveOutputWriter miss l...

2018-05-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19293 **[Test build #4176 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4176/testReport)** for PR 19293 at commit

[GitHub] spark issue #19293: [SPARK-22079][SQL] Serializer in HiveOutputWriter miss l...

2018-05-13 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/19293 @LantaoJin can you provide any more info about the bug here? but seems reasonable --- - To unsubscribe, e-mail:

[GitHub] spark pull request #20800: [SPARK-23627][SQL] Provide isEmpty in Dataset

2018-05-13 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20800#discussion_r187804796 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -511,6 +511,14 @@ class Dataset[T] private[sql]( */ def isLocal:

[GitHub] spark issue #21067: [SPARK-23980][K8S] Resilient Spark driver on Kubernetes

2018-05-13 Thread baluchicken
Github user baluchicken commented on the issue: https://github.com/apache/spark/pull/21067 Rebased again to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21313: [SPARK-24187][R][SQL]Add array_join function to SparkR

2018-05-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21313 **[Test build #90552 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90552/testReport)** for PR 21313 at commit

  1   2   3   >