[GitHub] [spark] jerryshao commented on a change in pull request #25007: [SPARK-28209][CORE][SHUFFLE] Proposed new shuffle writer API

2019-07-16 Thread GitBox
jerryshao commented on a change in pull request #25007: [SPARK-28209][CORE][SHUFFLE] Proposed new shuffle writer API URL: https://github.com/apache/spark/pull/25007#discussion_r304211829 ## File path: core/src/main/java/org/apache/spark/shuffle/api/ShuffleMapOutputWriter.java ###

[GitHub] [spark] AmplabJenkins commented on issue #25175: [SPARK-28411][PYTHON][SQL] InsertInto with overwrite is not honored

2019-07-16 Thread GitBox
AmplabJenkins commented on issue #25175: [SPARK-28411][PYTHON][SQL] InsertInto with overwrite is not honored URL: https://github.com/apache/spark/pull/25175#issuecomment-512093242 Merged build finished. Test PASSed. This is a

[GitHub] [spark] AmplabJenkins commented on issue #25175: [SPARK-28411][PYTHON][SQL] InsertInto with overwrite is not honored

2019-07-16 Thread GitBox
AmplabJenkins commented on issue #25175: [SPARK-28411][PYTHON][SQL] InsertInto with overwrite is not honored URL: https://github.com/apache/spark/pull/25175#issuecomment-512093246 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.c

[GitHub] [spark] AmplabJenkins removed a comment on issue #25175: [SPARK-28411][PYTHON][SQL] InsertInto with overwrite is not honored

2019-07-16 Thread GitBox
AmplabJenkins removed a comment on issue #25175: [SPARK-28411][PYTHON][SQL] InsertInto with overwrite is not honored URL: https://github.com/apache/spark/pull/25175#issuecomment-512093246 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://

[GitHub] [spark] AmplabJenkins removed a comment on issue #25175: [SPARK-28411][PYTHON][SQL] InsertInto with overwrite is not honored

2019-07-16 Thread GitBox
AmplabJenkins removed a comment on issue #25175: [SPARK-28411][PYTHON][SQL] InsertInto with overwrite is not honored URL: https://github.com/apache/spark/pull/25175#issuecomment-512093242 Merged build finished. Test PASSed. T

[GitHub] [spark] HyukjinKwon commented on a change in pull request #25175: [SPARK-28411][PYTHON][SQL] InsertInto with overwrite is not honored

2019-07-16 Thread GitBox
HyukjinKwon commented on a change in pull request #25175: [SPARK-28411][PYTHON][SQL] InsertInto with overwrite is not honored URL: https://github.com/apache/spark/pull/25175#discussion_r304212498 ## File path: python/pyspark/sql/tests/test_readwriter.py ## @@ -141,6 +141,2

[GitHub] [spark] maropu commented on issue #25172: [SPARK-28412][SQL] ANSI SQL: OVERLAY function support byte array

2019-07-16 Thread GitBox
maropu commented on issue #25172: [SPARK-28412][SQL] ANSI SQL: OVERLAY function support byte array URL: https://github.com/apache/spark/pull/25172#issuecomment-512094138 ok, thanks! I'll review later. This is an automated mes

[GitHub] [spark] AmplabJenkins commented on issue #25172: [SPARK-28412][SQL] ANSI SQL: OVERLAY function support byte array

2019-07-16 Thread GitBox
AmplabJenkins commented on issue #25172: [SPARK-28412][SQL] ANSI SQL: OVERLAY function support byte array URL: https://github.com/apache/spark/pull/25172#issuecomment-512094521 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.b

[GitHub] [spark] AmplabJenkins commented on issue #25130: [SPARK-28359][SQL][PYTHON][TESTS] Make integrated UDF tests robust by making UDFs (virtually) no-op

2019-07-16 Thread GitBox
AmplabJenkins commented on issue #25130: [SPARK-28359][SQL][PYTHON][TESTS] Make integrated UDF tests robust by making UDFs (virtually) no-op URL: https://github.com/apache/spark/pull/25130#issuecomment-512094487 Merged build finished. Test PASSed. --

[GitHub] [spark] AmplabJenkins commented on issue #25130: [SPARK-28359][SQL][PYTHON][TESTS] Make integrated UDF tests robust by making UDFs (virtually) no-op

2019-07-16 Thread GitBox
AmplabJenkins commented on issue #25130: [SPARK-28359][SQL][PYTHON][TESTS] Make integrated UDF tests robust by making UDFs (virtually) no-op URL: https://github.com/apache/spark/pull/25130#issuecomment-512094490 Test PASSed. Refer to this link for build results (access rights to CI serve

[GitHub] [spark] AmplabJenkins commented on issue #25172: [SPARK-28412][SQL] ANSI SQL: OVERLAY function support byte array

2019-07-16 Thread GitBox
AmplabJenkins commented on issue #25172: [SPARK-28412][SQL] ANSI SQL: OVERLAY function support byte array URL: https://github.com/apache/spark/pull/25172#issuecomment-512094517 Merged build finished. Test PASSed. This is an a

[GitHub] [spark] AmplabJenkins removed a comment on issue #25172: [SPARK-28412][SQL] ANSI SQL: OVERLAY function support byte array

2019-07-16 Thread GitBox
AmplabJenkins removed a comment on issue #25172: [SPARK-28412][SQL] ANSI SQL: OVERLAY function support byte array URL: https://github.com/apache/spark/pull/25172#issuecomment-512094517 Merged build finished. Test PASSed. This

[GitHub] [spark] AmplabJenkins removed a comment on issue #25130: [SPARK-28359][SQL][PYTHON][TESTS] Make integrated UDF tests robust by making UDFs (virtually) no-op

2019-07-16 Thread GitBox
AmplabJenkins removed a comment on issue #25130: [SPARK-28359][SQL][PYTHON][TESTS] Make integrated UDF tests robust by making UDFs (virtually) no-op URL: https://github.com/apache/spark/pull/25130#issuecomment-512094490 Test PASSed. Refer to this link for build results (access rights to

[GitHub] [spark] AmplabJenkins removed a comment on issue #25172: [SPARK-28412][SQL] ANSI SQL: OVERLAY function support byte array

2019-07-16 Thread GitBox
AmplabJenkins removed a comment on issue #25172: [SPARK-28412][SQL] ANSI SQL: OVERLAY function support byte array URL: https://github.com/apache/spark/pull/25172#issuecomment-512094521 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amp

[GitHub] [spark] AmplabJenkins removed a comment on issue #25130: [SPARK-28359][SQL][PYTHON][TESTS] Make integrated UDF tests robust by making UDFs (virtually) no-op

2019-07-16 Thread GitBox
AmplabJenkins removed a comment on issue #25130: [SPARK-28359][SQL][PYTHON][TESTS] Make integrated UDF tests robust by making UDFs (virtually) no-op URL: https://github.com/apache/spark/pull/25130#issuecomment-512094487 Merged build finished. Test PASSed. -

[GitHub] [spark] SparkQA commented on issue #25172: [SPARK-28412][SQL] ANSI SQL: OVERLAY function support byte array

2019-07-16 Thread GitBox
SparkQA commented on issue #25172: [SPARK-28412][SQL] ANSI SQL: OVERLAY function support byte array URL: https://github.com/apache/spark/pull/25172#issuecomment-512094841 **[Test build #107772 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/107772/testReport

[GitHub] [spark] SparkQA commented on issue #25130: [SPARK-28359][SQL][PYTHON][TESTS] Make integrated UDF tests robust by making UDFs (virtually) no-op

2019-07-16 Thread GitBox
SparkQA commented on issue #25130: [SPARK-28359][SQL][PYTHON][TESTS] Make integrated UDF tests robust by making UDFs (virtually) no-op URL: https://github.com/apache/spark/pull/25130#issuecomment-512094840 **[Test build #107773 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkP

[GitHub] [spark] beliefer commented on a change in pull request #25074: [SPARK-27924][SQL] Support ANSI SQL Boolean-Predicate syntax

2019-07-16 Thread GitBox
beliefer commented on a change in pull request #25074: [SPARK-27924][SQL] Support ANSI SQL Boolean-Predicate syntax URL: https://github.com/apache/spark/pull/25074#discussion_r304213725 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/booleanExpr

[GitHub] [spark] dilipbiswal commented on a change in pull request #25164: [SPARK-28375][SQL] Prevent the PullupCorrelatedPredicates optimizer rule from removing predicates if run multiple times

2019-07-16 Thread GitBox
dilipbiswal commented on a change in pull request #25164: [SPARK-28375][SQL] Prevent the PullupCorrelatedPredicates optimizer rule from removing predicates if run multiple times URL: https://github.com/apache/spark/pull/25164#discussion_r304214534 ## File path: sql/catalyst/src/ma

[GitHub] [spark] beliefer commented on a change in pull request #25074: [SPARK-27924][SQL] Support ANSI SQL Boolean-Predicate syntax

2019-07-16 Thread GitBox
beliefer commented on a change in pull request #25074: [SPARK-27924][SQL] Support ANSI SQL Boolean-Predicate syntax URL: https://github.com/apache/spark/pull/25074#discussion_r304214658 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/booleanExpr

[GitHub] [spark] SparkQA commented on issue #25175: [SPARK-28411][PYTHON][SQL] InsertInto with overwrite is not honored

2019-07-16 Thread GitBox
SparkQA commented on issue #25175: [SPARK-28411][PYTHON][SQL] InsertInto with overwrite is not honored URL: https://github.com/apache/spark/pull/25175#issuecomment-512096806 **[Test build #107771 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/107771/testRe

[GitHub] [spark] AmplabJenkins commented on issue #25175: [SPARK-28411][PYTHON][SQL] InsertInto with overwrite is not honored

2019-07-16 Thread GitBox
AmplabJenkins commented on issue #25175: [SPARK-28411][PYTHON][SQL] InsertInto with overwrite is not honored URL: https://github.com/apache/spark/pull/25175#issuecomment-512096981 Merged build finished. Test PASSed. This is a

[GitHub] [spark] AmplabJenkins commented on issue #25175: [SPARK-28411][PYTHON][SQL] InsertInto with overwrite is not honored

2019-07-16 Thread GitBox
AmplabJenkins commented on issue #25175: [SPARK-28411][PYTHON][SQL] InsertInto with overwrite is not honored URL: https://github.com/apache/spark/pull/25175#issuecomment-512096985 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs

[GitHub] [spark] beliefer commented on a change in pull request #25074: [SPARK-27924][SQL] Support ANSI SQL Boolean-Predicate syntax

2019-07-16 Thread GitBox
beliefer commented on a change in pull request #25074: [SPARK-27924][SQL] Support ANSI SQL Boolean-Predicate syntax URL: https://github.com/apache/spark/pull/25074#discussion_r304214922 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/booleanExpr

[GitHub] [spark] beliefer commented on a change in pull request #25074: [SPARK-27924][SQL] Support ANSI SQL Boolean-Predicate syntax

2019-07-16 Thread GitBox
beliefer commented on a change in pull request #25074: [SPARK-27924][SQL] Support ANSI SQL Boolean-Predicate syntax URL: https://github.com/apache/spark/pull/25074#discussion_r304214964 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala

[GitHub] [spark] SparkQA removed a comment on issue #25175: [SPARK-28411][PYTHON][SQL] InsertInto with overwrite is not honored

2019-07-16 Thread GitBox
SparkQA removed a comment on issue #25175: [SPARK-28411][PYTHON][SQL] InsertInto with overwrite is not honored URL: https://github.com/apache/spark/pull/25175#issuecomment-512092300 **[Test build #107771 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/107771

[GitHub] [spark] AmplabJenkins removed a comment on issue #25175: [SPARK-28411][PYTHON][SQL] InsertInto with overwrite is not honored

2019-07-16 Thread GitBox
AmplabJenkins removed a comment on issue #25175: [SPARK-28411][PYTHON][SQL] InsertInto with overwrite is not honored URL: https://github.com/apache/spark/pull/25175#issuecomment-512096985 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://a

[GitHub] [spark] AmplabJenkins removed a comment on issue #25175: [SPARK-28411][PYTHON][SQL] InsertInto with overwrite is not honored

2019-07-16 Thread GitBox
AmplabJenkins removed a comment on issue #25175: [SPARK-28411][PYTHON][SQL] InsertInto with overwrite is not honored URL: https://github.com/apache/spark/pull/25175#issuecomment-512096981 Merged build finished. Test PASSed. T

[GitHub] [spark] SparkQA commented on issue #25164: [SPARK-28375][SQL] Prevent the PullupCorrelatedPredicates optimizer rule from removing predicates if run multiple times

2019-07-16 Thread GitBox
SparkQA commented on issue #25164: [SPARK-28375][SQL] Prevent the PullupCorrelatedPredicates optimizer rule from removing predicates if run multiple times URL: https://github.com/apache/spark/pull/25164#issuecomment-512097138 **[Test build #107767 has finished](https://amplab.cs.berkeley.

[GitHub] [spark] SparkQA removed a comment on issue #25164: [SPARK-28375][SQL] Prevent the PullupCorrelatedPredicates optimizer rule from removing predicates if run multiple times

2019-07-16 Thread GitBox
SparkQA removed a comment on issue #25164: [SPARK-28375][SQL] Prevent the PullupCorrelatedPredicates optimizer rule from removing predicates if run multiple times URL: https://github.com/apache/spark/pull/25164#issuecomment-512061818 **[Test build #107767 has started](https://amplab.cs.be

[GitHub] [spark] SparkQA removed a comment on issue #25160: [SPARK-28399][ML] implement RobustScaler

2019-07-16 Thread GitBox
SparkQA removed a comment on issue #25160: [SPARK-28399][ML] implement RobustScaler URL: https://github.com/apache/spark/pull/25160#issuecomment-512086889 **[Test build #107770 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/107770/testReport)** for PR 2516

[GitHub] [spark] SparkQA commented on issue #25160: [SPARK-28399][ML] implement RobustScaler

2019-07-16 Thread GitBox
SparkQA commented on issue #25160: [SPARK-28399][ML] implement RobustScaler URL: https://github.com/apache/spark/pull/25160#issuecomment-512097276 **[Test build #107770 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/107770/testReport)** for PR 25160 at com

[GitHub] [spark] AmplabJenkins removed a comment on issue #25160: [SPARK-28399][ML] implement RobustScaler

2019-07-16 Thread GitBox
AmplabJenkins removed a comment on issue #25160: [SPARK-28399][ML] implement RobustScaler URL: https://github.com/apache/spark/pull/25160#issuecomment-512097465 Merged build finished. Test PASSed. This is an automated message

[GitHub] [spark] AmplabJenkins commented on issue #25164: [SPARK-28375][SQL] Prevent the PullupCorrelatedPredicates optimizer rule from removing predicates if run multiple times

2019-07-16 Thread GitBox
AmplabJenkins commented on issue #25164: [SPARK-28375][SQL] Prevent the PullupCorrelatedPredicates optimizer rule from removing predicates if run multiple times URL: https://github.com/apache/spark/pull/25164#issuecomment-512097488 Test PASSed. Refer to this link for build results (acce

[GitHub] [spark] AmplabJenkins commented on issue #25164: [SPARK-28375][SQL] Prevent the PullupCorrelatedPredicates optimizer rule from removing predicates if run multiple times

2019-07-16 Thread GitBox
AmplabJenkins commented on issue #25164: [SPARK-28375][SQL] Prevent the PullupCorrelatedPredicates optimizer rule from removing predicates if run multiple times URL: https://github.com/apache/spark/pull/25164#issuecomment-512097484 Merged build finished. Test PASSed. -

[GitHub] [spark] AmplabJenkins commented on issue #25160: [SPARK-28399][ML] implement RobustScaler

2019-07-16 Thread GitBox
AmplabJenkins commented on issue #25160: [SPARK-28399][ML] implement RobustScaler URL: https://github.com/apache/spark/pull/25160#issuecomment-512097471 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/

[GitHub] [spark] AmplabJenkins commented on issue #25160: [SPARK-28399][ML] implement RobustScaler

2019-07-16 Thread GitBox
AmplabJenkins commented on issue #25160: [SPARK-28399][ML] implement RobustScaler URL: https://github.com/apache/spark/pull/25160#issuecomment-512097465 Merged build finished. Test PASSed. This is an automated message from th

[GitHub] [spark] AmplabJenkins removed a comment on issue #25160: [SPARK-28399][ML] implement RobustScaler

2019-07-16 Thread GitBox
AmplabJenkins removed a comment on issue #25160: [SPARK-28399][ML] implement RobustScaler URL: https://github.com/apache/spark/pull/25160#issuecomment-512097471 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenki

[GitHub] [spark] AmplabJenkins removed a comment on issue #25164: [SPARK-28375][SQL] Prevent the PullupCorrelatedPredicates optimizer rule from removing predicates if run multiple times

2019-07-16 Thread GitBox
AmplabJenkins removed a comment on issue #25164: [SPARK-28375][SQL] Prevent the PullupCorrelatedPredicates optimizer rule from removing predicates if run multiple times URL: https://github.com/apache/spark/pull/25164#issuecomment-512097488 Test PASSed. Refer to this link for build resul

[GitHub] [spark] AmplabJenkins removed a comment on issue #25164: [SPARK-28375][SQL] Prevent the PullupCorrelatedPredicates optimizer rule from removing predicates if run multiple times

2019-07-16 Thread GitBox
AmplabJenkins removed a comment on issue #25164: [SPARK-28375][SQL] Prevent the PullupCorrelatedPredicates optimizer rule from removing predicates if run multiple times URL: https://github.com/apache/spark/pull/25164#issuecomment-512097484 Merged build finished. Test PASSed. -

[GitHub] [spark] yeshengm commented on a change in pull request #25164: [SPARK-28375][SQL] Prevent the PullupCorrelatedPredicates optimizer rule from removing predicates if run multiple times

2019-07-16 Thread GitBox
yeshengm commented on a change in pull request #25164: [SPARK-28375][SQL] Prevent the PullupCorrelatedPredicates optimizer rule from removing predicates if run multiple times URL: https://github.com/apache/spark/pull/25164#discussion_r304215574 ## File path: sql/catalyst/src/main/

[GitHub] [spark] beliefer commented on a change in pull request #25074: [SPARK-27924][SQL] Support ANSI SQL Boolean-Predicate syntax

2019-07-16 Thread GitBox
beliefer commented on a change in pull request #25074: [SPARK-27924][SQL] Support ANSI SQL Boolean-Predicate syntax URL: https://github.com/apache/spark/pull/25074#discussion_r304215899 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/BooleanExpr

[GitHub] [spark] dilipbiswal commented on a change in pull request #25164: [SPARK-28375][SQL] Prevent the PullupCorrelatedPredicates optimizer rule from removing predicates if run multiple times

2019-07-16 Thread GitBox
dilipbiswal commented on a change in pull request #25164: [SPARK-28375][SQL] Prevent the PullupCorrelatedPredicates optimizer rule from removing predicates if run multiple times URL: https://github.com/apache/spark/pull/25164#discussion_r304217306 ## File path: sql/catalyst/src/ma

[GitHub] [spark] AmplabJenkins commented on issue #25175: [SPARK-28411][PYTHON][SQL] InsertInto with overwrite is not honored

2019-07-16 Thread GitBox
AmplabJenkins commented on issue #25175: [SPARK-28411][PYTHON][SQL] InsertInto with overwrite is not honored URL: https://github.com/apache/spark/pull/25175#issuecomment-512101460 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.c

[GitHub] [spark] Udbhav30 commented on issue #24601: [SPARK-27702][K8S] Allow using some alternatives for service accounts

2019-07-16 Thread GitBox
Udbhav30 commented on issue #24601: [SPARK-27702][K8S] Allow using some alternatives for service accounts URL: https://github.com/apache/spark/pull/24601#issuecomment-512101459 Hii @dongjoon-hyun can you please check if we can merge this. ---

[GitHub] [spark] AmplabJenkins removed a comment on issue #25175: [SPARK-28411][PYTHON][SQL] InsertInto with overwrite is not honored

2019-07-16 Thread GitBox
AmplabJenkins removed a comment on issue #25175: [SPARK-28411][PYTHON][SQL] InsertInto with overwrite is not honored URL: https://github.com/apache/spark/pull/25175#issuecomment-512101460 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://

[GitHub] [spark] AmplabJenkins commented on issue #25175: [SPARK-28411][PYTHON][SQL] InsertInto with overwrite is not honored

2019-07-16 Thread GitBox
AmplabJenkins commented on issue #25175: [SPARK-28411][PYTHON][SQL] InsertInto with overwrite is not honored URL: https://github.com/apache/spark/pull/25175#issuecomment-512101452 Merged build finished. Test PASSed. This is a

[GitHub] [spark] AmplabJenkins removed a comment on issue #25175: [SPARK-28411][PYTHON][SQL] InsertInto with overwrite is not honored

2019-07-16 Thread GitBox
AmplabJenkins removed a comment on issue #25175: [SPARK-28411][PYTHON][SQL] InsertInto with overwrite is not honored URL: https://github.com/apache/spark/pull/25175#issuecomment-512101452 Merged build finished. Test PASSed. T

[GitHub] [spark] SparkQA commented on issue #25175: [SPARK-28411][PYTHON][SQL] InsertInto with overwrite is not honored

2019-07-16 Thread GitBox
SparkQA commented on issue #25175: [SPARK-28411][PYTHON][SQL] InsertInto with overwrite is not honored URL: https://github.com/apache/spark/pull/25175#issuecomment-512101822 **[Test build #107774 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/107774/testRep

[GitHub] [spark] yeshengm commented on a change in pull request #25164: [SPARK-28375][SQL] Prevent the PullupCorrelatedPredicates optimizer rule from removing predicates if run multiple times

2019-07-16 Thread GitBox
yeshengm commented on a change in pull request #25164: [SPARK-28375][SQL] Prevent the PullupCorrelatedPredicates optimizer rule from removing predicates if run multiple times URL: https://github.com/apache/spark/pull/25164#discussion_r304219902 ## File path: sql/catalyst/src/main/

[GitHub] [spark] wypoon commented on a change in pull request #23767: [SPARK-26329][CORE] Faster polling of executor memory metrics.

2019-07-16 Thread GitBox
wypoon commented on a change in pull request #23767: [SPARK-26329][CORE] Faster polling of executor memory metrics. URL: https://github.com/apache/spark/pull/23767#discussion_r304222033 ## File path: core/src/main/scala/org/apache/spark/executor/ExecutorMetricsPoller.scala ###

[GitHub] [spark] wypoon commented on a change in pull request #23767: [SPARK-26329][CORE] Faster polling of executor memory metrics.

2019-07-16 Thread GitBox
wypoon commented on a change in pull request #23767: [SPARK-26329][CORE] Faster polling of executor memory metrics. URL: https://github.com/apache/spark/pull/23767#discussion_r304222403 ## File path: core/src/main/scala/org/apache/spark/executor/ExecutorMetricsPoller.scala ###

[GitHub] [spark] wypoon commented on a change in pull request #23767: [SPARK-26329][CORE] Faster polling of executor memory metrics.

2019-07-16 Thread GitBox
wypoon commented on a change in pull request #23767: [SPARK-26329][CORE] Faster polling of executor memory metrics. URL: https://github.com/apache/spark/pull/23767#discussion_r304222908 ## File path: core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala ###

[GitHub] [spark] wypoon commented on a change in pull request #23767: [SPARK-26329][CORE] Faster polling of executor memory metrics.

2019-07-16 Thread GitBox
wypoon commented on a change in pull request #23767: [SPARK-26329][CORE] Faster polling of executor memory metrics. URL: https://github.com/apache/spark/pull/23767#discussion_r304223766 ## File path: core/src/test/scala/org/apache/spark/util/JsonProtocolSuite.scala ## @@ -

[GitHub] [spark] SparkQA commented on issue #25175: [SPARK-28411][PYTHON][SQL] InsertInto with overwrite is not honored

2019-07-16 Thread GitBox
SparkQA commented on issue #25175: [SPARK-28411][PYTHON][SQL] InsertInto with overwrite is not honored URL: https://github.com/apache/spark/pull/25175#issuecomment-512106810 **[Test build #107774 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/107774/testRe

[GitHub] [spark] SparkQA removed a comment on issue #25175: [SPARK-28411][PYTHON][SQL] InsertInto with overwrite is not honored

2019-07-16 Thread GitBox
SparkQA removed a comment on issue #25175: [SPARK-28411][PYTHON][SQL] InsertInto with overwrite is not honored URL: https://github.com/apache/spark/pull/25175#issuecomment-512101822 **[Test build #107774 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/107774

[GitHub] [spark] AmplabJenkins commented on issue #25175: [SPARK-28411][PYTHON][SQL] InsertInto with overwrite is not honored

2019-07-16 Thread GitBox
AmplabJenkins commented on issue #25175: [SPARK-28411][PYTHON][SQL] InsertInto with overwrite is not honored URL: https://github.com/apache/spark/pull/25175#issuecomment-512107019 Merged build finished. Test PASSed. This is a

[GitHub] [spark] AmplabJenkins removed a comment on issue #25175: [SPARK-28411][PYTHON][SQL] InsertInto with overwrite is not honored

2019-07-16 Thread GitBox
AmplabJenkins removed a comment on issue #25175: [SPARK-28411][PYTHON][SQL] InsertInto with overwrite is not honored URL: https://github.com/apache/spark/pull/25175#issuecomment-512107019 Merged build finished. Test PASSed. T

[GitHub] [spark] AmplabJenkins commented on issue #25175: [SPARK-28411][PYTHON][SQL] InsertInto with overwrite is not honored

2019-07-16 Thread GitBox
AmplabJenkins commented on issue #25175: [SPARK-28411][PYTHON][SQL] InsertInto with overwrite is not honored URL: https://github.com/apache/spark/pull/25175#issuecomment-512107022 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs

[GitHub] [spark] AmplabJenkins removed a comment on issue #25175: [SPARK-28411][PYTHON][SQL] InsertInto with overwrite is not honored

2019-07-16 Thread GitBox
AmplabJenkins removed a comment on issue #25175: [SPARK-28411][PYTHON][SQL] InsertInto with overwrite is not honored URL: https://github.com/apache/spark/pull/25175#issuecomment-512107022 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://a

[GitHub] [spark] SparkQA commented on issue #24903: [SPARK-28084][SQL] Resolving the partition column name based on the resolver in sql load command

2019-07-16 Thread GitBox
SparkQA commented on issue #24903: [SPARK-28084][SQL] Resolving the partition column name based on the resolver in sql load command URL: https://github.com/apache/spark/pull/24903#issuecomment-51279 **[Test build #107769 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPu

[GitHub] [spark] SparkQA removed a comment on issue #24903: [SPARK-28084][SQL] Resolving the partition column name based on the resolver in sql load command

2019-07-16 Thread GitBox
SparkQA removed a comment on issue #24903: [SPARK-28084][SQL] Resolving the partition column name based on the resolver in sql load command URL: https://github.com/apache/spark/pull/24903#issuecomment-512077360 **[Test build #107769 has started](https://amplab.cs.berkeley.edu/jenkins/job/

[GitHub] [spark] AmplabJenkins commented on issue #24903: [SPARK-28084][SQL] Resolving the partition column name based on the resolver in sql load command

2019-07-16 Thread GitBox
AmplabJenkins commented on issue #24903: [SPARK-28084][SQL] Resolving the partition column name based on the resolver in sql load command URL: https://github.com/apache/spark/pull/24903#issuecomment-512111605 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins commented on issue #24903: [SPARK-28084][SQL] Resolving the partition column name based on the resolver in sql load command

2019-07-16 Thread GitBox
AmplabJenkins commented on issue #24903: [SPARK-28084][SQL] Resolving the partition column name based on the resolver in sql load command URL: https://github.com/apache/spark/pull/24903#issuecomment-512111609 Test PASSed. Refer to this link for build results (access rights to CI server

[GitHub] [spark] AmplabJenkins removed a comment on issue #24903: [SPARK-28084][SQL] Resolving the partition column name based on the resolver in sql load command

2019-07-16 Thread GitBox
AmplabJenkins removed a comment on issue #24903: [SPARK-28084][SQL] Resolving the partition column name based on the resolver in sql load command URL: https://github.com/apache/spark/pull/24903#issuecomment-512111605 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins removed a comment on issue #24903: [SPARK-28084][SQL] Resolving the partition column name based on the resolver in sql load command

2019-07-16 Thread GitBox
AmplabJenkins removed a comment on issue #24903: [SPARK-28084][SQL] Resolving the partition column name based on the resolver in sql load command URL: https://github.com/apache/spark/pull/24903#issuecomment-512111609 Test PASSed. Refer to this link for build results (access rights to CI

[GitHub] [spark] gczsjdy commented on a change in pull request #25007: [SPARK-28209][CORE][SHUFFLE] Proposed new shuffle writer API

2019-07-16 Thread GitBox
gczsjdy commented on a change in pull request #25007: [SPARK-28209][CORE][SHUFFLE] Proposed new shuffle writer API URL: https://github.com/apache/spark/pull/25007#discussion_r304193877 ## File path: core/src/main/java/org/apache/spark/shuffle/api/ShuffleMapOutputWriter.java #

[GitHub] [spark] jerryshao commented on a change in pull request #25007: [SPARK-28209][CORE][SHUFFLE] Proposed new shuffle writer API

2019-07-16 Thread GitBox
jerryshao commented on a change in pull request #25007: [SPARK-28209][CORE][SHUFFLE] Proposed new shuffle writer API URL: https://github.com/apache/spark/pull/25007#discussion_r304230396 ## File path: core/src/main/java/org/apache/spark/shuffle/api/ShuffleExecutorComponents.java

[GitHub] [spark] ueshin commented on issue #25175: [SPARK-28411][PYTHON][SQL] InsertInto with overwrite is not honored

2019-07-16 Thread GitBox
ueshin commented on issue #25175: [SPARK-28411][PYTHON][SQL] InsertInto with overwrite is not honored URL: https://github.com/apache/spark/pull/25175#issuecomment-512115296 I'm wondering what if a user explicitly specifies `overwrite=False`? E.g., ```py df.write.mode("overwrite

[GitHub] [spark] yaooqinn commented on issue #24947: [SPARK-28143][SQL] Expressions without proper constructors should throw AnalysisException

2019-07-16 Thread GitBox
yaooqinn commented on issue #24947: [SPARK-28143][SQL] Expressions without proper constructors should throw AnalysisException URL: https://github.com/apache/spark/pull/24947#issuecomment-512117564 @srowen I have updated the pr description and title as last comment required. If anything I m

[GitHub] [spark] beliefer commented on a change in pull request #25074: [SPARK-27924][SQL] Support ANSI SQL Boolean-Predicate syntax

2019-07-16 Thread GitBox
beliefer commented on a change in pull request #25074: [SPARK-27924][SQL] Support ANSI SQL Boolean-Predicate syntax URL: https://github.com/apache/spark/pull/25074#discussion_r304235237 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/BooleanExpr

[GitHub] [spark] beliefer commented on a change in pull request #25074: [SPARK-27924][SQL] Support ANSI SQL Boolean-Predicate syntax

2019-07-16 Thread GitBox
beliefer commented on a change in pull request #25074: [SPARK-27924][SQL] Support ANSI SQL Boolean-Predicate syntax URL: https://github.com/apache/spark/pull/25074#discussion_r304235534 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/BooleanExpr

[GitHub] [spark] beliefer commented on a change in pull request #25074: [SPARK-27924][SQL] Support ANSI SQL Boolean-Predicate syntax

2019-07-16 Thread GitBox
beliefer commented on a change in pull request #25074: [SPARK-27924][SQL] Support ANSI SQL Boolean-Predicate syntax URL: https://github.com/apache/spark/pull/25074#discussion_r304235680 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/BooleanExpr

[GitHub] [spark] beliefer commented on a change in pull request #25074: [SPARK-27924][SQL] Support ANSI SQL Boolean-Predicate syntax

2019-07-16 Thread GitBox
beliefer commented on a change in pull request #25074: [SPARK-27924][SQL] Support ANSI SQL Boolean-Predicate syntax URL: https://github.com/apache/spark/pull/25074#discussion_r304235680 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/BooleanExpr

[GitHub] [spark] jerryshao commented on a change in pull request #25007: [SPARK-28209][CORE][SHUFFLE] Proposed new shuffle writer API

2019-07-16 Thread GitBox
jerryshao commented on a change in pull request #25007: [SPARK-28209][CORE][SHUFFLE] Proposed new shuffle writer API URL: https://github.com/apache/spark/pull/25007#discussion_r304237265 ## File path: core/src/main/scala/org/apache/spark/shuffle/sort/SortShuffleManager.scala

[GitHub] [spark] HyukjinKwon commented on issue #25175: [SPARK-28411][PYTHON][SQL] InsertInto with overwrite is not honored

2019-07-16 Thread GitBox
HyukjinKwon commented on issue #25175: [SPARK-28411][PYTHON][SQL] InsertInto with overwrite is not honored URL: https://github.com/apache/spark/pull/25175#issuecomment-512122461 Yup, that's fine with me too. This is an automa

[GitHub] [spark] beliefer commented on a change in pull request #25074: [SPARK-27924][SQL] Support ANSI SQL Boolean-Predicate syntax

2019-07-16 Thread GitBox
beliefer commented on a change in pull request #25074: [SPARK-27924][SQL] Support ANSI SQL Boolean-Predicate syntax URL: https://github.com/apache/spark/pull/25074#discussion_r304214964 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala

[GitHub] [spark] AmplabJenkins removed a comment on issue #25176: [SPARK-28417][CORE] Wrap File Glob Resolution in DoAs to use ProxyUser Credentials

2019-07-16 Thread GitBox
AmplabJenkins removed a comment on issue #25176: [SPARK-28417][CORE] Wrap File Glob Resolution in DoAs to use ProxyUser Credentials URL: https://github.com/apache/spark/pull/25176#issuecomment-512006177 Can one of the admins verify this patch? --

[GitHub] [spark] felixcheung commented on issue #25176: [SPARK-28417][CORE] Wrap File Glob Resolution in DoAs to use ProxyUser Credentials

2019-07-16 Thread GitBox
felixcheung commented on issue #25176: [SPARK-28417][CORE] Wrap File Glob Resolution in DoAs to use ProxyUser Credentials URL: https://github.com/apache/spark/pull/25176#issuecomment-512123665 Jenkins, ok to test This is an a

[GitHub] [spark] AmplabJenkins commented on issue #25176: [SPARK-28417][CORE] Wrap File Glob Resolution in DoAs to use ProxyUser Credentials

2019-07-16 Thread GitBox
AmplabJenkins commented on issue #25176: [SPARK-28417][CORE] Wrap File Glob Resolution in DoAs to use ProxyUser Credentials URL: https://github.com/apache/spark/pull/25176#issuecomment-512124081 Merged build finished. Test PASSed. ---

[GitHub] [spark] AmplabJenkins commented on issue #25176: [SPARK-28417][CORE] Wrap File Glob Resolution in DoAs to use ProxyUser Credentials

2019-07-16 Thread GitBox
AmplabJenkins commented on issue #25176: [SPARK-28417][CORE] Wrap File Glob Resolution in DoAs to use ProxyUser Credentials URL: https://github.com/apache/spark/pull/25176#issuecomment-512124087 Test PASSed. Refer to this link for build results (access rights to CI server needed): h

[GitHub] [spark] AmplabJenkins removed a comment on issue #25176: [SPARK-28417][CORE] Wrap File Glob Resolution in DoAs to use ProxyUser Credentials

2019-07-16 Thread GitBox
AmplabJenkins removed a comment on issue #25176: [SPARK-28417][CORE] Wrap File Glob Resolution in DoAs to use ProxyUser Credentials URL: https://github.com/apache/spark/pull/25176#issuecomment-512124087 Test PASSed. Refer to this link for build results (access rights to CI server needed)

[GitHub] [spark] AmplabJenkins removed a comment on issue #25176: [SPARK-28417][CORE] Wrap File Glob Resolution in DoAs to use ProxyUser Credentials

2019-07-16 Thread GitBox
AmplabJenkins removed a comment on issue #25176: [SPARK-28417][CORE] Wrap File Glob Resolution in DoAs to use ProxyUser Credentials URL: https://github.com/apache/spark/pull/25176#issuecomment-512124081 Merged build finished. Test PASSed. ---

[GitHub] [spark] SparkQA commented on issue #25176: [SPARK-28417][CORE] Wrap File Glob Resolution in DoAs to use ProxyUser Credentials

2019-07-16 Thread GitBox
SparkQA commented on issue #25176: [SPARK-28417][CORE] Wrap File Glob Resolution in DoAs to use ProxyUser Credentials URL: https://github.com/apache/spark/pull/25176#issuecomment-512124636 **[Test build #107775 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder

[GitHub] [spark] felixcheung commented on a change in pull request #24939: [SPARK-18569][ML][R] Support RFormula arithmetic, I() and spark functions

2019-07-16 Thread GitBox
felixcheung commented on a change in pull request #24939: [SPARK-18569][ML][R] Support RFormula arithmetic, I() and spark functions URL: https://github.com/apache/spark/pull/24939#discussion_r304240770 ## File path: mllib/src/main/scala/org/apache/spark/ml/feature/RFormula.scala ##

[GitHub] [spark] HyukjinKwon opened a new pull request #25177: [SPARK-28418][PYTHON][SQL] Wait for event process in 'test_query_execution_listener_on_collect'

2019-07-16 Thread GitBox
HyukjinKwon opened a new pull request #25177: [SPARK-28418][PYTHON][SQL] Wait for event process in 'test_query_execution_listener_on_collect' URL: https://github.com/apache/spark/pull/25177 ## What changes were proposed in this pull request? It fixes a flaky test: ``` ERROR

[GitHub] [spark] SparkQA commented on issue #25177: [SPARK-28418][PYTHON][SQL] Wait for event process in 'test_query_execution_listener_on_collect'

2019-07-16 Thread GitBox
SparkQA commented on issue #25177: [SPARK-28418][PYTHON][SQL] Wait for event process in 'test_query_execution_listener_on_collect' URL: https://github.com/apache/spark/pull/25177#issuecomment-512126752 **[Test build #107776 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullR

[GitHub] [spark] AmplabJenkins removed a comment on issue #25177: [SPARK-28418][PYTHON][SQL] Wait for event process in 'test_query_execution_listener_on_collect'

2019-07-16 Thread GitBox
AmplabJenkins removed a comment on issue #25177: [SPARK-28418][PYTHON][SQL] Wait for event process in 'test_query_execution_listener_on_collect' URL: https://github.com/apache/spark/pull/25177#issuecomment-512126144 Test PASSed. Refer to this link for build results (access rights to CI s

[GitHub] [spark] AmplabJenkins removed a comment on issue #25177: [SPARK-28418][PYTHON][SQL] Wait for event process in 'test_query_execution_listener_on_collect'

2019-07-16 Thread GitBox
AmplabJenkins removed a comment on issue #25177: [SPARK-28418][PYTHON][SQL] Wait for event process in 'test_query_execution_listener_on_collect' URL: https://github.com/apache/spark/pull/25177#issuecomment-512126140 Merged build finished. Test PASSed. --

[GitHub] [spark] felixcheung commented on issue #24796: [SPARK-27900][CORE] Add uncaught exception handler to the driver

2019-07-16 Thread GitBox
felixcheung commented on issue #24796: [SPARK-27900][CORE] Add uncaught exception handler to the driver URL: https://github.com/apache/spark/pull/24796#issuecomment-512126336 Yay from me. This is an automated message from the

[GitHub] [spark] AmplabJenkins commented on issue #25177: [SPARK-28418][PYTHON][SQL] Wait for event process in 'test_query_execution_listener_on_collect'

2019-07-16 Thread GitBox
AmplabJenkins commented on issue #25177: [SPARK-28418][PYTHON][SQL] Wait for event process in 'test_query_execution_listener_on_collect' URL: https://github.com/apache/spark/pull/25177#issuecomment-512126144 Test PASSed. Refer to this link for build results (access rights to CI server ne

[GitHub] [spark] AmplabJenkins commented on issue #25177: [SPARK-28418][PYTHON][SQL] Wait for event process in 'test_query_execution_listener_on_collect'

2019-07-16 Thread GitBox
AmplabJenkins commented on issue #25177: [SPARK-28418][PYTHON][SQL] Wait for event process in 'test_query_execution_listener_on_collect' URL: https://github.com/apache/spark/pull/25177#issuecomment-512126140 Merged build finished. Test PASSed. --

[GitHub] [spark] jerryshao commented on a change in pull request #25007: [SPARK-28209][CORE][SHUFFLE] Proposed new shuffle writer API

2019-07-16 Thread GitBox
jerryshao commented on a change in pull request #25007: [SPARK-28209][CORE][SHUFFLE] Proposed new shuffle writer API URL: https://github.com/apache/spark/pull/25007#discussion_r304243823 ## File path: core/src/main/java/org/apache/spark/shuffle/api/ShuffleMapOutputWriter.java ###

[GitHub] [spark] jerryshao commented on a change in pull request #25007: [SPARK-28209][CORE][SHUFFLE] Proposed new shuffle writer API

2019-07-16 Thread GitBox
jerryshao commented on a change in pull request #25007: [SPARK-28209][CORE][SHUFFLE] Proposed new shuffle writer API URL: https://github.com/apache/spark/pull/25007#discussion_r304245623 ## File path: core/src/main/java/org/apache/spark/shuffle/api/SupportsTransferTo.java ###

<    4   5   6   7   8   9