[GitHub] [spark] AmplabJenkins commented on pull request #29881: [SPARK-32852][SQL} spark.sql.hive.metastore.jars support HDFS location

2020-09-26 Thread GitBox
AmplabJenkins commented on pull request #29881: URL: https://github.com/apache/spark/pull/29881#issuecomment-699589036 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29881: [SPARK-32852][SQL} spark.sql.hive.metastore.jars support HDFS location

2020-09-26 Thread GitBox
AmplabJenkins removed a comment on pull request #29881: URL: https://github.com/apache/spark/pull/29881#issuecomment-699589036 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #29881: [SPARK-32852][SQL} spark.sql.hive.metastore.jars support HDFS location

2020-09-26 Thread GitBox
SparkQA commented on pull request #29881: URL: https://github.com/apache/spark/pull/29881#issuecomment-699589033 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33757/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29881: [SPARK-32852][SQL} spark.sql.hive.metastore.jars support HDFS location

2020-09-26 Thread GitBox
AmplabJenkins removed a comment on pull request #29881: URL: https://github.com/apache/spark/pull/29881#issuecomment-699588173 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #29881: [SPARK-32852][SQL} spark.sql.hive.metastore.jars support HDFS location

2020-09-26 Thread GitBox
SparkQA commented on pull request #29881: URL: https://github.com/apache/spark/pull/29881#issuecomment-699588171 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33756/

[GitHub] [spark] AmplabJenkins commented on pull request #29881: [SPARK-32852][SQL} spark.sql.hive.metastore.jars support HDFS location

2020-09-26 Thread GitBox
AmplabJenkins commented on pull request #29881: URL: https://github.com/apache/spark/pull/29881#issuecomment-699588173 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29881: [SPARK-32852][SQL} spark.sql.hive.metastore.jars support HDFS location

2020-09-26 Thread GitBox
SparkQA commented on pull request #29881: URL: https://github.com/apache/spark/pull/29881#issuecomment-699587803 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33757/

[GitHub] [spark] SparkQA commented on pull request #29881: [SPARK-32852][SQL} spark.sql.hive.metastore.jars support HDFS location

2020-09-26 Thread GitBox
SparkQA commented on pull request #29881: URL: https://github.com/apache/spark/pull/29881#issuecomment-699586571 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33756/

[GitHub] [spark] SparkQA commented on pull request #29881: [SPARK-32852][SQL} spark.sql.hive.metastore.jars support HDFS location

2020-09-26 Thread GitBox
SparkQA commented on pull request #29881: URL: https://github.com/apache/spark/pull/29881#issuecomment-699584677 **[Test build #129141 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129141/testReport)** for PR 29881 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29881: [SPARK-32852][SQL} spark.sql.hive.metastore.jars support HDFS location

2020-09-26 Thread GitBox
AmplabJenkins removed a comment on pull request #29881: URL: https://github.com/apache/spark/pull/29881#issuecomment-699583745 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA commented on pull request #29881: [SPARK-32852][SQL} spark.sql.hive.metastore.jars support HDFS location

2020-09-26 Thread GitBox
SparkQA commented on pull request #29881: URL: https://github.com/apache/spark/pull/29881#issuecomment-699583741 **[Test build #129140 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129140/testReport)** for PR 29881 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #29881: [SPARK-32852][SQL} spark.sql.hive.metastore.jars support HDFS location

2020-09-26 Thread GitBox
SparkQA removed a comment on pull request #29881: URL: https://github.com/apache/spark/pull/29881#issuecomment-699583671 **[Test build #129140 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129140/testReport)** for PR 29881 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29881: [SPARK-32852][SQL} spark.sql.hive.metastore.jars support HDFS location

2020-09-26 Thread GitBox
AmplabJenkins commented on pull request #29881: URL: https://github.com/apache/spark/pull/29881#issuecomment-699583743 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29881: [SPARK-32852][SQL} spark.sql.hive.metastore.jars support HDFS location

2020-09-26 Thread GitBox
AmplabJenkins removed a comment on pull request #29881: URL: https://github.com/apache/spark/pull/29881#issuecomment-699583743 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA commented on pull request #29881: [SPARK-32852][SQL} spark.sql.hive.metastore.jars support HDFS location

2020-09-26 Thread GitBox
SparkQA commented on pull request #29881: URL: https://github.com/apache/spark/pull/29881#issuecomment-699583671 **[Test build #129140 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129140/testReport)** for PR 29881 at commit

[GitHub] [spark] AngersZhuuuu opened a new pull request #29881: [SPARK-32852][SQL} spark.sql.hive.metastore.jars support HDFS location

2020-09-26 Thread GitBox
AngersZh opened a new pull request #29881: URL: https://github.com/apache/spark/pull/29881 ### What changes were proposed in this pull request? Support `spark.sql.hive.metastore.jars` use HDFS location. When user need to use path to set hive metastore jars, you should set

[GitHub] [spark] AngersZhuuuu commented on pull request #29881: [SPARK-32852][SQL} spark.sql.hive.metastore.jars support HDFS location

2020-09-26 Thread GitBox
AngersZh commented on pull request #29881: URL: https://github.com/apache/spark/pull/29881#issuecomment-699583150 ping @wangyum This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] AmplabJenkins commented on pull request #29880: [SPARK-33004][SQL] Migrate DESCRIBE column to use UnresolvedTableOrView to resolve the identifier

2020-09-26 Thread GitBox
AmplabJenkins commented on pull request #29880: URL: https://github.com/apache/spark/pull/29880#issuecomment-699582421 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29880: [SPARK-33004][SQL] Migrate DESCRIBE column to use UnresolvedTableOrView to resolve the identifier

2020-09-26 Thread GitBox
AmplabJenkins removed a comment on pull request #29880: URL: https://github.com/apache/spark/pull/29880#issuecomment-699582421 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA removed a comment on pull request #29880: [SPARK-33004][SQL] Migrate DESCRIBE column to use UnresolvedTableOrView to resolve the identifier

2020-09-26 Thread GitBox
SparkQA removed a comment on pull request #29880: URL: https://github.com/apache/spark/pull/29880#issuecomment-699560269 **[Test build #129138 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129138/testReport)** for PR 29880 at commit

[GitHub] [spark] SparkQA commented on pull request #29880: [SPARK-33004][SQL] Migrate DESCRIBE column to use UnresolvedTableOrView to resolve the identifier

2020-09-26 Thread GitBox
SparkQA commented on pull request #29880: URL: https://github.com/apache/spark/pull/29880#issuecomment-699582316 **[Test build #129138 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129138/testReport)** for PR 29880 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29868: [SPARK-32973][ML][DOC] FeatureHasher does not check categoricalCols in inputCols

2020-09-26 Thread GitBox
AmplabJenkins removed a comment on pull request #29868: URL: https://github.com/apache/spark/pull/29868#issuecomment-699580312 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29868: [SPARK-32973][ML][DOC] FeatureHasher does not check categoricalCols in inputCols

2020-09-26 Thread GitBox
AmplabJenkins commented on pull request #29868: URL: https://github.com/apache/spark/pull/29868#issuecomment-699580312 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] Tagar commented on pull request #29729: [SPARK-32032][SS] Avoid infinite wait in driver because of KafkaConsumer.poll(long) API

2020-09-26 Thread GitBox
Tagar commented on pull request #29729: URL: https://github.com/apache/spark/pull/29729#issuecomment-699580277 @HeartSaVioR I concur - we have seen issues with hanging kafka consumers in Spark Streaming applications. This would be great to fix.

[GitHub] [spark] SparkQA removed a comment on pull request #29868: [SPARK-32973][ML][DOC] FeatureHasher does not check categoricalCols in inputCols

2020-09-26 Thread GitBox
SparkQA removed a comment on pull request #29868: URL: https://github.com/apache/spark/pull/29868#issuecomment-699574853 **[Test build #129139 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129139/testReport)** for PR 29868 at commit

[GitHub] [spark] SparkQA commented on pull request #29868: [SPARK-32973][ML][DOC] FeatureHasher does not check categoricalCols in inputCols

2020-09-26 Thread GitBox
SparkQA commented on pull request #29868: URL: https://github.com/apache/spark/pull/29868#issuecomment-699580244 **[Test build #129139 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129139/testReport)** for PR 29868 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29868: [SPARK-32973][ML][DOC] FeatureHasher does not check categoricalCols in inputCols

2020-09-26 Thread GitBox
AmplabJenkins commented on pull request #29868: URL: https://github.com/apache/spark/pull/29868#issuecomment-699579897 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29868: [SPARK-32973][ML][DOC] FeatureHasher does not check categoricalCols in inputCols

2020-09-26 Thread GitBox
AmplabJenkins removed a comment on pull request #29868: URL: https://github.com/apache/spark/pull/29868#issuecomment-699579897 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #29868: [SPARK-32973][ML][DOC] FeatureHasher does not check categoricalCols in inputCols

2020-09-26 Thread GitBox
SparkQA commented on pull request #29868: URL: https://github.com/apache/spark/pull/29868#issuecomment-699579896 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33755/

[GitHub] [spark] nssalian commented on pull request #29877: [SPARK-27872][K8S][2.4] Fix executor service account inconsistency backport to branch-2.4

2020-09-26 Thread GitBox
nssalian commented on pull request #29877: URL: https://github.com/apache/spark/pull/29877#issuecomment-699579277 > Can we have java doc here? There's no javadoc in the original PR (https://github.com/apache/spark/pull/24748). This is simply a backport. Any improvements can follow

[GitHub] [spark] SparkQA commented on pull request #29868: [SPARK-32973][ML][DOC] FeatureHasher does not check categoricalCols in inputCols

2020-09-26 Thread GitBox
SparkQA commented on pull request #29868: URL: https://github.com/apache/spark/pull/29868#issuecomment-699578609 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33755/

[GitHub] [spark] LuciferYang commented on pull request #29857: [SPARK-32972][ML] Pass all UTs of `mllib` module in Scala 2.13

2020-09-26 Thread GitBox
LuciferYang commented on pull request #29857: URL: https://github.com/apache/spark/pull/29857#issuecomment-699576181 From the first round of code checking, may be this is the last module to fix Scala version compatibility

[GitHub] [spark] LuciferYang edited a comment on pull request #29871: [SPARK-32995][SQL] CostBasedJoinReorder optimizer rule should be idempotent

2020-09-26 Thread GitBox
LuciferYang edited a comment on pull request #29871: URL: https://github.com/apache/spark/pull/29871#issuecomment-699574633 If there are multiple candidate plans with same cost, I think `different inputs produce different outputs` is reasonableļ¼Œso only need ensure the same behavior in

[GitHub] [spark] SparkQA commented on pull request #29868: [SPARK-32973][ML][DOC] FeatureHasher does not check categoricalCols in inputCols

2020-09-26 Thread GitBox
SparkQA commented on pull request #29868: URL: https://github.com/apache/spark/pull/29868#issuecomment-699574853 **[Test build #129139 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129139/testReport)** for PR 29868 at commit

[GitHub] [spark] LuciferYang commented on pull request #29871: [SPARK-32995][SQL] CostBasedJoinReorder optimizer rule should be idempotent

2020-09-26 Thread GitBox
LuciferYang commented on pull request #29871: URL: https://github.com/apache/spark/pull/29871#issuecomment-699574633 If there are multiple candidate plans with the same cost, I think `different inputs produce different outputs` is reasonableļ¼Œso only need ensure the same behavior in Scala

[GitHub] [spark] viirya commented on pull request #29868: [SPARK-32973][ML][DOC] FeatureHasher does not check categoricalCols in inputCols

2020-09-26 Thread GitBox
viirya commented on pull request #29868: URL: https://github.com/apache/spark/pull/29868#issuecomment-699573214 Yeah, sounds good to me. Maybe also mention a warning will be shown? This is an automated message from the

[GitHub] [spark] zhengruifeng edited a comment on pull request #29868: [SPARK-32973][ML][DOC] FeatureHasher does not check categoricalCols in inputCols

2020-09-26 Thread GitBox
zhengruifeng edited a comment on pull request #29868: URL: https://github.com/apache/spark/pull/29868#issuecomment-699571847 > This is still 'required' right? yes. This algorithm was designed to have this requirement. But I guess `must` may suggest an exception/error, so what about

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #29054: [SPARK-32243][SQL]HiveSessionCatalog call super.makeFunctionExpression should throw earlier when got Spark UDAF Invalid argu

2020-09-26 Thread GitBox
AngersZh commented on a change in pull request #29054: URL: https://github.com/apache/spark/pull/29054#discussion_r495515099 ## File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveSessionCatalog.scala ## @@ -69,7 +69,14 @@ private[sql] class

[GitHub] [spark] zhengruifeng commented on pull request #29868: [SPARK-32973][ML][DOC] FeatureHasher does not check categoricalCols in inputCols

2020-09-26 Thread GitBox
zhengruifeng commented on pull request #29868: URL: https://github.com/apache/spark/pull/29868#issuecomment-699571847 > This is still 'required' right? yes. This algorithm was designed to have this requirement. But I guess `must` may suggest an exception/error, so what about

[GitHub] [spark] LuciferYang edited a comment on pull request #29857: [SPARK-32972][ML] Pass all UTs of `mllib` module in Scala 2.13

2020-09-26 Thread GitBox
LuciferYang edited a comment on pull request #29857: URL: https://github.com/apache/spark/pull/29857#issuecomment-699571003 @srowen Is there any other problem in this pr that needs to be fixed?thx ~ This is an automated

[GitHub] [spark] LuciferYang commented on pull request #29857: [SPARK-32972][ML] Pass all UTs of `mllib` module in Scala 2.13

2020-09-26 Thread GitBox
LuciferYang commented on pull request #29857: URL: https://github.com/apache/spark/pull/29857#issuecomment-699571003 @srowen Is there any other problem in this pr that needs to be fixed? This is an automated message from the

[GitHub] [spark] zhengruifeng commented on pull request #29850: [SPARK-32974][ML] FeatureHasher transform optimization

2020-09-26 Thread GitBox
zhengruifeng commented on pull request #29850: URL: https://github.com/apache/spark/pull/29850#issuecomment-699570531 Merged to master. Thanks @MLnick @srowen @huaxingao for reviewing! This is an automated message from the

[GitHub] [spark] zhengruifeng closed pull request #29850: [SPARK-32974][ML] FeatureHasher transform optimization

2020-09-26 Thread GitBox
zhengruifeng closed pull request #29850: URL: https://github.com/apache/spark/pull/29850 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] zhengruifeng commented on pull request #29852: [SPARK-21481][ML][FOLLOWUP][Trivial] HashingTF use util.collection.OpenHashMap instead of mutable.HashMap

2020-09-26 Thread GitBox
zhengruifeng commented on pull request #29852: URL: https://github.com/apache/spark/pull/29852#issuecomment-699570008 Thanks @srowen and @huaxingao for help reviewing This is an automated message from the Apache Git Service.

[GitHub] [spark] github-actions[bot] closed pull request #28778: [SPARK-31949][SQL] Add spark.default.parallelism in SQLConf for isolated across session

2020-09-26 Thread GitBox
github-actions[bot] closed pull request #28778: URL: https://github.com/apache/spark/pull/28778 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29880: [SPARK-33004][SQL] Migrate DESCRIBE column to use UnresolvedTableOrView to resolve the identifier

2020-09-26 Thread GitBox
AmplabJenkins removed a comment on pull request #29880: URL: https://github.com/apache/spark/pull/29880#issuecomment-699565422 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA commented on pull request #29880: [SPARK-33004][SQL] Migrate DESCRIBE column to use UnresolvedTableOrView to resolve the identifier

2020-09-26 Thread GitBox
SparkQA commented on pull request #29880: URL: https://github.com/apache/spark/pull/29880#issuecomment-699565417 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33754/

[GitHub] [spark] AmplabJenkins commented on pull request #29880: [SPARK-33004][SQL] Migrate DESCRIBE column to use UnresolvedTableOrView to resolve the identifier

2020-09-26 Thread GitBox
AmplabJenkins commented on pull request #29880: URL: https://github.com/apache/spark/pull/29880#issuecomment-699565420 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29880: [SPARK-33004][SQL] Migrate DESCRIBE column to use UnresolvedTableOrView to resolve the identifier

2020-09-26 Thread GitBox
AmplabJenkins removed a comment on pull request #29880: URL: https://github.com/apache/spark/pull/29880#issuecomment-699565420 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA commented on pull request #29880: [SPARK-33004][SQL] Migrate DESCRIBE column to use UnresolvedTableOrView to resolve the identifier

2020-09-26 Thread GitBox
SparkQA commented on pull request #29880: URL: https://github.com/apache/spark/pull/29880#issuecomment-699563439 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33754/

[GitHub] [spark] AmplabJenkins commented on pull request #29875: [SPARK-32999][SQL] Use Utils.getSimpleName to avoid hitting Malformed class name in TreeNode

2020-09-26 Thread GitBox
AmplabJenkins commented on pull request #29875: URL: https://github.com/apache/spark/pull/29875#issuecomment-699563137 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29875: [SPARK-32999][SQL] Use Utils.getSimpleName to avoid hitting Malformed class name in TreeNode

2020-09-26 Thread GitBox
AmplabJenkins removed a comment on pull request #29875: URL: https://github.com/apache/spark/pull/29875#issuecomment-699563137 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA removed a comment on pull request #29875: [SPARK-32999][SQL] Use Utils.getSimpleName to avoid hitting Malformed class name in TreeNode

2020-09-26 Thread GitBox
SparkQA removed a comment on pull request #29875: URL: https://github.com/apache/spark/pull/29875#issuecomment-699538332 **[Test build #129135 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129135/testReport)** for PR 29875 at commit

[GitHub] [spark] SparkQA commented on pull request #29875: [SPARK-32999][SQL] Use Utils.getSimpleName to avoid hitting Malformed class name in TreeNode

2020-09-26 Thread GitBox
SparkQA commented on pull request #29875: URL: https://github.com/apache/spark/pull/29875#issuecomment-699562968 **[Test build #129135 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129135/testReport)** for PR 29875 at commit

[GitHub] [spark] SparkQA commented on pull request #29880: [SPARK-33004][SQL] Migrate DESCRIBE column to use UnresolvedTableOrView to resolve the identifier

2020-09-26 Thread GitBox
SparkQA commented on pull request #29880: URL: https://github.com/apache/spark/pull/29880#issuecomment-699560269 **[Test build #129138 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129138/testReport)** for PR 29880 at commit

[GitHub] [spark] imback82 commented on a change in pull request #29880: [SPARK-33004][SQL] Migrate DESCRIBE column to use UnresolvedTableOrView to resolve the identifier

2020-09-26 Thread GitBox
imback82 commented on a change in pull request #29880: URL: https://github.com/apache/spark/pull/29880#discussion_r495505316 ## File path: sql/core/src/test/scala/org/apache/spark/sql/connector/DataSourceV2SQLSuite.scala ## @@ -168,7 +168,31 @@ class DataSourceV2SQLSuite

[GitHub] [spark] SparkQA commented on pull request #29879: [SPARK-33002][PYTHON] Remove non-API annotations.

2020-09-26 Thread GitBox
SparkQA commented on pull request #29879: URL: https://github.com/apache/spark/pull/29879#issuecomment-699559967 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33753/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29879: [SPARK-33002][PYTHON] Remove non-API annotations.

2020-09-26 Thread GitBox
AmplabJenkins removed a comment on pull request #29879: URL: https://github.com/apache/spark/pull/29879#issuecomment-699559970 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29879: [SPARK-33002][PYTHON] Remove non-API annotations.

2020-09-26 Thread GitBox
AmplabJenkins commented on pull request #29879: URL: https://github.com/apache/spark/pull/29879#issuecomment-699559970 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] imback82 opened a new pull request #29880: [SPARK-33004][SQL] Migrate DESCRIBE column to use UnresolvedTableOrView to resolve the identifier

2020-09-26 Thread GitBox
imback82 opened a new pull request #29880: URL: https://github.com/apache/spark/pull/29880 ### What changes were proposed in this pull request? This PR proposes to migrate `DESCRIBE tbl colname` to use `UnresolvedTableOrView` to resolve the table/view identifier. This allows

[GitHub] [spark] dongjoon-hyun commented on pull request #29875: [SPARK-32999][SQL] Use Utils.getSimpleName to avoid hitting Malformed class name in TreeNode

2020-09-26 Thread GitBox
dongjoon-hyun commented on pull request #29875: URL: https://github.com/apache/spark/pull/29875#issuecomment-699559351 Hi, @rednaxelafx . Could you make a backport to branch-2.4? This is an automated message from the Apache

[GitHub] [spark] dongjoon-hyun closed pull request #29875: [SPARK-32999][SQL] Use Utils.getSimpleName to avoid hitting Malformed class name in TreeNode

2020-09-26 Thread GitBox
dongjoon-hyun closed pull request #29875: URL: https://github.com/apache/spark/pull/29875 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] SparkQA commented on pull request #29879: [SPARK-33002][PYTHON] Remove non-API annotations.

2020-09-26 Thread GitBox
SparkQA commented on pull request #29879: URL: https://github.com/apache/spark/pull/29879#issuecomment-699558867 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33753/

[GitHub] [spark] gemelen edited a comment on pull request #29286: [WIP][SPARK-21708][BUILD] Migrate build to sbt 1.x

2020-09-26 Thread GitBox
gemelen edited a comment on pull request #29286: URL: https://github.com/apache/spark/pull/29286#issuecomment-699543632 @srowen yes, it's in Scaladoc/doc part of documentation generation task (and not StackOverflowException). Yes, it's probably classpath issue, since with different set of

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29879: [SPARK-33002][PYTHON] Remove non-API annotations.

2020-09-26 Thread GitBox
AmplabJenkins removed a comment on pull request #29879: URL: https://github.com/apache/spark/pull/29879#issuecomment-699557700 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29879: [SPARK-33002][PYTHON] Remove non-API annotations.

2020-09-26 Thread GitBox
AmplabJenkins commented on pull request #29879: URL: https://github.com/apache/spark/pull/29879#issuecomment-699557700 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29879: [SPARK-33002][PYTHON] Remove non-API annotations.

2020-09-26 Thread GitBox
SparkQA commented on pull request #29879: URL: https://github.com/apache/spark/pull/29879#issuecomment-699557647 **[Test build #129137 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129137/testReport)** for PR 29879 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #29879: [SPARK-33002][PYTHON] Remove non-API annotations.

2020-09-26 Thread GitBox
SparkQA removed a comment on pull request #29879: URL: https://github.com/apache/spark/pull/29879#issuecomment-699555766 **[Test build #129137 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129137/testReport)** for PR 29879 at commit

[GitHub] [spark] SparkQA commented on pull request #29879: [SPARK-33002][PYTHON] Remove non-API annotations.

2020-09-26 Thread GitBox
SparkQA commented on pull request #29879: URL: https://github.com/apache/spark/pull/29879#issuecomment-699555766 **[Test build #129137 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129137/testReport)** for PR 29879 at commit

[GitHub] [spark] zero323 opened a new pull request #29879: [SPARK-33002][PYTHON] Remove non-API annotations.

2020-09-26 Thread GitBox
zero323 opened a new pull request #29879: URL: https://github.com/apache/spark/pull/29879 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? ### How

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29878: [SPARK-32714][FOLLOW-UP][PYTHON] Address pyspark.install typing errors

2020-09-26 Thread GitBox
AmplabJenkins removed a comment on pull request #29878: URL: https://github.com/apache/spark/pull/29878#issuecomment-699548094 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29878: [SPARK-32714][FOLLOW-UP][PYTHON] Address pyspark.install typing errors

2020-09-26 Thread GitBox
AmplabJenkins commented on pull request #29878: URL: https://github.com/apache/spark/pull/29878#issuecomment-699548094 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29878: [SPARK-32714][FOLLOW-UP][PYTHON] Address pyspark.install typing errors

2020-09-26 Thread GitBox
SparkQA commented on pull request #29878: URL: https://github.com/apache/spark/pull/29878#issuecomment-699548091 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33752/

[GitHub] [spark] SparkQA commented on pull request #29878: [SPARK-32714][FOLLOW-UP][PYTHON] Address pyspark.install typing errors

2020-09-26 Thread GitBox
SparkQA commented on pull request #29878: URL: https://github.com/apache/spark/pull/29878#issuecomment-699546546 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33752/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29875: [SPARK-32999][SQL] Use Utils.getSimpleName to avoid hitting Malformed class name in TreeNode

2020-09-26 Thread GitBox
AmplabJenkins removed a comment on pull request #29875: URL: https://github.com/apache/spark/pull/29875#issuecomment-699545561 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29875: [SPARK-32999][SQL] Use Utils.getSimpleName to avoid hitting Malformed class name in TreeNode

2020-09-26 Thread GitBox
AmplabJenkins commented on pull request #29875: URL: https://github.com/apache/spark/pull/29875#issuecomment-699545561 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29875: [SPARK-32999][SQL] Use Utils.getSimpleName to avoid hitting Malformed class name in TreeNode

2020-09-26 Thread GitBox
SparkQA commented on pull request #29875: URL: https://github.com/apache/spark/pull/29875#issuecomment-699545558 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33751/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29878: [SPARK-32714][FOLLOW-UP][PYTHON] Address pyspark.install typing errors

2020-09-26 Thread GitBox
AmplabJenkins removed a comment on pull request #29878: URL: https://github.com/apache/spark/pull/29878#issuecomment-699544986 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29878: [SPARK-32714][FOLLOW-UP] Address pyspark.install typing errors

2020-09-26 Thread GitBox
AmplabJenkins commented on pull request #29878: URL: https://github.com/apache/spark/pull/29878#issuecomment-699544986 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29878: [SPARK-32714][FOLLOW-UP] Address pyspark.install typing errors

2020-09-26 Thread GitBox
SparkQA commented on pull request #29878: URL: https://github.com/apache/spark/pull/29878#issuecomment-699544918 **[Test build #129136 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129136/testReport)** for PR 29878 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #29878: [SPARK-32714][FOLLOW-UP] Address pyspark.install typing errors

2020-09-26 Thread GitBox
SparkQA removed a comment on pull request #29878: URL: https://github.com/apache/spark/pull/29878#issuecomment-699542375 **[Test build #129136 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129136/testReport)** for PR 29878 at commit

[GitHub] [spark] gemelen edited a comment on pull request #29286: [WIP][SPARK-21708][BUILD] Migrate build to sbt 1.x

2020-09-26 Thread GitBox
gemelen edited a comment on pull request #29286: URL: https://github.com/apache/spark/pull/29286#issuecomment-699543632 @srowen yes, it's in Scaladoc/doc part of documentation generation task (and not StackOverflowException). Yes, it's probably classpath issue, since with different set of

[GitHub] [spark] gemelen commented on pull request #29286: [WIP][SPARK-21708][BUILD] Migrate build to sbt 1.x

2020-09-26 Thread GitBox
gemelen commented on pull request #29286: URL: https://github.com/apache/spark/pull/29286#issuecomment-699543632 @srowen yes, it's in Scaladoc/doc part of documentation generation task (and not StackOverflowException). Yes, it's probably classpath issue, since with different set of

[GitHub] [spark] SparkQA commented on pull request #29875: [SPARK-32999][SQL] Use Utils.getSimpleName to avoid hitting Malformed class name in TreeNode

2020-09-26 Thread GitBox
SparkQA commented on pull request #29875: URL: https://github.com/apache/spark/pull/29875#issuecomment-699543400 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33751/

[GitHub] [spark] SparkQA commented on pull request #29878: [SPARK-32714][FOLLOW-UP] Address pyspark.install typing errors

2020-09-26 Thread GitBox
SparkQA commented on pull request #29878: URL: https://github.com/apache/spark/pull/29878#issuecomment-699542375 **[Test build #129136 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129136/testReport)** for PR 29878 at commit

[GitHub] [spark] zero323 commented on pull request #29878: [SPARK-32714][FOLLOW-UP] Address pyspark.install typing errors

2020-09-26 Thread GitBox
zero323 commented on pull request #29878: URL: https://github.com/apache/spark/pull/29878#issuecomment-699542233 cc @HyukjinKwon This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] zero323 opened a new pull request #29878: [SPARK-32714][FOLLOW-UP] Address pyspark.install typing errors

2020-09-26 Thread GitBox
zero323 opened a new pull request #29878: URL: https://github.com/apache/spark/pull/29878 ### What changes were proposed in this pull request? This PR adds two `type: ignores`, one in `pyspark.install` and one in related tests. ### Why are the changes needed?

[GitHub] [spark] srowen commented on pull request #29286: [WIP][SPARK-21708][BUILD] Migrate build to sbt 1.x

2020-09-26 Thread GitBox
srowen commented on pull request #29286: URL: https://github.com/apache/spark/pull/29286#issuecomment-699541340 I'm kind of catching up here, but it looks like Github PR builder test failures currently have to do with scaladoc build not understanding the classpath in the same way? not a

[GitHub] [spark] zero323 commented on a change in pull request #29180: [SPARK-17333][PYSPARK] Enable mypy on the repository

2020-09-26 Thread GitBox
zero323 commented on a change in pull request #29180: URL: https://github.com/apache/spark/pull/29180#discussion_r495487007 ## File path: dev/lint-python ## @@ -122,6 +123,31 @@ function pycodestyle_test { fi } +function mypy_test { +local MYPY_REPORT= +local

[GitHub] [spark] viirya commented on a change in pull request #29024: [SPARK-32001][SQL]Create JDBC authentication provider developer API

2020-09-26 Thread GitBox
viirya commented on a change in pull request #29024: URL: https://github.com/apache/spark/pull/29024#discussion_r495488559 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/connection/ConnectionProvider.scala ## @@ -18,60 +18,41 @@ package

[GitHub] [spark] SparkQA commented on pull request #29875: [SPARK-32999][SQL] Use Utils.getSimpleName to avoid hitting Malformed class name in TreeNode

2020-09-26 Thread GitBox
SparkQA commented on pull request #29875: URL: https://github.com/apache/spark/pull/29875#issuecomment-699538332 **[Test build #129135 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129135/testReport)** for PR 29875 at commit

[GitHub] [spark] viirya commented on pull request #29875: [SPARK-32999][SQL] Use Utils.getSimpleName to avoid hitting Malformed class name in TreeNode

2020-09-26 Thread GitBox
viirya commented on pull request #29875: URL: https://github.com/apache/spark/pull/29875#issuecomment-699537905 retest this please This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] AmplabJenkins commented on pull request #29286: [WIP][SPARK-21708][BUILD] Migrate build to sbt 1.x

2020-09-26 Thread GitBox
AmplabJenkins commented on pull request #29286: URL: https://github.com/apache/spark/pull/29286#issuecomment-699515462 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29286: [WIP][SPARK-21708][BUILD] Migrate build to sbt 1.x

2020-09-26 Thread GitBox
AmplabJenkins removed a comment on pull request #29286: URL: https://github.com/apache/spark/pull/29286#issuecomment-699515462 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #29286: [WIP][SPARK-21708][BUILD] Migrate build to sbt 1.x

2020-09-26 Thread GitBox
SparkQA commented on pull request #29286: URL: https://github.com/apache/spark/pull/29286#issuecomment-699515223 **[Test build #129134 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129134/testReport)** for PR 29286 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #29286: [WIP][SPARK-21708][BUILD] Migrate build to sbt 1.x

2020-09-26 Thread GitBox
SparkQA removed a comment on pull request #29286: URL: https://github.com/apache/spark/pull/29286#issuecomment-699494751 **[Test build #129134 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129134/testReport)** for PR 29286 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29054: [SPARK-32243][SQL]HiveSessionCatalog call super.makeFunctionExpression should throw earlier when got Spark UDAF Invalid arguments numbe

2020-09-26 Thread GitBox
AmplabJenkins commented on pull request #29054: URL: https://github.com/apache/spark/pull/29054#issuecomment-699509613 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29054: [SPARK-32243][SQL]HiveSessionCatalog call super.makeFunctionExpression should throw earlier when got Spark UDAF Invalid argumen

2020-09-26 Thread GitBox
AmplabJenkins removed a comment on pull request #29054: URL: https://github.com/apache/spark/pull/29054#issuecomment-699509613 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #29054: [SPARK-32243][SQL]HiveSessionCatalog call super.makeFunctionExpression should throw earlier when got Spark UDAF Invalid arguments number erro

2020-09-26 Thread GitBox
SparkQA commented on pull request #29054: URL: https://github.com/apache/spark/pull/29054#issuecomment-699509362 **[Test build #129133 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129133/testReport)** for PR 29054 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #29054: [SPARK-32243][SQL]HiveSessionCatalog call super.makeFunctionExpression should throw earlier when got Spark UDAF Invalid arguments num

2020-09-26 Thread GitBox
SparkQA removed a comment on pull request #29054: URL: https://github.com/apache/spark/pull/29054#issuecomment-699478815 **[Test build #129133 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129133/testReport)** for PR 29054 at commit

  1   2   >