[GitHub] [spark] HyukjinKwon commented on a change in pull request #29242: [SPARK-31448] [PYTHON] Fix storage level used in cache() in dataframe.py

HyukjinKwon commented on a change in pull request #29242: URL: https://github.com/apache/spark/pull/29242#discussion_r484197550 ## File path: python/pyspark/storagelevel.py ## @@ -56,3 +56,5 @@ def __str__(self): StorageLevel.MEMORY_AND_DISK = StorageLevel(True, True, False,

[GitHub] [spark] cloud-fan commented on a change in pull request #29242: [SPARK-31448] [PYTHON] Fix storage level used in cache() in dataframe.py

cloud-fan commented on a change in pull request #29242: URL: https://github.com/apache/spark/pull/29242#discussion_r484193839 ## File path: python/pyspark/sql/dataframe.py ## @@ -678,13 +678,14 @@ def cache(self): return self @since(1.3) -def persist(self,

[GitHub] [spark] HyukjinKwon commented on a change in pull request #29649: [SPARK-32779][SQL] Avoid using synchronized API of SessionCatalog in withClient flow, this leads to DeadLock

HyukjinKwon commented on a change in pull request #29649: URL: https://github.com/apache/spark/pull/29649#discussion_r484189882 ## File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala ## @@ -1329,8 +1329,7 @@ private[client] class Shim_v3_0

[GitHub] [spark] sandeep-katta commented on a change in pull request #29649: [SPARK-32779][SQL] Avoid using synchronized API of SessionCatalog in withClient flow, this leads to DeadLock

sandeep-katta commented on a change in pull request #29649: URL: https://github.com/apache/spark/pull/29649#discussion_r484189537 ## File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala ## @@ -1329,8 +1329,7 @@ private[client] class Shim_v3_0

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29605: [SPARK-31511][SQL][2.4] Make BytesToBytesMap iterators thread-safe

AmplabJenkins removed a comment on pull request #29605: URL: https://github.com/apache/spark/pull/29605#issuecomment-688043820 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] KevinSmile edited a comment on pull request #29653: [SPARK-32804][Launcher] Fix run-example command builder bug

KevinSmile edited a comment on pull request #29653: URL: https://github.com/apache/spark/pull/29653#issuecomment-688031240 I updated my patch code and maybe the new version patch is better to explain my point. The following snippet shows that **The first unrecognized arg is treated

[GitHub] [spark] AmplabJenkins commented on pull request #29605: [SPARK-31511][SQL][2.4] Make BytesToBytesMap iterators thread-safe

AmplabJenkins commented on pull request #29605: URL: https://github.com/apache/spark/pull/29605#issuecomment-688043820 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29605: [SPARK-31511][SQL][2.4] Make BytesToBytesMap iterators thread-safe

SparkQA commented on pull request #29605: URL: https://github.com/apache/spark/pull/29605#issuecomment-688043369 **[Test build #128336 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128336/testReport)** for PR 29605 at commit

[GitHub] [spark] HyukjinKwon commented on a change in pull request #29591: [SPARK-32714][PYTHON] Initial pyspark-stubs port.

HyukjinKwon commented on a change in pull request #29591: URL: https://github.com/apache/spark/pull/29591#discussion_r484188746 ## File path: examples/src/main/python/ml/estimator_transformer_param_example.py ## @@ -54,7 +56,7 @@ print(model1.extractParamMap()) #

[GitHub] [spark] sandeep-katta commented on a change in pull request #29649: [SPARK-32779][SQL] Avoid using synchronized API of SessionCatalog in withClient flow, this leads to DeadLock

sandeep-katta commented on a change in pull request #29649: URL: https://github.com/apache/spark/pull/29649#discussion_r484188369 ## File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala ## @@ -1329,8 +1329,7 @@ private[client] class Shim_v3_0

[GitHub] [spark] cloud-fan commented on a change in pull request #29579: [SPARK-32736][CORE] Avoid caching the removed decommissioned executors in TaskSchedulerImpl

cloud-fan commented on a change in pull request #29579: URL: https://github.com/apache/spark/pull/29579#discussion_r484188484 ## File path: core/src/main/scala/org/apache/spark/deploy/DeployMessage.scala ## @@ -188,7 +188,7 @@ private[deploy] object DeployMessages { }

[GitHub] [spark] cloud-fan commented on a change in pull request #29579: [SPARK-32736][CORE] Avoid caching the removed decommissioned executors in TaskSchedulerImpl

cloud-fan commented on a change in pull request #29579: URL: https://github.com/apache/spark/pull/29579#discussion_r484187912 ## File path: core/src/main/scala/org/apache/spark/deploy/DeployMessage.scala ## @@ -188,7 +188,7 @@ private[deploy] object DeployMessages { }

[GitHub] [spark] cloud-fan commented on pull request #29605: [SPARK-31511][SQL][2.4] Make BytesToBytesMap iterators thread-safe

cloud-fan commented on pull request #29605: URL: https://github.com/apache/spark/pull/29605#issuecomment-688041790 BTW, @cxzl25 you can use empty git commit to trigger test This is an automated message from the Apache Git

[GitHub] [spark] cloud-fan commented on pull request #29605: [SPARK-31511][SQL][2.4] Make BytesToBytesMap iterators thread-safe

cloud-fan commented on pull request #29605: URL: https://github.com/apache/spark/pull/29605#issuecomment-688041687 retest this please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] cloud-fan commented on a change in pull request #29649: [SPARK-32779][SQL] Avoid using synchronized API of SessionCatalog in withClient flow, this leads to DeadLock

cloud-fan commented on a change in pull request #29649: URL: https://github.com/apache/spark/pull/29649#discussion_r484187110 ## File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala ## @@ -1329,8 +1329,7 @@ private[client] class Shim_v3_0 extends

[GitHub] [spark] HyukjinKwon commented on a change in pull request #29649: [SPARK-32779][SQL] Avoid using synchronized API of SessionCatalog in withClient flow, this leads to DeadLock

HyukjinKwon commented on a change in pull request #29649: URL: https://github.com/apache/spark/pull/29649#discussion_r484186429 ## File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala ## @@ -1329,8 +1329,7 @@ private[client] class Shim_v3_0

[GitHub] [spark] cloud-fan commented on pull request #29635: [SPARK-32785][SQL] Interval with dangling parts should not results null

cloud-fan commented on pull request #29635: URL: https://github.com/apache/spark/pull/29635#issuecomment-688039775 thanks, merging to master! This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] cloud-fan closed pull request #29635: [SPARK-32785][SQL] Interval with dangling parts should not results null

cloud-fan closed pull request #29635: URL: https://github.com/apache/spark/pull/29635 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] cloud-fan commented on a change in pull request #29626: [SPARK-32777][SQL] Aggregation support aggregate function with multiple foldable expressions.

cloud-fan commented on a change in pull request #29626: URL: https://github.com/apache/spark/pull/29626#discussion_r484184462 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/RewriteDistinctAggregates.scala ## @@ -293,12 +297,16 @@ object

[GitHub] [spark] zero323 commented on a change in pull request #29591: [SPARK-32714][PYTHON] Initial pyspark-stubs port.

zero323 commented on a change in pull request #29591: URL: https://github.com/apache/spark/pull/29591#discussion_r484184260 ## File path: examples/src/main/python/ml/estimator_transformer_param_example.py ## @@ -54,7 +56,7 @@ print(model1.extractParamMap()) # We

[GitHub] [spark] wangyum commented on a change in pull request #27518: [SPARK-30768][SQL] Constraints inferred from inequality attributes

wangyum commented on a change in pull request #27518: URL: https://github.com/apache/spark/pull/27518#discussion_r484184236 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/QueryPlanConstraints.scala ## @@ -78,6 +91,72 @@ trait

[GitHub] [spark] cloud-fan commented on a change in pull request #29626: [SPARK-32777][SQL] Aggregation support aggregate function with multiple foldable expressions.

cloud-fan commented on a change in pull request #29626: URL: https://github.com/apache/spark/pull/29626#discussion_r484184121 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/RewriteDistinctAggregates.scala ## @@ -216,20 +216,24 @@ object

[GitHub] [spark] HyukjinKwon commented on a change in pull request #29649: [SPARK-32779][SQL] Avoid using synchronized API of SessionCatalog in withClient flow, this leads to DeadLock

HyukjinKwon commented on a change in pull request #29649: URL: https://github.com/apache/spark/pull/29649#discussion_r484183880 ## File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala ## @@ -1329,8 +1329,7 @@ private[client] class Shim_v3_0

[GitHub] [spark] cloud-fan commented on a change in pull request #29649: [SPARK-32779][SQL] Avoid using synchronized API of SessionCatalog in withClient flow, this leads to DeadLock

cloud-fan commented on a change in pull request #29649: URL: https://github.com/apache/spark/pull/29649#discussion_r484183375 ## File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala ## @@ -1329,8 +1329,7 @@ private[client] class Shim_v3_0 extends

[GitHub] [spark] KevinSmile commented on pull request #29653: [SPARK-32804][Launcher] Fix run-example command builder bug

KevinSmile commented on pull request #29653: URL: https://github.com/apache/spark/pull/29653#issuecomment-688031240 When you do `run-example`, you just specify the class name(e.g. SparkPi), and no need to specify the app-jar. So in backend code, `appResource` should be auto-find-and-set

[GitHub] [spark] KevinSmile commented on a change in pull request #29653: [SPARK-32804][Launcher] Fix run-example command builder bug

KevinSmile commented on a change in pull request #29653: URL: https://github.com/apache/spark/pull/29653#discussion_r484180285 ## File path: launcher/src/main/java/org/apache/spark/launcher/SparkSubmitCommandBuilder.java ## @@ -241,9 +241,11 @@ }

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29656: [SPARK-32807][SQL] ThriftServer open session use direct API to init current DB

AmplabJenkins removed a comment on pull request #29656: URL: https://github.com/apache/spark/pull/29656#issuecomment-688027437 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA removed a comment on pull request #29656: [SPARK-32807][SQL] ThriftServer open session use direct API to init current DB

SparkQA removed a comment on pull request #29656: URL: https://github.com/apache/spark/pull/29656#issuecomment-688016714 **[Test build #128333 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128333/testReport)** for PR 29656 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29656: [SPARK-32807][SQL] ThriftServer open session use direct API to init current DB

AmplabJenkins commented on pull request #29656: URL: https://github.com/apache/spark/pull/29656#issuecomment-688027437 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29639: [SPARK-32186][DOCS][PYTHON] Development - Debugging

AmplabJenkins removed a comment on pull request #29639: URL: https://github.com/apache/spark/pull/29639#issuecomment-688026864 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #29656: [SPARK-32807][SQL] ThriftServer open session use direct API to init current DB

SparkQA commented on pull request #29656: URL: https://github.com/apache/spark/pull/29656#issuecomment-688027227 **[Test build #128333 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128333/testReport)** for PR 29656 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #29639: [SPARK-32186][DOCS][PYTHON] Development - Debugging

SparkQA removed a comment on pull request #29639: URL: https://github.com/apache/spark/pull/29639#issuecomment-688018881 **[Test build #128335 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128335/testReport)** for PR 29639 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29639: [SPARK-32186][DOCS][PYTHON] Development - Debugging

AmplabJenkins commented on pull request #29639: URL: https://github.com/apache/spark/pull/29639#issuecomment-688026864 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29639: [SPARK-32186][DOCS][PYTHON] Development - Debugging

SparkQA commented on pull request #29639: URL: https://github.com/apache/spark/pull/29639#issuecomment-688026561 **[Test build #128335 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128335/testReport)** for PR 29639 at commit

[GitHub] [spark] HyukjinKwon commented on a change in pull request #29639: [SPARK-32186][DOCS][PYTHON] Development - Debugging

HyukjinKwon commented on a change in pull request #29639: URL: https://github.com/apache/spark/pull/29639#discussion_r484173595 ## File path: python/docs/source/development/debugging.rst ## @@ -0,0 +1,280 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +

[GitHub] [spark] SparkQA removed a comment on pull request #29656: [SPARK-32807][SQL] ThriftServer open session use direct API to init current DB

SparkQA removed a comment on pull request #29656: URL: https://github.com/apache/spark/pull/29656#issuecomment-688009132 **[Test build #128331 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128331/testReport)** for PR 29656 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29656: [SPARK-32807][SQL] ThriftServer open session use direct API to init current DB

AmplabJenkins removed a comment on pull request #29656: URL: https://github.com/apache/spark/pull/29656#issuecomment-688019831 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29656: [SPARK-32807][SQL] ThriftServer open session use direct API to init current DB

AmplabJenkins commented on pull request #29656: URL: https://github.com/apache/spark/pull/29656#issuecomment-688019831 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29639: [SPARK-32186][DOCS][PYTHON] Development - Debugging

AmplabJenkins removed a comment on pull request #29639: URL: https://github.com/apache/spark/pull/29639#issuecomment-688019481 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #29656: [SPARK-32807][SQL] ThriftServer open session use direct API to init current DB

SparkQA commented on pull request #29656: URL: https://github.com/apache/spark/pull/29656#issuecomment-688019603 **[Test build #128331 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128331/testReport)** for PR 29656 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29639: [SPARK-32186][DOCS][PYTHON] Development - Debugging

AmplabJenkins commented on pull request #29639: URL: https://github.com/apache/spark/pull/29639#issuecomment-688019481 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29639: [SPARK-32186][DOCS][PYTHON] Development - Debugging

AmplabJenkins removed a comment on pull request #29639: URL: https://github.com/apache/spark/pull/29639#issuecomment-688017442 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA commented on pull request #29639: [SPARK-32186][DOCS][PYTHON] Development - Debugging

SparkQA commented on pull request #29639: URL: https://github.com/apache/spark/pull/29639#issuecomment-688018881 **[Test build #128335 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128335/testReport)** for PR 29639 at commit

[GitHub] [spark] HyukjinKwon edited a comment on pull request #29639: [SPARK-32186][DOCS][PYTHON] Development - Debugging

HyukjinKwon edited a comment on pull request #29639: URL: https://github.com/apache/spark/pull/29639#issuecomment-688018098 The link here https://hyukjin-spark.readthedocs.io/en/stable/development/debugging.html is also updated. FYI, you might need to use incognito tab or something like

[GitHub] [spark] HyukjinKwon commented on pull request #29639: [SPARK-32186][DOCS][PYTHON] Development - Debugging

HyukjinKwon commented on pull request #29639: URL: https://github.com/apache/spark/pull/29639#issuecomment-688018098 The link here https://hyukjin-spark.readthedocs.io/en/stable/development/debugging.html is also updated.

[GitHub] [spark] HyukjinKwon commented on pull request #29639: [SPARK-32186][DOCS][PYTHON] Development - Debugging

HyukjinKwon commented on pull request #29639: URL: https://github.com/apache/spark/pull/29639#issuecomment-688017936 Hey guys, sorry for a bit of noise here. Would you guys mind if I ask take another look? I changed a bit a lot while I am addressing @BryanCutler and @zero323 comments.

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29639: [SPARK-32186][DOCS][PYTHON] Development - Debugging

AmplabJenkins removed a comment on pull request #29639: URL: https://github.com/apache/spark/pull/29639#issuecomment-688017434 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29639: [SPARK-32186][DOCS][PYTHON] Development - Debugging

AmplabJenkins removed a comment on pull request #29639: URL: https://github.com/apache/spark/pull/29639#issuecomment-688017081 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29639: [SPARK-32186][DOCS][PYTHON] Development - Debugging

AmplabJenkins commented on pull request #29639: URL: https://github.com/apache/spark/pull/29639#issuecomment-688017434 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29656: [SPARK-32807][SQL] ThriftServer open session use direct API to init current DB

AmplabJenkins removed a comment on pull request #29656: URL: https://github.com/apache/spark/pull/29656#issuecomment-688017138 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29656: [SPARK-32807][SQL] ThriftServer open session use direct API to init current DB

AmplabJenkins commented on pull request #29656: URL: https://github.com/apache/spark/pull/29656#issuecomment-688017138 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #29639: [SPARK-32186][DOCS][PYTHON] Development - Debugging

AmplabJenkins commented on pull request #29639: URL: https://github.com/apache/spark/pull/29639#issuecomment-688017081 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29639: [SPARK-32186][DOCS][PYTHON] Development - Debugging

SparkQA commented on pull request #29639: URL: https://github.com/apache/spark/pull/29639#issuecomment-688016741 **[Test build #128334 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128334/testReport)** for PR 29639 at commit

[GitHub] [spark] SparkQA commented on pull request #29656: [SPARK-32807][SQL] ThriftServer open session use direct API to init current DB

SparkQA commented on pull request #29656: URL: https://github.com/apache/spark/pull/29656#issuecomment-688016714 **[Test build #128333 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128333/testReport)** for PR 29656 at commit

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #29656: [SPARK-32807][SQL] ThriftServer open session use direct API to init current DB

AngersZh commented on a change in pull request #29656: URL: https://github.com/apache/spark/pull/29656#discussion_r484171144 ## File path: sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkSQLSessionManager.scala ## @@ -69,7 +69,7 @@

[GitHub] [spark] viirya commented on a change in pull request #29639: [SPARK-32186][DOCS][PYTHON] Development - Debugging

viirya commented on a change in pull request #29639: URL: https://github.com/apache/spark/pull/29639#discussion_r484171058 ## File path: python/docs/source/development/debugging.rst ## @@ -0,0 +1,280 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +or

[GitHub] [spark] wangyum commented on a change in pull request #29656: [SPARK-32807][SQL] ThriftServer open session use direct API to init current DB

wangyum commented on a change in pull request #29656: URL: https://github.com/apache/spark/pull/29656#discussion_r484170616 ## File path: sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkSQLSessionManager.scala ## @@ -69,7 +69,7 @@

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29639: [SPARK-32186][DOCS][PYTHON] Development - Debugging

AmplabJenkins removed a comment on pull request #29639: URL: https://github.com/apache/spark/pull/29639#issuecomment-688014688 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29639: [SPARK-32186][DOCS][PYTHON] Development - Debugging

AmplabJenkins commented on pull request #29639: URL: https://github.com/apache/spark/pull/29639#issuecomment-688014688 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29639: [SPARK-32186][DOCS][PYTHON] Development - Debugging

SparkQA commented on pull request #29639: URL: https://github.com/apache/spark/pull/29639#issuecomment-688014191 **[Test build #128332 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128332/testReport)** for PR 29639 at commit

[GitHub] [spark] cxzl25 commented on pull request #29605: [SPARK-31511][SQL][2.4] Make BytesToBytesMap iterators thread-safe

cxzl25 commented on pull request #29605: URL: https://github.com/apache/spark/pull/29605#issuecomment-688013875 I removed the code I changed in the commit (0e39f7a), but the jenkins and github action tests still don't pass, which is very strange.

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29656: [SPARK-32807][SQL] ThriftServer open session slow when high concurrent when init current DB

AmplabJenkins removed a comment on pull request #29656: URL: https://github.com/apache/spark/pull/29656#issuecomment-688009576 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] wangyum commented on a change in pull request #27518: [SPARK-30768][SQL] Constraints inferred from inequality attributes

wangyum commented on a change in pull request #27518: URL: https://github.com/apache/spark/pull/27518#discussion_r484168351 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/QueryPlanConstraints.scala ## @@ -78,6 +91,72 @@ trait

[GitHub] [spark] AmplabJenkins commented on pull request #29656: [SPARK-32807][SQL] ThriftServer open session slow when high concurrent when init current DB

AmplabJenkins commented on pull request #29656: URL: https://github.com/apache/spark/pull/29656#issuecomment-688009576 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] wangyum commented on a change in pull request #27518: [SPARK-30768][SQL] Constraints inferred from inequality attributes

wangyum commented on a change in pull request #27518: URL: https://github.com/apache/spark/pull/27518#discussion_r484167230 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/QueryPlanConstraints.scala ## @@ -78,6 +91,72 @@ trait

[GitHub] [spark] SparkQA commented on pull request #29656: [SPARK-32807][SQL] ThriftServer open session slow when high concurrent when init current DB

SparkQA commented on pull request #29656: URL: https://github.com/apache/spark/pull/29656#issuecomment-688009132 **[Test build #128331 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128331/testReport)** for PR 29656 at commit

[GitHub] [spark] AngersZhuuuu opened a new pull request #29656: [SPARK-32807][SQL] ThriftServer open session slow when high concurrent when init current DB

AngersZh opened a new pull request #29656: URL: https://github.com/apache/spark/pull/29656 ### What changes were proposed in this pull request? When init current database, we can use direct API, don't need to call SQL ### Why are the changes needed? No ### Does

[GitHub] [spark] AngersZhuuuu commented on pull request #29656: [SPARK-32807][SQL] ThriftServer open session slow when high concurrent when init current DB

AngersZh commented on pull request #29656: URL: https://github.com/apache/spark/pull/29656#issuecomment-688008917 cc @wangyum @juliuszsompolski This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] wangyum commented on a change in pull request #27518: [SPARK-30768][SQL] Constraints inferred from inequality attributes

wangyum commented on a change in pull request #27518: URL: https://github.com/apache/spark/pull/27518#discussion_r484166395 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/QueryPlanConstraints.scala ## @@ -78,6 +91,72 @@ trait

[GitHub] [spark] LuciferYang commented on pull request #29000: [SPARK-27194][SPARK-29302][SQL] Fix commit collision in dynamic partition overwrite mode

LuciferYang commented on pull request #29000: URL: https://github.com/apache/spark/pull/29000#issuecomment-688007790 Gentle ping @cloud-fan for further review This is an automated message from the Apache Git

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29655: [SPARK-32806][SQL] SortMergeJoin with partial hash distribution can be optimized to remove shuffle

AmplabJenkins removed a comment on pull request #29655: URL: https://github.com/apache/spark/pull/29655#issuecomment-687991585 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29655: [SPARK-32806][SQL] SortMergeJoin with partial hash distribution can be optimized to remove shuffle

AmplabJenkins commented on pull request #29655: URL: https://github.com/apache/spark/pull/29655#issuecomment-687991585 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29655: [SPARK-32806][SQL] SortMergeJoin with partial hash distribution can be optimized to remove shuffle

SparkQA commented on pull request #29655: URL: https://github.com/apache/spark/pull/29655#issuecomment-687991152 **[Test build #128330 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128330/testReport)** for PR 29655 at commit

[GitHub] [spark] imback82 opened a new pull request #29655: [SPARK-32806][SQL] SortMergeJoin with partial hash distribution can be optimized to remove shuffle

imback82 opened a new pull request #29655: URL: https://github.com/apache/spark/pull/29655 ### What changes were proposed in this pull request? This PR proposes to optimize SortMergeJoin (SMJ) if each of its children has hash output partitioning which "partially" satisfies

[GitHub] [spark] Ngone51 commented on pull request #29579: [SPARK-32736][CORE] Avoid caching the removed decommissioned executors in TaskSchedulerImpl

Ngone51 commented on pull request #29579: URL: https://github.com/apache/spark/pull/29579#issuecomment-687984920 @cloud-fan @holdenk Could you take a look? This is an automated message from the Apache Git Service. To

[GitHub] [spark] moomindani commented on pull request #28953: [SPARK-32013][SQL] Support query execution before reading DataFrame and before/after writing DataFrame over JDBC

moomindani commented on pull request #28953: URL: https://github.com/apache/spark/pull/28953#issuecomment-687959216 @gatorsmile Thank you for your comment. I understand that this kind of hooks are not specific to JDBC, it can be general to all the data sources. However, for JDBC, it

[GitHub] [spark] moomindani commented on pull request #29330: [SPARK-32432][SQL] Added support for reading ORC/Parquet files with SymlinkTextInputFormat

moomindani commented on pull request #29330: URL: https://github.com/apache/spark/pull/29330#issuecomment-687960686 Could you please take a look and review this PR? This is an automated message from the Apache Git Service.

[GitHub] [spark] HyukjinKwon commented on a change in pull request #29651: [SPARK-32794][SS] Fixed rare corner case error in micro-batch engine with some stateful queries + no-data-batches + V1 source

HyukjinKwon commented on a change in pull request #29651: URL: https://github.com/apache/spark/pull/29651#discussion_r484010520 ## File path: sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamTest.scala ## @@ -306,6 +306,14 @@ trait StreamTest extends QueryTest

[GitHub] [spark] HyukjinKwon commented on pull request #29591: [SPARK-32714][PYTHON] Initial pyspark-stubs port.

HyukjinKwon commented on pull request #29591: URL: https://github.com/apache/spark/pull/29591#issuecomment-687956805 @zero323, I usually prefer to don't block something by the env issue in Jenkins so such issue can be handled with enough time - @shaneknapp is sort of busy at this moment

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29639: [SPARK-32186][DOCS][PYTHON] Development - Debugging

AmplabJenkins removed a comment on pull request #29639: URL: https://github.com/apache/spark/pull/29639#issuecomment-687956266 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #29639: [SPARK-32186][DOCS][PYTHON] Development - Debugging

SparkQA commented on pull request #29639: URL: https://github.com/apache/spark/pull/29639#issuecomment-687956009 **[Test build #128329 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128329/testReport)** for PR 29639 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29639: [SPARK-32186][DOCS][PYTHON] Development - Debugging

AmplabJenkins commented on pull request #29639: URL: https://github.com/apache/spark/pull/29639#issuecomment-687956266 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #29639: [SPARK-32186][DOCS][PYTHON] Development - Debugging

SparkQA removed a comment on pull request #29639: URL: https://github.com/apache/spark/pull/29639#issuecomment-687948078 **[Test build #128329 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128329/testReport)** for PR 29639 at commit

[GitHub] [spark] HyukjinKwon commented on a change in pull request #29591: [SPARK-32714][PYTHON] Initial pyspark-stubs port.

HyukjinKwon commented on a change in pull request #29591: URL: https://github.com/apache/spark/pull/29591#discussion_r484138235 ## File path: dev/tox.ini ## @@ -20,5 +20,16 @@ exclude=python/pyspark/cloudpickle/*.py,shared.py,python/docs/source/conf.py,wor [flake8] select

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29591: [SPARK-32714][PYTHON] Initial pyspark-stubs port.

AmplabJenkins removed a comment on pull request #29591: URL: https://github.com/apache/spark/pull/29591#issuecomment-687955824 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] HyukjinKwon commented on a change in pull request #29591: [SPARK-32714][PYTHON] Initial pyspark-stubs port.

HyukjinKwon commented on a change in pull request #29591: URL: https://github.com/apache/spark/pull/29591#discussion_r484137854 ## File path: examples/src/main/python/ml/estimator_transformer_param_example.py ## @@ -54,7 +56,7 @@ print(model1.extractParamMap()) #

[GitHub] [spark] SparkQA removed a comment on pull request #29591: [SPARK-32714][PYTHON] Initial pyspark-stubs port.

SparkQA removed a comment on pull request #29591: URL: https://github.com/apache/spark/pull/29591#issuecomment-687917811 **[Test build #128328 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128328/testReport)** for PR 29591 at commit

[GitHub] [spark] SparkQA commented on pull request #29591: [SPARK-32714][PYTHON] Initial pyspark-stubs port.

SparkQA commented on pull request #29591: URL: https://github.com/apache/spark/pull/29591#issuecomment-687955217 **[Test build #128328 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128328/testReport)** for PR 29591 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29639: [SPARK-32186][DOCS][PYTHON] Development - Debugging

AmplabJenkins removed a comment on pull request #29639: URL: https://github.com/apache/spark/pull/29639#issuecomment-687948686 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29639: [SPARK-32186][DOCS][PYTHON] Development - Debugging

AmplabJenkins commented on pull request #29639: URL: https://github.com/apache/spark/pull/29639#issuecomment-687948686 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29639: [SPARK-32186][DOCS][PYTHON] Development - Debugging

SparkQA commented on pull request #29639: URL: https://github.com/apache/spark/pull/29639#issuecomment-687948078 **[Test build #128329 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128329/testReport)** for PR 29639 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29591: [SPARK-32714][PYTHON] Initial pyspark-stubs port.

AmplabJenkins removed a comment on pull request #29591: URL: https://github.com/apache/spark/pull/29591#issuecomment-687941944 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29591: [SPARK-32714][PYTHON] Initial pyspark-stubs port.

AmplabJenkins removed a comment on pull request #29591: URL: https://github.com/apache/spark/pull/29591#issuecomment-687941936 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #29591: [SPARK-32714][PYTHON] Initial pyspark-stubs port.

AmplabJenkins commented on pull request #29591: URL: https://github.com/apache/spark/pull/29591#issuecomment-687941936 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond

[GitHub] [spark] SparkQA removed a comment on pull request #29591: [SPARK-32714][PYTHON] Initial pyspark-stubs port.

SparkQA removed a comment on pull request #29591: URL: https://github.com/apache/spark/pull/29591#issuecomment-687916040 **[Test build #128327 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128327/testReport)** for PR 29591 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29591: [SPARK-32714][PYTHON] Initial pyspark-stubs port.

AmplabJenkins commented on pull request #29591: URL: https://github.com/apache/spark/pull/29591#issuecomment-687941944 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/128327/

[GitHub] [spark] SparkQA commented on pull request #29591: [SPARK-32714][PYTHON] Initial pyspark-stubs port.

SparkQA commented on pull request #29591: URL: https://github.com/apache/spark/pull/29591#issuecomment-687941639 **[Test build #128327 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128327/testReport)** for PR 29591 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29591: [SPARK-32714][PYTHON] Initial pyspark-stubs port.

AmplabJenkins removed a comment on pull request #29591: URL: https://github.com/apache/spark/pull/29591#issuecomment-687929852 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29591: [SPARK-32714][PYTHON] Initial pyspark-stubs port.

AmplabJenkins removed a comment on pull request #29591: URL: https://github.com/apache/spark/pull/29591#issuecomment-687929845 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #29591: [SPARK-32714][PYTHON] Initial pyspark-stubs port.

AmplabJenkins commented on pull request #29591: URL: https://github.com/apache/spark/pull/29591#issuecomment-687929845 This is an automated message from the Apache Git Service. To respond to the message, please log on to

  1   2   >