[jira] [Commented] (SPARK-36876) Support Dynamic Partition pruning for HiveTableScanExec

[ https://issues.apache.org/jira/browse/SPARK-36876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421944#comment-17421944 ] Apache Spark commented on SPARK-36876: -- User 'AngersZh' has created a pull request for this

[jira] [Commented] (SPARK-36876) Support Dynamic Partition pruning for HiveTableScanExec

[ https://issues.apache.org/jira/browse/SPARK-36876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421943#comment-17421943 ] Apache Spark commented on SPARK-36876: -- User 'AngersZh' has created a pull request for this

[jira] [Assigned] (SPARK-36876) Support Dynamic Partition pruning for HiveTableScanExec

[ https://issues.apache.org/jira/browse/SPARK-36876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36876: Assignee: (was: Apache Spark) > Support Dynamic Partition pruning for

[jira] [Assigned] (SPARK-36876) Support Dynamic Partition pruning for HiveTableScanExec

[ https://issues.apache.org/jira/browse/SPARK-36876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36876: Assignee: Apache Spark > Support Dynamic Partition pruning for HiveTableScanExec >

[jira] [Commented] (SPARK-36883) Upgrade R version to 4.1.1 in CI images

[ https://issues.apache.org/jira/browse/SPARK-36883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421941#comment-17421941 ] Apache Spark commented on SPARK-36883: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-36883) Upgrade R version to 4.1.1 in CI images

[ https://issues.apache.org/jira/browse/SPARK-36883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36883: Assignee: (was: Apache Spark) > Upgrade R version to 4.1.1 in CI images >

[jira] [Assigned] (SPARK-36883) Upgrade R version to 4.1.1 in CI images

[ https://issues.apache.org/jira/browse/SPARK-36883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36883: Assignee: (was: Apache Spark) > Upgrade R version to 4.1.1 in CI images >

[jira] [Assigned] (SPARK-36883) Upgrade R version to 4.1.1 in CI images

[ https://issues.apache.org/jira/browse/SPARK-36883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36883: Assignee: Apache Spark > Upgrade R version to 4.1.1 in CI images >

[jira] [Commented] (SPARK-36877) Calling ds.rdd with AQE enabled leads to jobs being run, eventually causing reruns

[ https://issues.apache.org/jira/browse/SPARK-36877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421939#comment-17421939 ] Hyukjin Kwon commented on SPARK-36877: -- cc [~maryannxue] too FYI > Calling ds.rdd with AQE enabled

[jira] [Resolved] (SPARK-36875) 21/09/28 11:18:51 WARN YarnScheduler: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resour

[ https://issues.apache.org/jira/browse/SPARK-36875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-36875. -- Resolution: Invalid Apparently It failed to connect to the Yarn cluster. Seems like an

[jira] [Updated] (SPARK-36875) 21/09/28 11:18:51 WARN YarnScheduler: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resourc

[ https://issues.apache.org/jira/browse/SPARK-36875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-36875: - Target Version/s: (was: 3.1.2) > 21/09/28 11:18:51 WARN YarnScheduler: Initial job has not

[jira] [Updated] (SPARK-36875) 21/09/28 11:18:51 WARN YarnScheduler: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resourc

[ https://issues.apache.org/jira/browse/SPARK-36875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-36875: - Fix Version/s: (was: 3.1.2) > 21/09/28 11:18:51 WARN YarnScheduler: Initial job has not

[jira] [Updated] (SPARK-36875) 21/09/28 11:18:51 WARN YarnScheduler: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resourc

[ https://issues.apache.org/jira/browse/SPARK-36875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-36875: - Language: (was: JAVA,) > 21/09/28 11:18:51 WARN YarnScheduler: Initial job has not accepted

[jira] [Updated] (SPARK-36875) 21/09/28 11:18:51 WARN YarnScheduler: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resourc

[ https://issues.apache.org/jira/browse/SPARK-36875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-36875: - Flags: (was: Important) > 21/09/28 11:18:51 WARN YarnScheduler: Initial job has not accepted

[jira] [Commented] (SPARK-36874) Ambiguous Self-Join detected only on right dataframe

[ https://issues.apache.org/jira/browse/SPARK-36874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421937#comment-17421937 ] Hyukjin Kwon commented on SPARK-36874: -- cc [~cloud_fan] FYI > Ambiguous Self-Join detected only on

[jira] [Comment Edited] (SPARK-36860) Create the external hive table for HBase failed

[ https://issues.apache.org/jira/browse/SPARK-36860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421936#comment-17421936 ] Hyukjin Kwon edited comment on SPARK-36860 at 9/29/21, 5:26 AM: Unless

[jira] [Commented] (SPARK-36860) Create the external hive table for HBase failed

[ https://issues.apache.org/jira/browse/SPARK-36860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421936#comment-17421936 ] Hyukjin Kwon commented on SPARK-36860: -- Unless it's explicitly documented, nothing is supported. >

[jira] [Commented] (SPARK-36858) Spark API to apply same function to multiple columns

[ https://issues.apache.org/jira/browse/SPARK-36858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421935#comment-17421935 ] Hyukjin Kwon commented on SPARK-36858: -- Can't we simply do this in a for loop? > Spark API to

[jira] [Commented] (SPARK-36888) Sha2 with bit_length 512 not being tested

[ https://issues.apache.org/jira/browse/SPARK-36888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421934#comment-17421934 ] Hyukjin Kwon commented on SPARK-36888: -- Please go ahead and add a test > Sha2 with bit_length 512

[jira] [Commented] (SPARK-36588) Use v2 commands by default

[ https://issues.apache.org/jira/browse/SPARK-36588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421929#comment-17421929 ] Apache Spark commented on SPARK-36588: -- User 'imback82' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36588) Use v2 commands by default

[ https://issues.apache.org/jira/browse/SPARK-36588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36588: Assignee: (was: Apache Spark) > Use v2 commands by default >

[jira] [Assigned] (SPARK-36588) Use v2 commands by default

[ https://issues.apache.org/jira/browse/SPARK-36588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36588: Assignee: Apache Spark > Use v2 commands by default > -- > >

[jira] [Commented] (SPARK-36883) Upgrade R version to 4.1.1 in CI images

[ https://issues.apache.org/jira/browse/SPARK-36883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421928#comment-17421928 ] Hyukjin Kwon commented on SPARK-36883: -- Nice! > Upgrade R version to 4.1.1 in CI images >

[jira] [Commented] (SPARK-36883) Upgrade R version to 4.1.1 in CI images

[ https://issues.apache.org/jira/browse/SPARK-36883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421925#comment-17421925 ] Dongjoon Hyun commented on SPARK-36883: --- Oh, it seems to be fixed a few minutes ago. > Upgrade R

[jira] [Updated] (SPARK-36888) Sha2 with bit_length 512 not being tested

[ https://issues.apache.org/jira/browse/SPARK-36888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] H. Vetinari updated SPARK-36888: Description: Looking at

[jira] [Updated] (SPARK-36888) Sha2 with bit_length 512 not being tested

[ https://issues.apache.org/jira/browse/SPARK-36888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] H. Vetinari updated SPARK-36888: Description: Looking at

[jira] [Created] (SPARK-36888) Sha2 with bit_length 512 not being tested

H. Vetinari created SPARK-36888: --- Summary: Sha2 with bit_length 512 not being tested Key: SPARK-36888 URL: https://issues.apache.org/jira/browse/SPARK-36888 Project: Spark Issue Type: Task

[jira] [Commented] (SPARK-36883) Upgrade R version to 4.1.1 in CI images

[ https://issues.apache.org/jira/browse/SPARK-36883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421921#comment-17421921 ] Dongjoon Hyun commented on SPARK-36883: --- BTW, [~hyukjin.kwon]. CRAN mirror is out of sync as of

[jira] [Updated] (SPARK-36886) Inline type hints for python/pyspark/sql/context.py

[ https://issues.apache.org/jira/browse/SPARK-36886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dgd_contributor updated SPARK-36886: Description: Inline type hints for python/pyspark/sql/context.py from Inline type hints

[jira] [Updated] (SPARK-36886) Inline type hints for python/pyspark/sql/context.py

[ https://issues.apache.org/jira/browse/SPARK-36886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dgd_contributor updated SPARK-36886: Summary: Inline type hints for python/pyspark/sql/context.py (was: Inline type hints for

[jira] [Commented] (SPARK-36883) Upgrade R version to 4.1.1 in CI images

[ https://issues.apache.org/jira/browse/SPARK-36883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421908#comment-17421908 ] Dongjoon Hyun commented on SPARK-36883: --- Let me update the image. > Upgrade R version to 4.1.1 in

[jira] [Commented] (SPARK-36883) Upgrade R version to 4.1.1 in CI images

[ https://issues.apache.org/jira/browse/SPARK-36883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421907#comment-17421907 ] Dongjoon Hyun commented on SPARK-36883: --- Thank you, [~hyukjin.kwon]. > Upgrade R version to 4.1.1

[jira] [Assigned] (SPARK-36526) Add supportsIndex interface

[ https://issues.apache.org/jira/browse/SPARK-36526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-36526: --- Assignee: Huaxin Gao > Add supportsIndex interface > --- > >

[jira] [Resolved] (SPARK-36526) Add supportsIndex interface

[ https://issues.apache.org/jira/browse/SPARK-36526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-36526. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 33754

[jira] [Assigned] (SPARK-36884) Inline type hints for python/pyspark/sql/session.py

[ https://issues.apache.org/jira/browse/SPARK-36884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36884: Assignee: (was: Apache Spark) > Inline type hints for python/pyspark/sql/session.py

[jira] [Assigned] (SPARK-36884) Inline type hints for python/pyspark/sql/session.py

[ https://issues.apache.org/jira/browse/SPARK-36884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36884: Assignee: Apache Spark > Inline type hints for python/pyspark/sql/session.py >

[jira] [Commented] (SPARK-36884) Inline type hints for python/pyspark/sql/session.py

[ https://issues.apache.org/jira/browse/SPARK-36884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421898#comment-17421898 ] Apache Spark commented on SPARK-36884: -- User 'ueshin' has created a pull request for this issue:

[jira] [Commented] (SPARK-36849) Migrate UseStatement to v2 command framework

[ https://issues.apache.org/jira/browse/SPARK-36849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421888#comment-17421888 ] dohongdayi commented on SPARK-36849: [~huaxingao] Thank you a lot ! > Migrate UseStatement to v2

[jira] [Commented] (SPARK-36887) Inline type hints for python/pyspark/sql/conf.py

[ https://issues.apache.org/jira/browse/SPARK-36887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421880#comment-17421880 ] dgd_contributor commented on SPARK-36887: - I'm working on this. > Inline type hints for

[jira] [Created] (SPARK-36887) Inline type hints for python/pyspark/sql/conf.py

dgd_contributor created SPARK-36887: --- Summary: Inline type hints for python/pyspark/sql/conf.py Key: SPARK-36887 URL: https://issues.apache.org/jira/browse/SPARK-36887 Project: Spark Issue

[jira] [Commented] (SPARK-36886) Inline type hints for python/pyspark/sql/column.py

[ https://issues.apache.org/jira/browse/SPARK-36886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421878#comment-17421878 ] dgd_contributor commented on SPARK-36886: - Working on this. > Inline type hints for

[jira] [Created] (SPARK-36886) Inline type hints for python/pyspark/sql/column.py

dgd_contributor created SPARK-36886: --- Summary: Inline type hints for python/pyspark/sql/column.py Key: SPARK-36886 URL: https://issues.apache.org/jira/browse/SPARK-36886 Project: Spark

[jira] [Commented] (SPARK-36845) Inline type hint files

[ https://issues.apache.org/jira/browse/SPARK-36845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421865#comment-17421865 ] Hyukjin Kwon commented on SPARK-36845: -- please go ahead. > Inline type hint files >

[jira] [Created] (SPARK-36885) Inline type hints for python/pyspark/sql/dataframe.py

Takuya Ueshin created SPARK-36885: - Summary: Inline type hints for python/pyspark/sql/dataframe.py Key: SPARK-36885 URL: https://issues.apache.org/jira/browse/SPARK-36885 Project: Spark

[jira] [Commented] (SPARK-36885) Inline type hints for python/pyspark/sql/dataframe.py

[ https://issues.apache.org/jira/browse/SPARK-36885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421863#comment-17421863 ] Takuya Ueshin commented on SPARK-36885: --- I'm working on this. > Inline type hints for

[jira] [Commented] (SPARK-36884) Inline type hints for python/pyspark/sql/session.py

[ https://issues.apache.org/jira/browse/SPARK-36884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421861#comment-17421861 ] Takuya Ueshin commented on SPARK-36884: --- I'm working on this. > Inline type hints for

[jira] [Created] (SPARK-36884) Inline type hints for python/pyspark/sql/session.py

Takuya Ueshin created SPARK-36884: - Summary: Inline type hints for python/pyspark/sql/session.py Key: SPARK-36884 URL: https://issues.apache.org/jira/browse/SPARK-36884 Project: Spark Issue

[jira] [Commented] (SPARK-36845) Inline type hint files

[ https://issues.apache.org/jira/browse/SPARK-36845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421859#comment-17421859 ] dgd_contributor commented on SPARK-36845: - Can i work on this :D ? > Inline type hint files >

[jira] [Commented] (SPARK-36882) Support ILIKE API on Python

[ https://issues.apache.org/jira/browse/SPARK-36882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421858#comment-17421858 ] Apache Spark commented on SPARK-36882: -- User 'yoda-mon' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36882) Support ILIKE API on Python

[ https://issues.apache.org/jira/browse/SPARK-36882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36882: Assignee: (was: Apache Spark) > Support ILIKE API on Python >

[jira] [Assigned] (SPARK-36882) Support ILIKE API on Python

[ https://issues.apache.org/jira/browse/SPARK-36882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36882: Assignee: Apache Spark > Support ILIKE API on Python > --- > >

[jira] [Commented] (SPARK-36883) Upgrade R version to 4.1.1 in CI images

[ https://issues.apache.org/jira/browse/SPARK-36883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421854#comment-17421854 ] Hyukjin Kwon commented on SPARK-36883: -- [~dongjoon], FYI. BTW, I think this could fix the issue in

[jira] [Created] (SPARK-36883) Upgrade R version to 4.1.1 in CI images

Hyukjin Kwon created SPARK-36883: Summary: Upgrade R version to 4.1.1 in CI images Key: SPARK-36883 URL: https://issues.apache.org/jira/browse/SPARK-36883 Project: Spark Issue Type: Test

[jira] [Created] (SPARK-36882) Support ILIKE API on Python

Leona Yoda created SPARK-36882: -- Summary: Support ILIKE API on Python Key: SPARK-36882 URL: https://issues.apache.org/jira/browse/SPARK-36882 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-36752) Support ILIKE by Scala/Java, PySpark and R APIs

[ https://issues.apache.org/jira/browse/SPARK-36752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421852#comment-17421852 ] Hyukjin Kwon commented on SPARK-36752: -- it's totally no problem. I was just checking :-). >

[jira] [Commented] (SPARK-36752) Support ILIKE by Scala/Java, PySpark and R APIs

[ https://issues.apache.org/jira/browse/SPARK-36752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421847#comment-17421847 ] Leona Yoda commented on SPARK-36752: [~hyukjin.kwon] I'm sorry for worrying you ( I took a

[jira] [Commented] (SPARK-36849) Migrate UseStatement to v2 command framework

[ https://issues.apache.org/jira/browse/SPARK-36849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421845#comment-17421845 ] Huaxin Gao commented on SPARK-36849: I just started the test > Migrate UseStatement to v2 command

[jira] [Commented] (SPARK-36849) Migrate UseStatement to v2 command framework

[ https://issues.apache.org/jira/browse/SPARK-36849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421843#comment-17421843 ] dohongdayi commented on SPARK-36849: Hi [~huaxingao], I'm not sure why SparkQA didn't run

[jira] [Resolved] (SPARK-36813) Implement ps.merge_asof

[ https://issues.apache.org/jira/browse/SPARK-36813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-36813. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34053

[jira] [Assigned] (SPARK-36813) Implement ps.merge_asof

[ https://issues.apache.org/jira/browse/SPARK-36813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-36813: Assignee: Takuya Ueshin > Implement ps.merge_asof > --- > >

[jira] [Resolved] (SPARK-36846) Inline most of type hint files under pyspark/sql/pandas folder

[ https://issues.apache.org/jira/browse/SPARK-36846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-36846. -- Fix Version/s: 3.3.0 Assignee: Takuya Ueshin Resolution: Fixed Fixed in

[jira] [Commented] (SPARK-36664) Log time spent waiting for cluster resources

[ https://issues.apache.org/jira/browse/SPARK-36664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421824#comment-17421824 ] John Zhuge commented on SPARK-36664: This can be very useful. For example, we'd like to track how

[jira] [Commented] (SPARK-36862) ERROR CodeGenerator: failed to compile: org.codehaus.commons.compiler.CompileException: File 'generated.java'

[ https://issues.apache.org/jira/browse/SPARK-36862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421823#comment-17421823 ] Jungtaek Lim commented on SPARK-36862: -- No, I meant you're encouraged to paste the "generated" code

[jira] [Resolved] (SPARK-36878) Optimization in PushDownPredicates to push all filters in a single iteration has broken some optimizations in PruneFilter rule

[ https://issues.apache.org/jira/browse/SPARK-36878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Asif resolved SPARK-36878. -- Resolution: Not A Bug Further inputs from my company colleague, who pointed out that there might be some

[jira] [Commented] (SPARK-36881) Inline type hints for python/pyspark/sql/catalog.py

[ https://issues.apache.org/jira/browse/SPARK-36881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421817#comment-17421817 ] Apache Spark commented on SPARK-36881: -- User 'xinrong-databricks' has created a pull request for

[jira] [Assigned] (SPARK-36881) Inline type hints for python/pyspark/sql/catalog.py

[ https://issues.apache.org/jira/browse/SPARK-36881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36881: Assignee: (was: Apache Spark) > Inline type hints for python/pyspark/sql/catalog.py

[jira] [Commented] (SPARK-36881) Inline type hints for python/pyspark/sql/catalog.py

[ https://issues.apache.org/jira/browse/SPARK-36881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421816#comment-17421816 ] Apache Spark commented on SPARK-36881: -- User 'xinrong-databricks' has created a pull request for

[jira] [Assigned] (SPARK-36881) Inline type hints for python/pyspark/sql/catalog.py

[ https://issues.apache.org/jira/browse/SPARK-36881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36881: Assignee: Apache Spark > Inline type hints for python/pyspark/sql/catalog.py >

[jira] [Created] (SPARK-36881) Inline type hints for python/pyspark/sql/catalog.py

Xinrong Meng created SPARK-36881: Summary: Inline type hints for python/pyspark/sql/catalog.py Key: SPARK-36881 URL: https://issues.apache.org/jira/browse/SPARK-36881 Project: Spark Issue

[jira] [Commented] (SPARK-35874) AQE Shuffle should wait for its subqueries to finish before materializing

[ https://issues.apache.org/jira/browse/SPARK-35874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421780#comment-17421780 ] Shardul Mahadik commented on SPARK-35874: - [~dongjoon] Should this be linked in SPARK-33828? >

[jira] [Commented] (SPARK-36871) Migrate CreateViewStatement to v2 command

[ https://issues.apache.org/jira/browse/SPARK-36871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421779#comment-17421779 ] Apache Spark commented on SPARK-36871: -- User 'huaxingao' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36871) Migrate CreateViewStatement to v2 command

[ https://issues.apache.org/jira/browse/SPARK-36871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36871: Assignee: Apache Spark > Migrate CreateViewStatement to v2 command >

[jira] [Commented] (SPARK-36871) Migrate CreateViewStatement to v2 command

[ https://issues.apache.org/jira/browse/SPARK-36871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421778#comment-17421778 ] Apache Spark commented on SPARK-36871: -- User 'huaxingao' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36871) Migrate CreateViewStatement to v2 command

[ https://issues.apache.org/jira/browse/SPARK-36871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36871: Assignee: (was: Apache Spark) > Migrate CreateViewStatement to v2 command >

[jira] [Closed] (SPARK-36869) Spark job fails due to java.io.InvalidClassException: scala.collection.mutable.WrappedArray$ofRef; local class incompatible

[ https://issues.apache.org/jira/browse/SPARK-36869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-36869. - > Spark job fails due to java.io.InvalidClassException: >

[jira] [Resolved] (SPARK-36869) Spark job fails due to java.io.InvalidClassException: scala.collection.mutable.WrappedArray$ofRef; local class incompatible

[ https://issues.apache.org/jira/browse/SPARK-36869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-36869. --- Resolution: Duplicate This is superseded by SPARK-36759 . > Spark job fails due to

[jira] [Resolved] (SPARK-34276) Check the unreleased/unresolved JIRAs/PRs of Parquet 1.11 and 1.12

[ https://issues.apache.org/jira/browse/SPARK-34276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-34276. --- Assignee: Chao Sun Resolution: Done This is superseded by SPARK-36726 by [~csun]. I'm

[jira] [Commented] (SPARK-36880) Inline type hints for python/pyspark/sql/functions.py

[ https://issues.apache.org/jira/browse/SPARK-36880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421602#comment-17421602 ] Apache Spark commented on SPARK-36880: -- User 'xinrong-databricks' has created a pull request for

[jira] [Commented] (SPARK-36880) Inline type hints for python/pyspark/sql/functions.py

[ https://issues.apache.org/jira/browse/SPARK-36880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421603#comment-17421603 ] Apache Spark commented on SPARK-36880: -- User 'xinrong-databricks' has created a pull request for

[jira] [Assigned] (SPARK-36880) Inline type hints for python/pyspark/sql/functions.py

[ https://issues.apache.org/jira/browse/SPARK-36880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36880: Assignee: Apache Spark > Inline type hints for python/pyspark/sql/functions.py >

[jira] [Assigned] (SPARK-36880) Inline type hints for python/pyspark/sql/functions.py

[ https://issues.apache.org/jira/browse/SPARK-36880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36880: Assignee: (was: Apache Spark) > Inline type hints for

[jira] [Created] (SPARK-36880) Inline type hints for python/pyspark/sql/functions.py

Xinrong Meng created SPARK-36880: Summary: Inline type hints for python/pyspark/sql/functions.py Key: SPARK-36880 URL: https://issues.apache.org/jira/browse/SPARK-36880 Project: Spark Issue

[jira] [Created] (SPARK-36879) Support Parquet v2 data page encodings for the vectorized path

Chao Sun created SPARK-36879: Summary: Support Parquet v2 data page encodings for the vectorized path Key: SPARK-36879 URL: https://issues.apache.org/jira/browse/SPARK-36879 Project: Spark

[jira] [Updated] (SPARK-36877) Calling ds.rdd with AQE enabled leads to jobs being run, eventually causing reruns

[ https://issues.apache.org/jira/browse/SPARK-36877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shardul Mahadik updated SPARK-36877: Summary: Calling ds.rdd with AQE enabled leads to jobs being run, eventually causing

[jira] [Updated] (SPARK-36786) SPIP: Improving the compile time performance, by improving a couple of rules, from 24 hrs to under 8 minutes

[ https://issues.apache.org/jira/browse/SPARK-36786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Asif updated SPARK-36786: - Description: h2. Q1. What are you trying to do? Articulate your objectives using absolutely no jargon. The

[jira] [Updated] (SPARK-36786) SPIP: Improving the compile time performance, by improving a couple of rules, from 24 hrs to under 8 minutes

[ https://issues.apache.org/jira/browse/SPARK-36786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Asif updated SPARK-36786: - Description: h2. Q1. What are you trying to do? Articulate your objectives using absolutely no jargon. The

[jira] [Created] (SPARK-36878) Optimization in PushDownPredicates to push all filters in a single iteration has broken some optimizations in PruneFilter rule

Asif created SPARK-36878: Summary: Optimization in PushDownPredicates to push all filters in a single iteration has broken some optimizations in PruneFilter rule Key: SPARK-36878 URL:

[jira] [Commented] (SPARK-36877) Calling ds.rdd with AQE enabled leads to being jobs being run, eventually causing reruns

[ https://issues.apache.org/jira/browse/SPARK-36877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421510#comment-17421510 ] Shardul Mahadik commented on SPARK-36877: - cc: [~cloud_fan] [~mridulm80] > Calling ds.rdd with

[jira] [Created] (SPARK-36877) Calling ds.rdd with AQE enabled leads to being jobs being run, eventually causing reruns

Shardul Mahadik created SPARK-36877: --- Summary: Calling ds.rdd with AQE enabled leads to being jobs being run, eventually causing reruns Key: SPARK-36877 URL: https://issues.apache.org/jira/browse/SPARK-36877

[jira] [Updated] (SPARK-36877) Calling ds.rdd with AQE enabled leads to being jobs being run, eventually causing reruns

[ https://issues.apache.org/jira/browse/SPARK-36877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shardul Mahadik updated SPARK-36877: Attachment: Screen Shot 2021-09-28 at 09.32.20.png > Calling ds.rdd with AQE enabled

[jira] [Assigned] (SPARK-36873) Add provided Guava dependency for network-yarn module

[ https://issues.apache.org/jira/browse/SPARK-36873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-36873: - Assignee: Chao Sun (was: Apache Spark) > Add provided Guava dependency for

[jira] [Updated] (SPARK-36862) ERROR CodeGenerator: failed to compile: org.codehaus.commons.compiler.CompileException: File 'generated.java'

[ https://issues.apache.org/jira/browse/SPARK-36862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Magdalena Pilawska updated SPARK-36862: --- Affects Version/s: (was: 3.1.2) Description: Hi, I am getting the

[jira] [Comment Edited] (SPARK-36862) ERROR CodeGenerator: failed to compile: org.codehaus.commons.compiler.CompileException: File 'generated.java'

[ https://issues.apache.org/jira/browse/SPARK-36862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421374#comment-17421374 ] Magdalena Pilawska edited comment on SPARK-36862 at 9/28/21, 1:11 PM:

[jira] [Commented] (SPARK-36862) ERROR CodeGenerator: failed to compile: org.codehaus.commons.compiler.CompileException: File 'generated.java'

[ https://issues.apache.org/jira/browse/SPARK-36862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421374#comment-17421374 ] Magdalena Pilawska commented on SPARK-36862: I get the physical execution plan as a part of

[jira] [Commented] (SPARK-36848) Migrate ShowCurrentNamespaceStatement to v2 command framework

[ https://issues.apache.org/jira/browse/SPARK-36848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421369#comment-17421369 ] Apache Spark commented on SPARK-36848: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Commented] (SPARK-36849) Migrate UseStatement to v2 command framework

[ https://issues.apache.org/jira/browse/SPARK-36849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421358#comment-17421358 ] Apache Spark commented on SPARK-36849: -- User 'dohongdayi' has created a pull request for this

[jira] [Resolved] (SPARK-36866) Pushdown filters with ANSI interval values to parquet

[ https://issues.apache.org/jira/browse/SPARK-36866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-36866. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34115

[jira] [Commented] (SPARK-36861) Partition columns are overly eagerly parsed as dates

[ https://issues.apache.org/jira/browse/SPARK-36861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421334#comment-17421334 ] Tanel Kiis commented on SPARK-36861: Yes, in 3.1 it is parsed as string. In 3.3 (master) it is

[jira] [Commented] (SPARK-36861) Partition columns are overly eagerly parsed as dates

[ https://issues.apache.org/jira/browse/SPARK-36861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421328#comment-17421328 ] Senthil Kumar commented on SPARK-36861: --- [~tanelk] This is issue not reproducable even in 3.1.2  

[jira] [Updated] (SPARK-36876) Support Dynamic Partition pruning for HiveTableScanExec

[ https://issues.apache.org/jira/browse/SPARK-36876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-36876: -- Description: Support dynamic partition pruning for hive serde scan > Support Dynamic Partition

  1   2   >