[jira] [Commented] (SPARK-43526) when shuffle hash join is enabled, q95 performance deteriorates

2023-05-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17723338#comment-17723338 ] Yuming Wang commented on SPARK-43526: - Why do you prefer shuffle hash join? > when

[jira] [Updated] (SPARK-43461) Skip compiling useless files when making distribution

2023-05-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-43461: Fix Version/s: 3.5.0 > Skip compiling useless files when making distribution > ---

[jira] [Resolved] (SPARK-43524) Memory leak in Spark UI

2023-05-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-43524. - Resolution: Duplicate > Memory leak in Spark UI > --- > > Ke

[jira] [Updated] (SPARK-43521) Support CREATE TABLE LIKE FILE for PARQUET

2023-05-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-43521: Issue Type: New Feature (was: Bug) > Support CREATE TABLE LIKE FILE for PARQUET > ---

[jira] [Updated] (SPARK-43461) Skip compiling useless files when making distribution

2023-05-15 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-43461: Summary: Skip compiling useless files when making distribution (was: Skip compiling javadoc.jar a

[jira] [Updated] (SPARK-43488) bitmap function

2023-05-12 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-43488: Fix Version/s: (was: 3.4.1) > bitmap function > --- > > Key: SPARK

[jira] [Commented] (SPARK-43486) number of files read is incorrect if it is bucket table

2023-05-12 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17722313#comment-17722313 ] Yuming Wang commented on SPARK-43486: - [~panbingkun] Yes, please. > number of files

[jira] [Updated] (SPARK-43486) number of files read is incorrect if it is bucket table

2023-05-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-43486: Description: !screenshot-1.png! > number of files read is incorrect if it is bucket table >

[jira] [Created] (SPARK-43486) number of files read is incorrect if it is bucket table

2023-05-11 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-43486: --- Summary: number of files read is incorrect if it is bucket table Key: SPARK-43486 URL: https://issues.apache.org/jira/browse/SPARK-43486 Project: Spark Issue T

[jira] [Updated] (SPARK-43486) number of files read is incorrect if it is bucket table

2023-05-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-43486: Attachment: screenshot-1.png > number of files read is incorrect if it is bucket table > -

[jira] [Updated] (SPARK-43461) Skip compiling javadoc.jar and sources.jar when making distribution

2023-05-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-43461: Description: -Dmaven.javadoc.skip=true to skip java doc -Dskip=true to skip scala doc. Please see:

[jira] [Updated] (SPARK-43461) Skip compiling javadoc.jar and sources.jar when making distribution

2023-05-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-43461: Summary: Skip compiling javadoc.jar and sources.jar when making distribution (was: Skip compiling

[jira] [Updated] (SPARK-43461) Skip compiling javadoc.jar, sources.jar and test-jar when making distribution

2023-05-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-43461: Summary: Skip compiling javadoc.jar, sources.jar and test-jar when making distribution (was: Skip

[jira] [Updated] (SPARK-43461) Skip make javadoc.jar, sources.jar and test-jar when making distribution

2023-05-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-43461: Summary: Skip make javadoc.jar, sources.jar and test-jar when making distribution (was: Skip make

[jira] [Created] (SPARK-43461) Skip make javadoc.jar, sources.jar and test-jar when make distribution

2023-05-11 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-43461: --- Summary: Skip make javadoc.jar, sources.jar and test-jar when make distribution Key: SPARK-43461 URL: https://issues.apache.org/jira/browse/SPARK-43461 Project: Spark

[jira] [Updated] (SPARK-43425) Add TimestampNTZType to ColumnarBatchRow

2023-05-09 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-43425: Fix Version/s: (was: 3.5.0) (was: 3.4.1) > Add TimestampNTZType to Colu

[jira] [Updated] (SPARK-43383) Add statistics rowCount for LocalRelation

2023-05-06 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-43383: Fix Version/s: (was: 3.5.0) > Add statistics rowCount for LocalRelation >

[jira] [Assigned] (SPARK-43174) Fix SparkSQLCLIDriver completer

2023-04-21 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-43174: --- Assignee: Yuming Wang > Fix SparkSQLCLIDriver completer > --- >

[jira] [Resolved] (SPARK-43174) Fix SparkSQLCLIDriver completer

2023-04-21 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-43174. - Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40838 [https://gith

[jira] [Updated] (SPARK-43142) DSL expressions fail on attribute with special characters

2023-04-21 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-43142: Fix Version/s: (was: 3.5.0) > DSL expressions fail on attribute with special characters >

[jira] [Reopened] (SPARK-43142) DSL expressions fail on attribute with special characters

2023-04-21 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reopened SPARK-43142: - > DSL expressions fail on attribute with special characters > --

[jira] [Commented] (SPARK-43228) Join keys also match PartitioningCollection

2023-04-21 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17714907#comment-17714907 ] Yuming Wang commented on SPARK-43228: - https://github.com/apache/spark/pull/40897 >

[jira] [Created] (SPARK-43228) Join keys also match PartitioningCollection

2023-04-21 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-43228: --- Summary: Join keys also match PartitioningCollection Key: SPARK-43228 URL: https://issues.apache.org/jira/browse/SPARK-43228 Project: Spark Issue Type: Improve

[jira] [Updated] (SPARK-43225) Remove jackson-core-asl and jackson-mapper-asl from pre-built distribution

2023-04-20 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-43225: Summary: Remove jackson-core-asl and jackson-mapper-asl from pre-built distribution (was: Change

[jira] [Created] (SPARK-43225) Change the scope of jackson-mapper-asl from compile to test

2023-04-20 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-43225: --- Summary: Change the scope of jackson-mapper-asl from compile to test Key: SPARK-43225 URL: https://issues.apache.org/jira/browse/SPARK-43225 Project: Spark Is

[jira] [Resolved] (SPARK-43170) The spark sql like statement is pushed down to parquet for execution, but the data cannot be queried

2023-04-18 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-43170. - Resolution: Not A Bug > The spark sql like statement is pushed down to parquet for execution, bu

[jira] [Updated] (SPARK-43170) The spark sql like statement is pushed down to parquet for execution, but the data cannot be queried

2023-04-18 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-43170: Attachment: screenshot-1.png > The spark sql like statement is pushed down to parquet for executio

[jira] [Commented] (SPARK-43170) The spark sql like statement is pushed down to parquet for execution, but the data cannot be queried

2023-04-18 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17713861#comment-17713861 ] Yuming Wang commented on SPARK-43170: - Maybe your partition exists, but there is no

[jira] [Commented] (SPARK-43170) The spark sql like statement is pushed down to parquet for execution, but the data cannot be queried

2023-04-18 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17713858#comment-17713858 ] Yuming Wang commented on SPARK-43170: - I can't reproduce this issue: !image-2023-04

[jira] [Updated] (SPARK-43170) The spark sql like statement is pushed down to parquet for execution, but the data cannot be queried

2023-04-18 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-43170: Attachment: image-2023-04-19-10-59-44-118.png > The spark sql like statement is pushed down to par

[jira] [Commented] (SPARK-43170) The spark sql like statement is pushed down to parquet for execution, but the data cannot be queried

2023-04-18 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17713576#comment-17713576 ] Yuming Wang commented on SPARK-43170: - Why it is {{CachedRDDBuilder}}? {noformat} (2

[jira] [Created] (SPARK-43174) Fix SparkSQLCLIDriver completer

2023-04-18 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-43174: --- Summary: Fix SparkSQLCLIDriver completer Key: SPARK-43174 URL: https://issues.apache.org/jira/browse/SPARK-43174 Project: Spark Issue Type: Bug Compo

[jira] [Commented] (SPARK-43170) The spark sql like statement is pushed down to parquet for execution, but the data cannot be queried

2023-04-18 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17713503#comment-17713503 ] Yuming Wang commented on SPARK-43170: - Have you cached dwm_user_app_action_sum_all?

[jira] [Commented] (SPARK-43163) An exception occurred while hive table join tidb table

2023-04-17 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17713069#comment-17713069 ] Yuming Wang commented on SPARK-43163: - It seems like TiSpark issue. > An exception

[jira] [Commented] (SPARK-43154) Pyspark 3.4 fails when running "pivot" function on a dataframe using the values arguement

2023-04-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17712774#comment-17712774 ] Yuming Wang commented on SPARK-43154: - cc [~gurwls223] > Pyspark 3.4 fails when run

[jira] [Resolved] (SPARK-43139) Bug in INSERT INTO documentation

2023-04-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-43139. - Fix Version/s: 3.5.0 3.4.1 Resolution: Fixed Issue resolved by pull re

[jira] [Assigned] (SPARK-43139) Bug in INSERT INTO documentation

2023-04-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-43139: --- Assignee: Yuming Wang > Bug in INSERT INTO documentation >

[jira] [Commented] (SPARK-43139) Bug in INSERT INTO documentation

2023-04-15 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17712743#comment-17712743 ] Yuming Wang commented on SPARK-43139: - https://github.com/apache/spark/pull/40807 >

[jira] [Commented] (SPARK-40609) Casts types according to bucket info for Equality expression

2023-04-15 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17712663#comment-17712663 ] Yuming Wang commented on SPARK-40609: - https://github.com/apache/spark/pull/40805 >

[jira] [Resolved] (SPARK-43050) Fix construct aggregate expressions by replacing grouping functions

2023-04-14 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-43050. - Fix Version/s: 3.3.3 3.4.1 3.5.0 Assignee: Yuming

[jira] [Resolved] (SPARK-43095) Avoid Once strategy's idempotence is broken for batch: Infer Filters

2023-04-14 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-43095. - Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40742 [https://gith

[jira] [Assigned] (SPARK-43095) Avoid Once strategy's idempotence is broken for batch: Infer Filters

2023-04-14 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-43095: --- Assignee: Yuming Wang > Avoid Once strategy's idempotence is broken for batch: Infer Filter

[jira] [Resolved] (SPARK-42926) Upgrade Parquet to 1.13.0

2023-04-14 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-42926. - Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40555 [https://gith

[jira] [Assigned] (SPARK-42926) Upgrade Parquet to 1.13.0

2023-04-14 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-42926: --- Assignee: Yuming Wang > Upgrade Parquet to 1.13.0 > - > >

[jira] [Resolved] (SPARK-43107) Coalesce buckets in join applied on broadcast join stream side

2023-04-14 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-43107. - Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40756 [https://gith

[jira] [Assigned] (SPARK-43107) Coalesce buckets in join applied on broadcast join stream side

2023-04-14 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-43107: --- Assignee: Yuming Wang > Coalesce buckets in join applied on broadcast join stream side > --

[jira] [Commented] (SPARK-43140) Override computeStats in DummyLeafNode

2023-04-14 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17712315#comment-17712315 ] Yuming Wang commented on SPARK-43140: - https://github.com/apache/spark/pull/40791 >

[jira] [Created] (SPARK-43140) Override computeStats in DummyLeafNode

2023-04-14 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-43140: --- Summary: Override computeStats in DummyLeafNode Key: SPARK-43140 URL: https://issues.apache.org/jira/browse/SPARK-43140 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-43138) ClassNotFound during RDD block replication/migration

2023-04-13 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17712212#comment-17712212 ] Yuming Wang commented on SPARK-43138: - Did you set some config to `com.class.from.us

[jira] [Resolved] (SPARK-43135) Remove `branch-3.2` from `publish_snapshot` GitHub Action job

2023-04-13 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-43135. - Fix Version/s: 3.5.0 Assignee: Dongjoon Hyun Resolution: Fixed Issue resolved by

[jira] [Resolved] (SPARK-42975) Cast result type to timestamp type for string +/- interval

2023-04-12 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-42975. - Resolution: Won't Fix Disable it in ANSI mode. > Cast result type to timestamp type for string

[jira] [Updated] (SPARK-43116) Fix Cast.forceNullable

2023-04-12 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-43116: Description: https://github.com/apache/spark/blob/ac105ccebf5f144f2d506cbe102c362a195afa9a/sql/cat

[jira] [Updated] (SPARK-43116) Fix Cast.forceNullable

2023-04-12 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-43116: Description: https://github.com/apache/spark/blob/ac105ccebf5f144f2d506cbe102c362a195afa9a/sql/cat

[jira] [Created] (SPARK-43116) Fix Cast.forceNullable

2023-04-12 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-43116: --- Summary: Fix Cast.forceNullable Key: SPARK-43116 URL: https://issues.apache.org/jira/browse/SPARK-43116 Project: Spark Issue Type: Bug Components: SQ

[jira] [Updated] (SPARK-43107) Coalesce buckets in join applied on broadcast join stream side

2023-04-12 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-43107: Summary: Coalesce buckets in join applied on broadcast join stream side (was: Coalesce applied on

[jira] [Commented] (SPARK-43107) Coalesce applied on broadcast join stream side

2023-04-12 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17711605#comment-17711605 ] Yuming Wang commented on SPARK-43107: - https://github.com/apache/spark/pull/40756 >

[jira] [Commented] (SPARK-43114) Add interval types to TypeCoercionSuite

2023-04-12 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17711604#comment-17711604 ] Yuming Wang commented on SPARK-43114: - https://github.com/apache/spark/pull/40763 >

[jira] [Created] (SPARK-43114) Add interval types to TypeCoercionSuite

2023-04-12 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-43114: --- Summary: Add interval types to TypeCoercionSuite Key: SPARK-43114 URL: https://issues.apache.org/jira/browse/SPARK-43114 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-43107) Coalesce applied on broadcast join stream side

2023-04-12 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-43107: --- Summary: Coalesce applied on broadcast join stream side Key: SPARK-43107 URL: https://issues.apache.org/jira/browse/SPARK-43107 Project: Spark Issue Type: Impr

[jira] [Resolved] (SPARK-43087) Support coalesce buckets in join in AQE

2023-04-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-43087. - Resolution: Duplicate > Support coalesce buckets in join in AQE > --

[jira] [Created] (SPARK-43095) Avoid Once strategy's idempotence is broken for batch: Infer Filters

2023-04-11 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-43095: --- Summary: Avoid Once strategy's idempotence is broken for batch: Infer Filters Key: SPARK-43095 URL: https://issues.apache.org/jira/browse/SPARK-43095 Project: Spark

[jira] [Commented] (SPARK-43087) Support coalesce buckets in join in AQE

2023-04-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17710952#comment-17710952 ] Yuming Wang commented on SPARK-43087: - https://github.com/apache/spark/pull/40731 >

[jira] [Created] (SPARK-43087) Support coalesce buckets in join in AQE

2023-04-10 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-43087: --- Summary: Support coalesce buckets in join in AQE Key: SPARK-43087 URL: https://issues.apache.org/jira/browse/SPARK-43087 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-40609) Casts types according to bucket info for Equality expression

2023-04-10 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17710080#comment-17710080 ] Yuming Wang commented on SPARK-40609: - {code:scala} import org.apache.spark.benchma

[jira] [Updated] (SPARK-43050) Fix construct aggregate expressions by replacing grouping functions

2023-04-06 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-43050: Description: {code:sql} CREATE TEMPORARY VIEW grouping AS SELECT * FROM VALUES ("1", "2", "3",

[jira] [Commented] (SPARK-43053) Possible logic issue

2023-04-06 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17709536#comment-17709536 ] Yuming Wang commented on SPARK-43053: - It's not a issue. > Possible logic issue > -

[jira] [Created] (SPARK-43050) Fix construct aggregate expressions by replacing grouping functions

2023-04-06 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-43050: --- Summary: Fix construct aggregate expressions by replacing grouping functions Key: SPARK-43050 URL: https://issues.apache.org/jira/browse/SPARK-43050 Project: Spark

[jira] [Updated] (SPARK-42926) Upgrade Parquet to 1.13.0

2023-04-02 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-42926: Summary: Upgrade Parquet to 1.13.0 (was: Upgrade Parquet to 1.12.4) > Upgrade Parquet to 1.13.0 >

[jira] [Created] (SPARK-43000) Do not cast to double type if one side is AnsiIntervalType in BinaryArithmetic

2023-04-01 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-43000: --- Summary: Do not cast to double type if one side is AnsiIntervalType in BinaryArithmetic Key: SPARK-43000 URL: https://issues.apache.org/jira/browse/SPARK-43000 Project:

[jira] [Updated] (SPARK-42991) Disable string type +/- interval in ANSI mode

2023-03-31 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-42991: Summary: Disable string type +/- interval in ANSI mode (was: Disable string +/- interval in ANSI

[jira] [Commented] (SPARK-42991) Disable string +/- interval in ANSI mode

2023-03-31 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17707158#comment-17707158 ] Yuming Wang commented on SPARK-42991: - https://github.com/apache/spark/pull/40616 >

[jira] [Created] (SPARK-42991) Disable string +/- interval in ANSI mode

2023-03-30 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-42991: --- Summary: Disable string +/- interval in ANSI mode Key: SPARK-42991 URL: https://issues.apache.org/jira/browse/SPARK-42991 Project: Spark Issue Type: Improvemen

[jira] [Created] (SPARK-42975) Cast result type to timestamp type for string +/- interval

2023-03-29 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-42975: --- Summary: Cast result type to timestamp type for string +/- interval Key: SPARK-42975 URL: https://issues.apache.org/jira/browse/SPARK-42975 Project: Spark Issu

[jira] [Commented] (SPARK-42932) Spark 3.3.2, with hadoop3, Error with java.io.IOException: Mkdirs failed to create file

2023-03-29 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17706702#comment-17706702 ] Yuming Wang commented on SPARK-42932: - I can't reproduce this issue. Maybe it's your

[jira] [Resolved] (SPARK-40610) Spark fall back to use getPartitions instead of getPartitionsByFilter when date_add functions used in where clause

2023-03-29 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-40610. - Resolution: Won't Fix > Spark fall back to use getPartitions instead of getPartitionsByFilter wh

[jira] [Commented] (SPARK-40610) Spark fall back to use getPartitions instead of getPartitionsByFilter when date_add functions used in where clause

2023-03-29 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17706665#comment-17706665 ] Yuming Wang commented on SPARK-40610: - Users canĀ {{set spark.sql.legacy.typeCoercion

[jira] [Reopened] (SPARK-42832) Remove repartition if it is the child of LocalLimit

2023-03-28 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reopened SPARK-42832: - Assignee: (was: Yuming Wang) > Remove repartition if it is the child of LocalLimit > -

[jira] [Updated] (SPARK-42832) Remove repartition if it is the child of LocalLimit

2023-03-28 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-42832: Fix Version/s: (was: 3.5.0) > Remove repartition if it is the child of LocalLimit > --

[jira] [Resolved] (SPARK-42832) Remove repartition if it is the child of LocalLimit

2023-03-28 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-42832. - Resolution: Won't Do > Remove repartition if it is the child of LocalLimit > ---

[jira] [Updated] (SPARK-42935) Optimze shuffle for union spark plan

2023-03-27 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-42935: Fix Version/s: (was: 3.5.0) > Optimze shuffle for union spark plan > -

[jira] [Created] (SPARK-42926) Upgrade Parquet to 1.12.4

2023-03-26 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-42926: --- Summary: Upgrade Parquet to 1.12.4 Key: SPARK-42926 URL: https://issues.apache.org/jira/browse/SPARK-42926 Project: Spark Issue Type: Improvement Com

[jira] [Resolved] (SPARK-33307) Refactor GROUPING ANALYTICS

2023-03-21 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-33307. - Fix Version/s: 3.2.0 Assignee: angerszhu Resolution: Fixed > Refactor GROUPING A

[jira] [Created] (SPARK-42868) Support eliminate sorts in AQE Optimizer

2023-03-20 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-42868: --- Summary: Support eliminate sorts in AQE Optimizer Key: SPARK-42868 URL: https://issues.apache.org/jira/browse/SPARK-42868 Project: Spark Issue Type: Improvemen

[jira] (SPARK-42760) The partition of result data frame of join is always 1

2023-03-17 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42760 ] Yuming Wang deleted comment on SPARK-42760: - was (Author: apachespark): User '1511351836' has created a pull request for this issue: https://github.com/apache/spark/pull/40380 > The partitio

[jira] [Created] (SPARK-42832) Remove repartition if it is the child of LocalLimit

2023-03-16 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-42832: --- Summary: Remove repartition if it is the child of LocalLimit Key: SPARK-42832 URL: https://issues.apache.org/jira/browse/SPARK-42832 Project: Spark Issue Type:

[jira] [Commented] (SPARK-42804) when target table format is textfile using `insert into select` will got error

2023-03-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17701006#comment-17701006 ] Yuming Wang commented on SPARK-42804: - I can't reproduce it. Did you set any configs

[jira] [Commented] (SPARK-42804) when target table format is textfile using `insert into select` will got error

2023-03-15 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700984#comment-17700984 ] Yuming Wang commented on SPARK-42804: - How to reproduce? > when target table format

[jira] [Comment Edited] (SPARK-42513) Push down topK through join

2023-03-15 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700945#comment-17700945 ] Yuming Wang edited comment on SPARK-42513 at 3/16/23 3:12 AM:

[jira] [Commented] (SPARK-42513) Push down topK through join

2023-03-15 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700945#comment-17700945 ] Yuming Wang commented on SPARK-42513: - TiDB: {code:sql} create table t1(a int, b in

[jira] [Resolved] (SPARK-42804) when target table format is textfile using `insert into select` will got error

2023-03-15 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-42804. - Resolution: Invalid It looks like this is a problem of your metastore. > when target table form

[jira] [Updated] (SPARK-42789) Rewrite multiple GetJsonObjects to a JsonTuple if their json expression is the same

2023-03-14 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-42789: Description: Benchmark result: {noformat} Running benchmark: Benchmark rewrite GetJsonObjects R

[jira] [Updated] (SPARK-42789) Rewrite multiple GetJsonObjects to a JsonTuple if their json expression is the same

2023-03-14 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-42789: Summary: Rewrite multiple GetJsonObjects to a JsonTuple if their json expression is the same (was

[jira] [Updated] (SPARK-42789) rewrites multiple GetJsonObjects to a JsonTuple if their json expression is the same

2023-03-14 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-42789: Description: Benchmark result: {noformat} Running benchmark: Benchmark rewrite GetJsonObjects R

[jira] [Created] (SPARK-42789) rewrites multiple GetJsonObjects to a JsonTuple if their json expression is the same

2023-03-14 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-42789: --- Summary: rewrites multiple GetJsonObjects to a JsonTuple if their json expression is the same Key: SPARK-42789 URL: https://issues.apache.org/jira/browse/SPARK-42789 Pr

[jira] [Assigned] (SPARK-42741) Do not unwrap casts in binary comparison when literal is null

2023-03-14 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-42741: --- Assignee: Yuming Wang > Do not unwrap casts in binary comparison when literal is null > ---

[jira] [Resolved] (SPARK-42741) Do not unwrap casts in binary comparison when literal is null

2023-03-14 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-42741. - Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40360 [https://gith

[jira] [Updated] (SPARK-42782) Port the tests for get_json_object from the Apache Hive project

2023-03-14 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-42782: Summary: Port the tests for get_json_object from the Apache Hive project (was: Port TestUDFJson f

[jira] [Created] (SPARK-42782) Port TestUDFJson from Hive

2023-03-14 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-42782: --- Summary: Port TestUDFJson from Hive Key: SPARK-42782 URL: https://issues.apache.org/jira/browse/SPARK-42782 Project: Spark Issue Type: Improvement Co

[jira] [Updated] (SPARK-42597) Support unwrap date type to timestamp type

2023-03-13 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-42597: Summary: Support unwrap date type to timestamp type (was: UnwrapCastInBinaryComparison support un

[jira] [Resolved] (SPARK-42597) UnwrapCastInBinaryComparison support unwrap timestamp type

2023-03-13 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-42597. - Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40190 [https://gith

<    1   2   3   4   5   6   7   8   9   10   >