[jira] [Created] (SPARK-34258) scala.reflect.internal.Symbols$CyclicReference: illegal cyclic reference involving type BuilderType

2021-01-26 Thread wuyi (Jira)
wuyi created SPARK-34258: Summary: scala.reflect.internal.Symbols$CyclicReference: illegal cyclic reference involving type BuilderType Key: SPARK-34258 URL: https://issues.apache.org/jira/browse/SPARK-34258 P

[jira] [Commented] (SPARK-34255) DataSource V2: support static partitioning on required distribution and ordering

2021-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17272633#comment-17272633 ] Apache Spark commented on SPARK-34255: -- User 'HeartSaVioR' has created a pull reque

[jira] [Assigned] (SPARK-34255) DataSource V2: support static partitioning on required distribution and ordering

2021-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34255: Assignee: (was: Apache Spark) > DataSource V2: support static partitioning on require

[jira] [Assigned] (SPARK-34255) DataSource V2: support static partitioning on required distribution and ordering

2021-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34255: Assignee: Apache Spark > DataSource V2: support static partitioning on required distribut

[jira] [Resolved] (SPARK-34231) AvroSuite has test failure when run from IDE due to bad loading of resource file

2021-01-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-34231. -- Fix Version/s: 3.1.2 3.0.2 2.4.8 Assignee: Erik

[jira] [Assigned] (SPARK-34231) AvroSuite has test failure when run from IDE due to bad loading of resource file

2021-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34231: Assignee: Apache Spark > AvroSuite has test failure when run from IDE due to bad loading

[jira] [Assigned] (SPARK-34231) AvroSuite has test failure when run from IDE due to bad loading of resource file

2021-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34231: Assignee: Apache Spark > AvroSuite has test failure when run from IDE due to bad loading

[jira] [Commented] (SPARK-34231) AvroSuite has test failure when run from IDE due to bad loading of resource file

2021-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17272625#comment-17272625 ] Apache Spark commented on SPARK-34231: -- User 'xkrogen' has created a pull request f

[jira] [Assigned] (SPARK-34231) AvroSuite has test failure when run from IDE due to bad loading of resource file

2021-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34231: Assignee: (was: Apache Spark) > AvroSuite has test failure when run from IDE due to b

[jira] [Commented] (SPARK-34231) AvroSuite has test failure when run from IDE due to bad loading of resource file

2021-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17272626#comment-17272626 ] Apache Spark commented on SPARK-34231: -- User 'xkrogen' has created a pull request f

[jira] [Updated] (SPARK-34231) [AVRO][TEST] AvroSuite has test failure when run from IDE due to bad loading of resource file

2021-01-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-34231: - Component/s: (was: Input/Output) SQL > [AVRO][TEST] AvroSuite has test fail

[jira] [Updated] (SPARK-34231) AvroSuite has test failure when run from IDE due to bad loading of resource file

2021-01-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-34231: - Summary: AvroSuite has test failure when run from IDE due to bad loading of resource file (was:

[jira] [Updated] (SPARK-34257) Improve performance for last_value ignore nulls

2021-01-26 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-34257: --- Description: The current implement of `last_value` will execute `updateExpressions` multiple times.

[jira] [Updated] (SPARK-34257) Improve performance for last_value ignore nulls

2021-01-26 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-34257: --- Description: The current implement > Improve performance for last_value ignore nulls > -

[jira] [Commented] (SPARK-34257) Improve performance for last_value ignore nulls

2021-01-26 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17272620#comment-17272620 ] jiaan.geng commented on SPARK-34257: I'm working on. > Improve performance for last

[jira] [Created] (SPARK-34257) Improve performance for last_value ignore nulls

2021-01-26 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-34257: -- Summary: Improve performance for last_value ignore nulls Key: SPARK-34257 URL: https://issues.apache.org/jira/browse/SPARK-34257 Project: Spark Issue Type: Sub-t

[jira] [Resolved] (SPARK-34251) TRUNCATE TABLE resets stats for non-empty v1 table

2021-01-26 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-34251. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31350 [https://gith

[jira] [Assigned] (SPARK-34251) TRUNCATE TABLE resets stats for non-empty v1 table

2021-01-26 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-34251: --- Assignee: Maxim Gekk > TRUNCATE TABLE resets stats for non-empty v1 table > ---

[jira] [Assigned] (SPARK-34233) NPE for char padding in binary comparison

2021-01-26 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-34233: --- Assignee: Kent Yao > NPE for char padding in binary comparison > --

[jira] [Resolved] (SPARK-34233) NPE for char padding in binary comparison

2021-01-26 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-34233. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 31336 [https://gith

[jira] [Assigned] (SPARK-34256) VectorSlicer refine numFeatures checking and toString method

2021-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34256: Assignee: (was: Apache Spark) > VectorSlicer refine numFeatures checking and toString

[jira] [Assigned] (SPARK-34256) VectorSlicer refine numFeatures checking and toString method

2021-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34256: Assignee: Apache Spark > VectorSlicer refine numFeatures checking and toString method > -

[jira] [Commented] (SPARK-34256) VectorSlicer refine numFeatures checking and toString method

2021-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17272612#comment-17272612 ] Apache Spark commented on SPARK-34256: -- User 'zhengruifeng' has created a pull requ

[jira] [Created] (SPARK-34256) VectorSlicer refine numFeatures checking and toString method

2021-01-26 Thread zhengruifeng (Jira)
zhengruifeng created SPARK-34256: Summary: VectorSlicer refine numFeatures checking and toString method Key: SPARK-34256 URL: https://issues.apache.org/jira/browse/SPARK-34256 Project: Spark

[jira] [Created] (SPARK-34255) DataSource V2: support static partitioning on required distribution and ordering

2021-01-26 Thread Jungtaek Lim (Jira)
Jungtaek Lim created SPARK-34255: Summary: DataSource V2: support static partitioning on required distribution and ordering Key: SPARK-34255 URL: https://issues.apache.org/jira/browse/SPARK-34255 Proj

[jira] [Commented] (SPARK-34254) Document CREATE EXTERNAL datasource TABLE

2021-01-26 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17272599#comment-17272599 ] Huaxin Gao commented on SPARK-34254: [~hyukjin.kwon], this is not urgent, right? I d

[jira] [Updated] (SPARK-34254) Document CREATE EXTERNAL datasource TABLE

2021-01-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-34254: - Affects Version/s: (was: 3.2.0) 3.1.0 > Document CREATE EXTERNAL data

[jira] [Updated] (SPARK-34254) Document CREATE EXTERNAL datasource TABLE

2021-01-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-34254: - Description: We should mention what EXTERNAL means in CREAT TABLE data source one as well at ht

[jira] [Updated] (SPARK-34254) Document CREATE EXTERNAL datasource TABLE

2021-01-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-34254: - Summary: Document CREATE EXTERNAL datasource TABLE (was: Document CREATE EXTERNAL datasource TA

[jira] [Updated] (SPARK-34254) Document CREATE EXTERNAL datasource TABLE with LOCATION

2021-01-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-34254: - Summary: Document CREATE EXTERNAL datasource TABLE with LOCATION (was: Document CREATE EXTERNAL

[jira] [Commented] (SPARK-34254) Document CREATE EXTERNAL TABLE with LOCATION

2021-01-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17272594#comment-17272594 ] Hyukjin Kwon commented on SPARK-34254: -- [~huaxingao], would you mind taking a look

[jira] [Updated] (SPARK-34254) Document CREATE EXTERNAL TABLE with LOCATION

2021-01-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-34254: - Component/s: SQL > Document CREATE EXTERNAL TABLE with LOCATION > --

[jira] [Created] (SPARK-34254) Document CREATE EXTERNAL TABLE with LOCATION

2021-01-26 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-34254: Summary: Document CREATE EXTERNAL TABLE with LOCATION Key: SPARK-34254 URL: https://issues.apache.org/jira/browse/SPARK-34254 Project: Spark Issue Type: Docu

[jira] [Commented] (SPARK-34052) A cached view should become invalid after a table is dropped

2021-01-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17272568#comment-17272568 ] Hyukjin Kwon commented on SPARK-34052: -- [~csun] Yeah, seems properly added to branc

[jira] [Resolved] (SPARK-34236) v2 Overwrite w/ null static partition raise Cannot translate expression to source filter: null

2021-01-26 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-34236. - Fix Version/s: 3.1.1 Resolution: Fixed Issue resolved by pull request 31339 [https://gith

[jira] [Assigned] (SPARK-34236) v2 Overwrite w/ null static partition raise Cannot translate expression to source filter: null

2021-01-26 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-34236: --- Assignee: Kent Yao > v2 Overwrite w/ null static partition raise Cannot translate expressio

[jira] [Resolved] (SPARK-34189) w2v findSynonyms optimization

2021-01-26 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng resolved SPARK-34189. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31276 [https://gi

[jira] [Assigned] (SPARK-34189) w2v findSynonyms optimization

2021-01-26 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng reassigned SPARK-34189: Assignee: zhengruifeng (was: Apache Spark) > w2v findSynonyms optimization > ---

[jira] [Resolved] (SPARK-34220) BucketedRandomProjectionLSH transform opt

2021-01-26 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng resolved SPARK-34220. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31313 [https://gi

[jira] [Assigned] (SPARK-34220) BucketedRandomProjectionLSH transform opt

2021-01-26 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng reassigned SPARK-34220: Assignee: zhengruifeng > BucketedRandomProjectionLSH transform opt >

[jira] [Assigned] (SPARK-34253) Object hash aggregate should not fallback if no more input rows

2021-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34253: Assignee: (was: Apache Spark) > Object hash aggregate should not fallback if no more

[jira] [Assigned] (SPARK-34253) Object hash aggregate should not fallback if no more input rows

2021-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34253: Assignee: Apache Spark > Object hash aggregate should not fallback if no more input rows

[jira] [Commented] (SPARK-34253) Object hash aggregate should not fallback if no more input rows

2021-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17272495#comment-17272495 ] Apache Spark commented on SPARK-34253: -- User 'c21' has created a pull request for t

[jira] [Created] (SPARK-34253) Object hash aggregate should not fallback if no more input rows

2021-01-26 Thread Cheng Su (Jira)
Cheng Su created SPARK-34253: Summary: Object hash aggregate should not fallback if no more input rows Key: SPARK-34253 URL: https://issues.apache.org/jira/browse/SPARK-34253 Project: Spark Issu

[jira] [Updated] (SPARK-34212) For parquet table, after changing the precision and scale of decimal type in hive, spark reads incorrect value

2021-01-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-34212: -- Fix Version/s: (was: 3.1.0) 3.1.1 > For parquet table, after changing t

[jira] [Resolved] (SPARK-34212) For parquet table, after changing the precision and scale of decimal type in hive, spark reads incorrect value

2021-01-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-34212. --- Fix Version/s: 2.4.8 3.0.2 3.1.0 Resolution: Fix

[jira] [Assigned] (SPARK-34212) For parquet table, after changing the precision and scale of decimal type in hive, spark reads incorrect value

2021-01-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-34212: - Assignee: Dongjoon Hyun > For parquet table, after changing the precision and scale of

[jira] [Resolved] (SPARK-34244) Remove the Scala function version of regexp_extract_all

2021-01-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-34244. --- Fix Version/s: 3.1.1 Resolution: Fixed Issue resolved by pull request 31346 [https://

[jira] [Assigned] (SPARK-34244) Remove the Scala function version of regexp_extract_all

2021-01-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-34244: - Assignee: jiaan.geng > Remove the Scala function version of regexp_extract_all > --

[jira] [Commented] (SPARK-34252) View subqueries in aggregate's grouping expression fail during the analysis check

2021-01-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17272437#comment-17272437 ] Dongjoon Hyun commented on SPARK-34252: --- Hi, [~imback82]. I cannot reproduce the e

[jira] [Commented] (SPARK-27196) Beginning offset 115204574 is after the ending offset 115204516 for topic

2021-01-26 Thread Mustafa Aksakal (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17272431#comment-17272431 ] Mustafa Aksakal commented on SPARK-27196: - Is there any update? I have the same

[jira] [Issue Comment Deleted] (SPARK-27196) Beginning offset 115204574 is after the ending offset 115204516 for topic

2021-01-26 Thread Mustafa Aksakal (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mustafa Aksakal updated SPARK-27196: Comment: was deleted (was: Is there any update? I have the same issue for a long time and

[jira] [Assigned] (SPARK-34252) View subqueries in aggregate's grouping expression fail during the analysis check

2021-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34252: Assignee: (was: Apache Spark) > View subqueries in aggregate's grouping expression fa

[jira] [Assigned] (SPARK-34252) View subqueries in aggregate's grouping expression fail during the analysis check

2021-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34252: Assignee: Apache Spark > View subqueries in aggregate's grouping expression fail during t

[jira] [Commented] (SPARK-34252) View subqueries in aggregate's grouping expression fail during the analysis check

2021-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17272389#comment-17272389 ] Apache Spark commented on SPARK-34252: -- User 'imback82' has created a pull request

[jira] [Commented] (SPARK-34052) A cached view should become invalid after a table is dropped

2021-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17272382#comment-17272382 ] Apache Spark commented on SPARK-34052: -- User 'sunchao' has created a pull request f

[jira] [Commented] (SPARK-34052) A cached view should become invalid after a table is dropped

2021-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17272381#comment-17272381 ] Apache Spark commented on SPARK-34052: -- User 'sunchao' has created a pull request f

[jira] [Commented] (SPARK-34052) A cached view should become invalid after a table is dropped

2021-01-26 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17272333#comment-17272333 ] Chao Sun commented on SPARK-34052: -- [~hyukjin.kwon] [~cloud_fan] do you think we should

[jira] [Updated] (SPARK-34252) View subqueries in aggregate's grouping expression fail during the analysis check

2021-01-26 Thread Terry Kim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Terry Kim updated SPARK-34252: -- Description: To repro: {code:java} sql("create temporary view ta(a, b) as select 1, 2") sql("create t

[jira] [Updated] (SPARK-34252) View subqueries in aggregate's grouping expression fail during the analysis check

2021-01-26 Thread Terry Kim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Terry Kim updated SPARK-34252: -- Summary: View subqueries in aggregate's grouping expression fail during the analysis check (was: Subq

[jira] [Commented] (SPARK-34252) View subqueries in aggregate's grouping expression fail during the analysis check

2021-01-26 Thread Terry Kim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17272295#comment-17272295 ] Terry Kim commented on SPARK-34252: --- [~cloud_fan], [~dongjoon] Would this be a blocker

[jira] [Updated] (SPARK-34252) Subquery (views) in aggregate's grouping expression fails the analysis check

2021-01-26 Thread Terry Kim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Terry Kim updated SPARK-34252: -- Summary: Subquery (views) in aggregate's grouping expression fails the analysis check (was: Subquery

[jira] [Commented] (SPARK-34252) Subquery in aggregate's grouping expression fails the analysis check

2021-01-26 Thread Terry Kim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17272294#comment-17272294 ] Terry Kim commented on SPARK-34252: --- I am working on the fix. > Subquery in aggregate

[jira] [Created] (SPARK-34252) Subquery in aggregate's grouping expression fails the analysis check

2021-01-26 Thread Terry Kim (Jira)
Terry Kim created SPARK-34252: - Summary: Subquery in aggregate's grouping expression fails the analysis check Key: SPARK-34252 URL: https://issues.apache.org/jira/browse/SPARK-34252 Project: Spark

[jira] [Assigned] (SPARK-34251) TRUNCATE TABLE resets stats for non-empty v1 table

2021-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34251: Assignee: (was: Apache Spark) > TRUNCATE TABLE resets stats for non-empty v1 table >

[jira] [Commented] (SPARK-34251) TRUNCATE TABLE resets stats for non-empty v1 table

2021-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17272285#comment-17272285 ] Apache Spark commented on SPARK-34251: -- User 'MaxGekk' has created a pull request f

[jira] [Assigned] (SPARK-34251) TRUNCATE TABLE resets stats for non-empty v1 table

2021-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34251: Assignee: Apache Spark > TRUNCATE TABLE resets stats for non-empty v1 table > ---

[jira] [Commented] (SPARK-26961) Found Java-level deadlock in Spark Driver

2021-01-26 Thread Ajith S (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17272272#comment-17272272 ] Ajith S commented on SPARK-26961: - Scala issue : https://github.com/scala/bug/issues/114

[jira] [Commented] (SPARK-34241) Make Command's producedAttribute as it's output

2021-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17272242#comment-17272242 ] Apache Spark commented on SPARK-34241: -- User 'xkrogen' has created a pull request f

[jira] [Assigned] (SPARK-32041) Exchange reuse won't work in cases when DPP, subqueries are involved

2021-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32041: Assignee: (was: Apache Spark) > Exchange reuse won't work in cases when DPP, subqueri

[jira] [Assigned] (SPARK-32041) Exchange reuse won't work in cases when DPP, subqueries are involved

2021-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32041: Assignee: Apache Spark > Exchange reuse won't work in cases when DPP, subqueries are invo

[jira] [Created] (SPARK-34251) TRUNCATE TABLE resets stats for non-empty v1 table

2021-01-26 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-34251: -- Summary: TRUNCATE TABLE resets stats for non-empty v1 table Key: SPARK-34251 URL: https://issues.apache.org/jira/browse/SPARK-34251 Project: Spark Issue Type: Bu

[jira] [Commented] (SPARK-34251) TRUNCATE TABLE resets stats for non-empty v1 table

2021-01-26 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17272231#comment-17272231 ] Maxim Gekk commented on SPARK-34251: I am working on bug fix. > TRUNCATE TABLE rese

[jira] [Updated] (SPARK-33726) Duplicate field names causes wrong answers during aggregation

2021-01-26 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-33726: Fix Version/s: 2.4.8 > Duplicate field names causes wrong answers during aggregation > ---

[jira] [Updated] (SPARK-34246) New type coercion syntax rules in ANSI mode

2021-01-26 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-34246: --- Summary: New type coercion syntax rules in ANSI mode (was: New implicit cast syntax rules i

[jira] [Created] (SPARK-34250) Add rule WindowFrameCoercion into ANSI implicit cast rules

2021-01-26 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-34250: -- Summary: Add rule WindowFrameCoercion into ANSI implicit cast rules Key: SPARK-34250 URL: https://issues.apache.org/jira/browse/SPARK-34250 Project: Spark

[jira] [Created] (SPARK-34248) Allow implicit casting string literal to other data types under ANSI mode

2021-01-26 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-34248: -- Summary: Allow implicit casting string literal to other data types under ANSI mode Key: SPARK-34248 URL: https://issues.apache.org/jira/browse/SPARK-34248 Project

[jira] [Created] (SPARK-34249) Add documentation for ANSI implicit cast rules

2021-01-26 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-34249: -- Summary: Add documentation for ANSI implicit cast rules Key: SPARK-34249 URL: https://issues.apache.org/jira/browse/SPARK-34249 Project: Spark Issue Type

[jira] [Created] (SPARK-34247) Add end-to-end tests for ANSI implicit cast rules

2021-01-26 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-34247: -- Summary: Add end-to-end tests for ANSI implicit cast rules Key: SPARK-34247 URL: https://issues.apache.org/jira/browse/SPARK-34247 Project: Spark Issue T

[jira] [Commented] (SPARK-34246) New implicit cast syntax rules in ANSI mode

2021-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17272182#comment-17272182 ] Apache Spark commented on SPARK-34246: -- User 'gengliangwang' has created a pull req

[jira] [Assigned] (SPARK-34246) New implicit cast syntax rules in ANSI mode

2021-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34246: Assignee: Gengliang Wang (was: Apache Spark) > New implicit cast syntax rules in ANSI mo

[jira] [Assigned] (SPARK-34246) New implicit cast syntax rules in ANSI mode

2021-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34246: Assignee: Apache Spark (was: Gengliang Wang) > New implicit cast syntax rules in ANSI mo

[jira] [Resolved] (SPARK-34215) Keep table cached after truncation

2021-01-26 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-34215. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31308 [https://gith

[jira] [Assigned] (SPARK-34215) Keep table cached after truncation

2021-01-26 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-34215: --- Assignee: Maxim Gekk > Keep table cached after truncation > ---

[jira] [Commented] (SPARK-34245) Master may not remove the finished executor when Worker fails to send ExecutorStateChanged

2021-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17272170#comment-17272170 ] Apache Spark commented on SPARK-34245: -- User 'Ngone51' has created a pull request f

[jira] [Assigned] (SPARK-34245) Master may not remove the finished executor when Worker fails to send ExecutorStateChanged

2021-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34245: Assignee: Apache Spark > Master may not remove the finished executor when Worker fails to

[jira] [Assigned] (SPARK-34245) Master may not remove the finished executor when Worker fails to send ExecutorStateChanged

2021-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34245: Assignee: (was: Apache Spark) > Master may not remove the finished executor when Work

[jira] [Updated] (SPARK-34246) New implicit cast syntax rules in ANSI mode

2021-01-26 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-34246: --- Description: Add new implicit cast syntax rules in ANSI mode. In Spark ANSI mode, the type c

[jira] [Created] (SPARK-34246) New implicit cast syntax rules in ANSI mode

2021-01-26 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-34246: -- Summary: New implicit cast syntax rules in ANSI mode Key: SPARK-34246 URL: https://issues.apache.org/jira/browse/SPARK-34246 Project: Spark Issue Type: N

[jira] [Assigned] (SPARK-34241) Make Command's producedAttribute as it's output

2021-01-26 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-34241: --- Assignee: angerszhu > Make Command's producedAttribute as it's output > ---

[jira] [Resolved] (SPARK-34241) Make Command's producedAttribute as it's output

2021-01-26 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-34241. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31342 [https://gith

[jira] [Resolved] (SPARK-34026) DataSource V2: Inject repartition and sort nodes to satisfy required distribution and ordering

2021-01-26 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-34026. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31083 [https://gith

[jira] [Assigned] (SPARK-34026) DataSource V2: Inject repartition and sort nodes to satisfy required distribution and ordering

2021-01-26 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-34026: --- Assignee: Anton Okolnychyi > DataSource V2: Inject repartition and sort nodes to satisfy re

[jira] [Created] (SPARK-34245) Master may not remove the finished executor when Worker fails to send ExecutorStateChanged

2021-01-26 Thread wuyi (Jira)
wuyi created SPARK-34245: Summary: Master may not remove the finished executor when Worker fails to send ExecutorStateChanged Key: SPARK-34245 URL: https://issues.apache.org/jira/browse/SPARK-34245 Project: S

[jira] [Updated] (SPARK-34224) Ensure all resource opened by `Source.fromXXX` method are closed

2021-01-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-34224: - Fix Version/s: 3.1.2 > Ensure all resource opened by `Source.fromXXX` method are closed > --

[jira] [Resolved] (SPARK-34224) Ensure all resource opened by `Source.fromXXX` method are closed

2021-01-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-34224. -- Fix Version/s: 3.0.2 Resolution: Fixed Issue resolved by pull request 31344 [https://gi

[jira] [Assigned] (SPARK-34224) Ensure all resource opened by `Source.fromXXX` method are closed

2021-01-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-34224: Assignee: Yang Jie > Ensure all resource opened by `Source.fromXXX` method are closed > -

[jira] [Commented] (SPARK-34212) For parquet table, after changing the precision and scale of decimal type in hive, spark reads incorrect value

2021-01-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17272128#comment-17272128 ] Apache Spark commented on SPARK-34212: -- User 'cloud-fan' has created a pull request

[jira] [Resolved] (SPARK-34214) Expose regexp_extract_all to PySpark

2021-01-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-34214. -- Resolution: Won't Fix > Expose regexp_extract_all to PySpark > ---

[jira] [Commented] (SPARK-32041) Exchange reuse won't work in cases when DPP, subqueries are involved

2021-01-26 Thread Peter Toth (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17272102#comment-17272102 ] Peter Toth commented on SPARK-32041: Let me reopen this ticket as this is not a dupl

  1   2   >