[jira] [Created] (SPARK-38326) aditya

Vallepu Durga Aditya created SPARK-38326: Summary: aditya Key: SPARK-38326 URL: https://issues.apache.org/jira/browse/SPARK-38326 Project: Spark Issue Type: New Feature

[jira] [Resolved] (SPARK-38316) Fix SQLViewSuite/TriggerAvailableNowSuite/UnwrapCastInBinaryComparisonSuite/UnwrapCastInComparisonEndToEndSuite under ANSI mode

[ https://issues.apache.org/jira/browse/SPARK-38316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-38316. Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35652

[jira] [Resolved] (SPARK-38322) Support query stage show runtime statistics in formatted explain mode

[ https://issues.apache.org/jira/browse/SPARK-38322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-38322. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35658

[jira] [Assigned] (SPARK-38322) Support query stage show runtime statistics in formatted explain mode

[ https://issues.apache.org/jira/browse/SPARK-38322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-38322: --- Assignee: XiDuo You > Support query stage show runtime statistics in formatted explain

[jira] [Resolved] (SPARK-38317) Encoding of java.time.Period always results in "INTERVAL '0-0' YEAR TO MONTH"

[ https://issues.apache.org/jira/browse/SPARK-38317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-38317. -- Resolution: Not A Problem > Encoding of java.time.Period always results in "INTERVAL '0-0'

[jira] [Commented] (SPARK-38317) Encoding of java.time.Period always results in "INTERVAL '0-0' YEAR TO MONTH"

[ https://issues.apache.org/jira/browse/SPARK-38317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17497926#comment-17497926 ] Max Gekk commented on SPARK-38317: -- This is the expected behavior, Spark truncates java.time.Period to

[jira] [Assigned] (SPARK-38189) Support priority scheduling (Introduce priorityClass) with volcano implementations

[ https://issues.apache.org/jira/browse/SPARK-38189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38189: Assignee: Apache Spark > Support priority scheduling (Introduce priorityClass) with

[jira] [Commented] (SPARK-38189) Support priority scheduling (Introduce priorityClass) with volcano implementations

[ https://issues.apache.org/jira/browse/SPARK-38189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17497924#comment-17497924 ] Apache Spark commented on SPARK-38189: -- User 'Yikun' has created a pull request for this issue:

[jira] [Assigned] (SPARK-38189) Support priority scheduling (Introduce priorityClass) with volcano implementations

[ https://issues.apache.org/jira/browse/SPARK-38189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38189: Assignee: (was: Apache Spark) > Support priority scheduling (Introduce

[jira] [Updated] (SPARK-38323) Support the hidden file metadata in Streaming

[ https://issues.apache.org/jira/browse/SPARK-38323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaohua Zhao updated SPARK-38323: Description: Currently, querying the hidden file metadata struct `_metadata` will fail with

[jira] [Commented] (SPARK-38325) ANSI mode: avoid potential runtime error in HashJoin.extractKeyExprAt()

[ https://issues.apache.org/jira/browse/SPARK-38325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17497919#comment-17497919 ] Apache Spark commented on SPARK-38325: -- User 'gengliangwang' has created a pull request for this

[jira] [Assigned] (SPARK-38325) ANSI mode: avoid potential runtime error in HashJoin.extractKeyExprAt()

[ https://issues.apache.org/jira/browse/SPARK-38325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38325: Assignee: Gengliang Wang (was: Apache Spark) > ANSI mode: avoid potential runtime error

[jira] [Assigned] (SPARK-38325) ANSI mode: avoid potential runtime error in HashJoin.extractKeyExprAt()

[ https://issues.apache.org/jira/browse/SPARK-38325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38325: Assignee: Apache Spark (was: Gengliang Wang) > ANSI mode: avoid potential runtime error

[jira] [Created] (SPARK-38325) ANSI mode: avoid potential runtime error in HashJoin.extractKeyExprAt()

Gengliang Wang created SPARK-38325: -- Summary: ANSI mode: avoid potential runtime error in HashJoin.extractKeyExprAt() Key: SPARK-38325 URL: https://issues.apache.org/jira/browse/SPARK-38325

[jira] [Commented] (SPARK-38324) The second range is not [0, 59] in the day time ANSI interval

[ https://issues.apache.org/jira/browse/SPARK-38324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17497905#comment-17497905 ] Hyukjin Kwon commented on SPARK-38324: -- cc [~Gengliang.Wang] FYI > The second range is not [0, 59]

[jira] [Commented] (SPARK-38317) Encoding of java.time.Period always results in "INTERVAL '0-0' YEAR TO MONTH"

[ https://issues.apache.org/jira/browse/SPARK-38317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17497904#comment-17497904 ] Hyukjin Kwon commented on SPARK-38317: -- cc [~maxgekk] FYI > Encoding of java.time.Period always

[jira] [Updated] (SPARK-38317) Encoding of java.time.Period always results in "INTERVAL '0-0' YEAR TO MONTH"

[ https://issues.apache.org/jira/browse/SPARK-38317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-38317: - Description: {code} val dates = Seq(     Period.ZERO,     Period.ofWeeks(2), ).toDS()

[jira] [Commented] (SPARK-38323) Support the hidden file metadata in Streaming

[ https://issues.apache.org/jira/browse/SPARK-38323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17497903#comment-17497903 ] Hyukjin Kwon commented on SPARK-38323: -- [~yaohua] mind fill the description? > Support the hidden

[jira] [Updated] (SPARK-38317) Encoding of java.time.Period always results in "INTERVAL '0-0' YEAR TO MONTH"

[ https://issues.apache.org/jira/browse/SPARK-38317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-38317: - Component/s: SQL (was: Spark Core) > Encoding of java.time.Period always

[jira] [Resolved] (SPARK-37614) Support ANSI Aggregate Function: regr_avgx & regr_avgy

[ https://issues.apache.org/jira/browse/SPARK-37614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-37614. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34868

[jira] [Assigned] (SPARK-37614) Support ANSI Aggregate Function: regr_avgx & regr_avgy

[ https://issues.apache.org/jira/browse/SPARK-37614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-37614: --- Assignee: jiaan.geng > Support ANSI Aggregate Function: regr_avgx & regr_avgy >

[jira] [Assigned] (SPARK-38322) Support query stage show runtime statistics in formatted explain mode

[ https://issues.apache.org/jira/browse/SPARK-38322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38322: Assignee: (was: Apache Spark) > Support query stage show runtime statistics in

[jira] [Commented] (SPARK-38322) Support query stage show runtime statistics in formatted explain mode

[ https://issues.apache.org/jira/browse/SPARK-38322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17497899#comment-17497899 ] Apache Spark commented on SPARK-38322: -- User 'ulysses-you' has created a pull request for this

[jira] [Assigned] (SPARK-38322) Support query stage show runtime statistics in formatted explain mode

[ https://issues.apache.org/jira/browse/SPARK-38322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38322: Assignee: Apache Spark > Support query stage show runtime statistics in formatted

[jira] [Created] (SPARK-38324) The second range is not [0, 59] in the day time ANSI interval

chong created SPARK-38324: - Summary: The second range is not [0, 59] in the day time ANSI interval Key: SPARK-38324 URL: https://issues.apache.org/jira/browse/SPARK-38324 Project: Spark Issue Type:

[jira] [Created] (SPARK-38323) Support the hidden file metadata in Streaming

Yaohua Zhao created SPARK-38323: --- Summary: Support the hidden file metadata in Streaming Key: SPARK-38323 URL: https://issues.apache.org/jira/browse/SPARK-38323 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-38298) Fix DataExpressionSuite, NullExpressionsSuite, StringExpressionsSuite, complexTypesSuite, CastSuite under ANSI mode

[ https://issues.apache.org/jira/browse/SPARK-38298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-38298. Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35618

[jira] [Assigned] (SPARK-38298) Fix DataExpressionSuite, NullExpressionsSuite, StringExpressionsSuite, complexTypesSuite, CastSuite under ANSI mode

[ https://issues.apache.org/jira/browse/SPARK-38298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang reassigned SPARK-38298: -- Assignee: Xinyi Yu > Fix DataExpressionSuite, NullExpressionsSuite,

[jira] [Updated] (SPARK-38322) Support query stage show runtime statistics in formatted explain mode

[ https://issues.apache.org/jira/browse/SPARK-38322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiDuo You updated SPARK-38322: -- Description: The formatted explalin mode is the powerful explain mode to show the details of query

[jira] [Created] (SPARK-38322) Support query stage show runtime statistics in formatted explain mode

XiDuo You created SPARK-38322: - Summary: Support query stage show runtime statistics in formatted explain mode Key: SPARK-38322 URL: https://issues.apache.org/jira/browse/SPARK-38322 Project: Spark

[jira] [Resolved] (SPARK-38311) Fix DynamicPartitionPruning/BucketedReadSuite/ExpressionInfoSuite under ANSI mode

[ https://issues.apache.org/jira/browse/SPARK-38311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-38311. Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35644

[jira] [Resolved] (SPARK-38275) Consider to include WriteBatch's memory in the memory usage of RocksDB state store

[ https://issues.apache.org/jira/browse/SPARK-38275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-38275. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35600

[jira] [Assigned] (SPARK-38275) Consider to include WriteBatch's memory in the memory usage of RocksDB state store

[ https://issues.apache.org/jira/browse/SPARK-38275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-38275: Assignee: Yun Tang > Consider to include WriteBatch's memory in the memory usage of

[jira] [Commented] (SPARK-38172) Adaptive coalesce not working with df persist

[ https://issues.apache.org/jira/browse/SPARK-38172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17497870#comment-17497870 ] XiDuo You commented on SPARK-38172: --- thanks [~Naveenmts]  for the confirming ! > Adaptive coalesce

[jira] [Resolved] (SPARK-38172) Adaptive coalesce not working with df persist

[ https://issues.apache.org/jira/browse/SPARK-38172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiDuo You resolved SPARK-38172. --- Resolution: Won't Fix > Adaptive coalesce not working with df persist >

[jira] [Resolved] (SPARK-38303) Upgrade ansi-regex from 5.0.0 to 5.0.1 in /dev

[ https://issues.apache.org/jira/browse/SPARK-38303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-38303. Fix Version/s: 3.3.0 3.2.2 Assignee: Bjørn Jørgensen

[jira] [Updated] (SPARK-38303) Upgrade ansi-regex from 5.0.0 to 5.0.1 in /dev

[ https://issues.apache.org/jira/browse/SPARK-38303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-38303: --- Affects Version/s: 3.2.1 > Upgrade ansi-regex from 5.0.0 to 5.0.1 in /dev >

[jira] [Commented] (SPARK-38302) Use Java 17 in K8S integration tests when setting spark-tgz

[ https://issues.apache.org/jira/browse/SPARK-38302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17497833#comment-17497833 ] qian commented on SPARK-38302: -- [~dongjoon] Thanks for your work :) > Use Java 17 in K8S integration tests

[jira] [Resolved] (SPARK-38191) The staging directory of write job only needs to be initialized once in HadoopMapReduceCommitProtocol.

[ https://issues.apache.org/jira/browse/SPARK-38191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-38191. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35492

[jira] [Assigned] (SPARK-38191) The staging directory of write job only needs to be initialized once in HadoopMapReduceCommitProtocol.

[ https://issues.apache.org/jira/browse/SPARK-38191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-38191: Assignee: weixiuli > The staging directory of write job only needs to be initialized

[jira] [Commented] (SPARK-38285) ClassCastException: GenericArrayData cannot be cast to InternalRow

[ https://issues.apache.org/jira/browse/SPARK-38285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17497826#comment-17497826 ] L. C. Hsieh commented on SPARK-38285: - Thanks for reporting this. I will take a look. >

[jira] [Assigned] (SPARK-37377) Refactor V2 Partitioning interface and remove deprecated usage of Distribution

[ https://issues.apache.org/jira/browse/SPARK-37377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37377: Assignee: (was: Apache Spark) > Refactor V2 Partitioning interface and remove

[jira] [Assigned] (SPARK-37377) Refactor V2 Partitioning interface and remove deprecated usage of Distribution

[ https://issues.apache.org/jira/browse/SPARK-37377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37377: Assignee: Apache Spark > Refactor V2 Partitioning interface and remove deprecated usage

[jira] [Commented] (SPARK-37377) Refactor V2 Partitioning interface and remove deprecated usage of Distribution

[ https://issues.apache.org/jira/browse/SPARK-37377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17497819#comment-17497819 ] Apache Spark commented on SPARK-37377: -- User 'sunchao' has created a pull request for this issue:

[jira] [Assigned] (SPARK-38107) Use error classes in the compilation errors of python/pandas UDFs

[ https://issues.apache.org/jira/browse/SPARK-38107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38107: Assignee: (was: Apache Spark) > Use error classes in the compilation errors of

[jira] [Assigned] (SPARK-38107) Use error classes in the compilation errors of python/pandas UDFs

[ https://issues.apache.org/jira/browse/SPARK-38107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38107: Assignee: Apache Spark > Use error classes in the compilation errors of python/pandas

[jira] [Commented] (SPARK-38107) Use error classes in the compilation errors of python/pandas UDFs

[ https://issues.apache.org/jira/browse/SPARK-38107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17497799#comment-17497799 ] Apache Spark commented on SPARK-38107: -- User 'itholic' has created a pull request for this issue:

[jira] [Commented] (SPARK-38315) Add a config to control decoding of datetime as Java 8 classes

[ https://issues.apache.org/jira/browse/SPARK-38315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17497740#comment-17497740 ] Apache Spark commented on SPARK-38315: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Commented] (SPARK-38315) Add a config to control decoding of datetime as Java 8 classes

[ https://issues.apache.org/jira/browse/SPARK-38315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17497738#comment-17497738 ] Apache Spark commented on SPARK-38315: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-38315) Add a config to control decoding of datetime as Java 8 classes

[ https://issues.apache.org/jira/browse/SPARK-38315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38315: Assignee: Max Gekk (was: Apache Spark) > Add a config to control decoding of datetime

[jira] [Assigned] (SPARK-38315) Add a config to control decoding of datetime as Java 8 classes

[ https://issues.apache.org/jira/browse/SPARK-38315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38315: Assignee: Apache Spark (was: Max Gekk) > Add a config to control decoding of datetime

[jira] [Updated] (SPARK-38315) Add a config to control decoding of datetime as Java 8 classes

[ https://issues.apache.org/jira/browse/SPARK-38315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-38315: - Summary: Add a config to control decoding of datetime as Java 8 classes (was: Add a config to collect

[jira] [Updated] (SPARK-38315) Add a config to control decoding of datetime as Java 8 classes

[ https://issues.apache.org/jira/browse/SPARK-38315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-38315: - Description: Add new config that should control collect(), in particular, and allow to enable/disable

[jira] [Assigned] (SPARK-38302) Use Java 17 in K8S integration tests when setting spark-tgz

[ https://issues.apache.org/jira/browse/SPARK-38302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-38302: - Assignee: qian > Use Java 17 in K8S integration tests when setting spark-tgz >

[jira] [Updated] (SPARK-38302) Use Java 17 in K8S integration tests when setting spark-tgz

[ https://issues.apache.org/jira/browse/SPARK-38302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-38302: -- Summary: Use Java 17 in K8S integration tests when setting spark-tgz (was: Dockerfile.java17

[jira] [Commented] (SPARK-38302) Dockerfile.java17 can't be used in K8s integration tests when

[ https://issues.apache.org/jira/browse/SPARK-38302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17497676#comment-17497676 ] Dongjoon Hyun commented on SPARK-38302: --- I collected this to a subtask of SPARK-33772 to give more

[jira] [Updated] (SPARK-38302) Dockerfile.java17 can't be used in K8s integration tests when

[ https://issues.apache.org/jira/browse/SPARK-38302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-38302: -- Parent: SPARK-33772 Issue Type: Sub-task (was: Improvement) > Dockerfile.java17

[jira] [Comment Edited] (SPARK-38285) ClassCastException: GenericArrayData cannot be cast to InternalRow

[ https://issues.apache.org/jira/browse/SPARK-38285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17497662#comment-17497662 ] Bruce Robbins edited comment on SPARK-38285 at 2/24/22, 7:19 PM: - I see

[jira] [Commented] (SPARK-38321) Fix BooleanSimplificationSuite under ANSI mode

[ https://issues.apache.org/jira/browse/SPARK-38321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17497673#comment-17497673 ] Apache Spark commented on SPARK-38321: -- User 'anchovYu' has created a pull request for this issue:

[jira] [Assigned] (SPARK-38321) Fix BooleanSimplificationSuite under ANSI mode

[ https://issues.apache.org/jira/browse/SPARK-38321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38321: Assignee: (was: Apache Spark) > Fix BooleanSimplificationSuite under ANSI mode >

[jira] [Assigned] (SPARK-38321) Fix BooleanSimplificationSuite under ANSI mode

[ https://issues.apache.org/jira/browse/SPARK-38321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38321: Assignee: Apache Spark > Fix BooleanSimplificationSuite under ANSI mode >

[jira] [Commented] (SPARK-38321) Fix BooleanSimplificationSuite under ANSI mode

[ https://issues.apache.org/jira/browse/SPARK-38321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17497672#comment-17497672 ] Apache Spark commented on SPARK-38321: -- User 'anchovYu' has created a pull request for this issue:

[jira] [Created] (SPARK-38321) Fix BooleanSimplificationSuite under ANSI mode

Xinyi Yu created SPARK-38321: Summary: Fix BooleanSimplificationSuite under ANSI mode Key: SPARK-38321 URL: https://issues.apache.org/jira/browse/SPARK-38321 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-38285) ClassCastException: GenericArrayData cannot be cast to InternalRow

[ https://issues.apache.org/jira/browse/SPARK-38285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17497662#comment-17497662 ] Bruce Robbins commented on SPARK-38285: --- I see your point. It appears to be caused by [this

[jira] [Commented] (SPARK-38318) regression when replacing a dataset view

[ https://issues.apache.org/jira/browse/SPARK-38318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17497602#comment-17497602 ] Apache Spark commented on SPARK-38318: -- User 'linhongliu-db' has created a pull request for this

[jira] [Updated] (SPARK-38320) (flat)MapGroupsWithState can timeout groups which just received inputs in the same microbatch

[ https://issues.apache.org/jira/browse/SPARK-38320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Balikov updated SPARK-38320: - Description: We have identified an issue where the RocksDB state store iterator will not pick

[jira] [Created] (SPARK-38320) (flat)MapGroupsWithState can timeout groups which just received inputs in the same microbatch

Alex Balikov created SPARK-38320: Summary: (flat)MapGroupsWithState can timeout groups which just received inputs in the same microbatch Key: SPARK-38320 URL: https://issues.apache.org/jira/browse/SPARK-38320

[jira] [Assigned] (SPARK-38318) regression when replacing a dataset view

[ https://issues.apache.org/jira/browse/SPARK-38318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38318: Assignee: (was: Apache Spark) > regression when replacing a dataset view >

[jira] [Assigned] (SPARK-38318) regression when replacing a dataset view

[ https://issues.apache.org/jira/browse/SPARK-38318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38318: Assignee: Apache Spark > regression when replacing a dataset view >

[jira] [Commented] (SPARK-38318) regression when replacing a dataset view

[ https://issues.apache.org/jira/browse/SPARK-38318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17497601#comment-17497601 ] Apache Spark commented on SPARK-38318: -- User 'linhongliu-db' has created a pull request for this

[jira] [Commented] (SPARK-37318) Make FallbackStorageSuite robust in terms of DNS

[ https://issues.apache.org/jira/browse/SPARK-37318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17497598#comment-17497598 ] Erik Krogen commented on SPARK-37318: - Great point [~dongjoon], thanks for pointing it out! > Make

[jira] [Created] (SPARK-38319) Implement Strict Mode to prevent QUERY the entire table

dimtiris kanoute created SPARK-38319: Summary: Implement Strict Mode to prevent QUERY the entire table Key: SPARK-38319 URL: https://issues.apache.org/jira/browse/SPARK-38319 Project: Spark

[jira] [Commented] (SPARK-37318) Make FallbackStorageSuite robust in terms of DNS

[ https://issues.apache.org/jira/browse/SPARK-37318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17497578#comment-17497578 ] Dongjoon Hyun commented on SPARK-37318: --- For a record, - Apache Spark 3.2.1 ~ 3.2.x has this test

[jira] [Commented] (SPARK-37318) Make FallbackStorageSuite robust in terms of DNS

[ https://issues.apache.org/jira/browse/SPARK-37318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17497576#comment-17497576 ] Dongjoon Hyun commented on SPARK-37318: --- It's wrong in branch-3.2, [~xkrogen]. Please be careful

[jira] [Created] (SPARK-38318) regression when replacing a dataset view

Linhong Liu created SPARK-38318: --- Summary: regression when replacing a dataset view Key: SPARK-38318 URL: https://issues.apache.org/jira/browse/SPARK-38318 Project: Spark Issue Type: Task

[jira] [Resolved] (SPARK-38273) Native memory leak in SparkPlan

[ https://issues.apache.org/jira/browse/SPARK-38273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-38273. --- Fix Version/s: 3.3.0 3.2.2 Resolution: Fixed Issue resolved by

[jira] [Updated] (SPARK-38273) decodeUnsafeRows's iterators should close underl… …ying input streams

[ https://issues.apache.org/jira/browse/SPARK-38273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-38273: -- Summary: decodeUnsafeRows's iterators should close underl… …ying input streams (was: Native

[jira] [Updated] (SPARK-38273) decodeUnsafeRows's iterators should close underlying input streams

[ https://issues.apache.org/jira/browse/SPARK-38273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-38273: -- Summary: decodeUnsafeRows's iterators should close underlying input streams (was:

[jira] [Assigned] (SPARK-38273) Native memory leak in SparkPlan

[ https://issues.apache.org/jira/browse/SPARK-38273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-38273: - Assignee: Kevin Sewell > Native memory leak in SparkPlan >

[jira] [Resolved] (SPARK-38300) Refactor `fileToString` and `resourceToBytes` in catalyst.util to clean up duplicate codes

[ https://issues.apache.org/jira/browse/SPARK-38300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-38300. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35622

[jira] [Updated] (SPARK-38300) Use ByteStreams.toByteArray to simplify fileToString and resourceToBytes in catalyst.uti

[ https://issues.apache.org/jira/browse/SPARK-38300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-38300: -- Summary: Use ByteStreams.toByteArray to simplify fileToString and resourceToBytes in

[jira] [Assigned] (SPARK-38300) Refactor `fileToString` and `resourceToBytes` in catalyst.util to clean up duplicate codes

[ https://issues.apache.org/jira/browse/SPARK-38300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-38300: - Assignee: Yang Jie > Refactor `fileToString` and `resourceToBytes` in catalyst.util to

[jira] [Created] (SPARK-38317) Encoding of java.time.Period always results in "INTERVAL '0-0' YEAR TO MONTH"

Jolan Rensen created SPARK-38317: Summary: Encoding of java.time.Period always results in "INTERVAL '0-0' YEAR TO MONTH" Key: SPARK-38317 URL: https://issues.apache.org/jira/browse/SPARK-38317

[jira] [Commented] (SPARK-38316) Fix SQLViewSuite/TriggerAvailableNowSuite/UnwrapCastInBinaryComparisonSuite/UnwrapCastInComparisonEndToEndSuite under ANSI mode

[ https://issues.apache.org/jira/browse/SPARK-38316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17497418#comment-17497418 ] Apache Spark commented on SPARK-38316: -- User 'gengliangwang' has created a pull request for this

[jira] [Assigned] (SPARK-38316) Fix SQLViewSuite/TriggerAvailableNowSuite/UnwrapCastInBinaryComparisonSuite/UnwrapCastInComparisonEndToEndSuite under ANSI mode

[ https://issues.apache.org/jira/browse/SPARK-38316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38316: Assignee: Gengliang Wang (was: Apache Spark) > Fix >

[jira] [Assigned] (SPARK-38316) Fix SQLViewSuite/TriggerAvailableNowSuite/UnwrapCastInBinaryComparisonSuite/UnwrapCastInComparisonEndToEndSuite under ANSI mode

[ https://issues.apache.org/jira/browse/SPARK-38316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38316: Assignee: Apache Spark (was: Gengliang Wang) > Fix >

[jira] [Created] (SPARK-38316) Fix SQLViewSuite/TriggerAvailableNowSuite/UnwrapCastInBinaryComparisonSuite/UnwrapCastInComparisonEndToEndSuite under ANSI mode

Gengliang Wang created SPARK-38316: -- Summary: Fix SQLViewSuite/TriggerAvailableNowSuite/UnwrapCastInBinaryComparisonSuite/UnwrapCastInComparisonEndToEndSuite under ANSI mode Key: SPARK-38316 URL:

[jira] [Commented] (SPARK-36194) Remove the aggregation from left semi/anti join if the same aggregation has already been done on left side

[ https://issues.apache.org/jira/browse/SPARK-36194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17497410#comment-17497410 ] Apache Spark commented on SPARK-36194: -- User 'wangyum' has created a pull request for this issue:

[jira] [Commented] (SPARK-38314) Fail to read parquet files after writing the hidden file metadata in

[ https://issues.apache.org/jira/browse/SPARK-38314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17497407#comment-17497407 ] Apache Spark commented on SPARK-38314: -- User 'Yaohua628' has created a pull request for this issue:

[jira] [Commented] (SPARK-38314) Fail to read parquet files after writing the hidden file metadata in

[ https://issues.apache.org/jira/browse/SPARK-38314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17497406#comment-17497406 ] Apache Spark commented on SPARK-38314: -- User 'Yaohua628' has created a pull request for this issue:

[jira] [Assigned] (SPARK-38314) Fail to read parquet files after writing the hidden file metadata in

[ https://issues.apache.org/jira/browse/SPARK-38314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38314: Assignee: Apache Spark > Fail to read parquet files after writing the hidden file

[jira] [Assigned] (SPARK-38314) Fail to read parquet files after writing the hidden file metadata in

[ https://issues.apache.org/jira/browse/SPARK-38314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38314: Assignee: (was: Apache Spark) > Fail to read parquet files after writing the hidden

[jira] [Commented] (SPARK-38315) Add a config to collect objects as Java 8 types in the Thrift server

[ https://issues.apache.org/jira/browse/SPARK-38315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17497362#comment-17497362 ] Max Gekk commented on SPARK-38315: -- I am working on this. > Add a config to collect objects as Java 8

[jira] [Created] (SPARK-38315) Add a config to collect objects as Java 8 types in the Thrift server

Max Gekk created SPARK-38315: Summary: Add a config to collect objects as Java 8 types in the Thrift server Key: SPARK-38315 URL: https://issues.apache.org/jira/browse/SPARK-38315 Project: Spark

[jira] [Commented] (SPARK-37932) Analyzer can fail when join left side and right side are the same view

[ https://issues.apache.org/jira/browse/SPARK-37932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17497329#comment-17497329 ] Apache Spark commented on SPARK-37932: -- User 'chenzhx' has created a pull request for this issue:

[jira] [Commented] (SPARK-37932) Analyzer can fail when join left side and right side are the same view

[ https://issues.apache.org/jira/browse/SPARK-37932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17497324#comment-17497324 ] Apache Spark commented on SPARK-37932: -- User 'chenzhx' has created a pull request for this issue:

[jira] [Updated] (SPARK-38314) Fail to read parquet files after writing the hidden file metadata in

[ https://issues.apache.org/jira/browse/SPARK-38314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaohua Zhao updated SPARK-38314: Description: Selecting and then writing df containing hidden file metadata column `_metadata`

[jira] [Created] (SPARK-38314) Fail to read parquet files after writing the hidden file metadata in

Yaohua Zhao created SPARK-38314: --- Summary: Fail to read parquet files after writing the hidden file metadata in Key: SPARK-38314 URL: https://issues.apache.org/jira/browse/SPARK-38314 Project: Spark

[jira] [Updated] (SPARK-38285) ClassCastException: GenericArrayData cannot be cast to InternalRow

[ https://issues.apache.org/jira/browse/SPARK-38285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alessandro Bacchini updated SPARK-38285: Description: The following code with Spark 3.2.1 raises an exception:

[jira] [Updated] (SPARK-38285) ClassCastException: GenericArrayData cannot be cast to InternalRow

[ https://issues.apache.org/jira/browse/SPARK-38285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alessandro Bacchini updated SPARK-38285: Description: The following code with Spark 3.2.1 raises an exception:

  1   2   >