[jira] [Created] (SPARK-30413) Avoid unnecessary WrappedArray roundtrip in GenericArrayData constructor

Josh Rosen created SPARK-30413: -- Summary: Avoid unnecessary WrappedArray roundtrip in GenericArrayData constructor Key: SPARK-30413 URL: https://issues.apache.org/jira/browse/SPARK-30413 Project: Spark

[jira] [Updated] (SPARK-30214) A new framework to resolve v2 commands with a case of COMMENT ON syntax implementation

[ https://issues.apache.org/jira/browse/SPARK-30214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-30214: - Description: Currently, we have a v2 adapter for v1 catalog (V2SessionCatalog), all the

[jira] [Updated] (SPARK-30214) A new framework to resolve v2 commands with a case of COMMENT ON syntax implementation

[ https://issues.apache.org/jira/browse/SPARK-30214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-30214: - Description: Currently, we have a v2 adapter for v1 catalog (V2SessionCatalog), all the

[jira] [Updated] (SPARK-30214) A new framework to resolve v2 commands with a case of COMMENT ON syntax implementation

[ https://issues.apache.org/jira/browse/SPARK-30214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-30214: - Summary: A new framework to resolve v2 commands with a case of COMMENT ON syntax implementation (was:

[jira] [Resolved] (SPARK-30384) Needs to improve the Column name and tooltips for the Fair Scheduler Pool Table

[ https://issues.apache.org/jira/browse/SPARK-30384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-30384. Assignee: Ankit Raj Boudh Resolution: Fixed This issue is resolved in

[jira] [Updated] (SPARK-30365) When deploy mode is a client, why doesn't it support remote "spark.files" download?

[ https://issues.apache.org/jira/browse/SPARK-30365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangzhun updated SPARK-30365: - Priority: Major (was: Minor) > When deploy mode is a client, why doesn't it support remote

[jira] [Resolved] (SPARK-30412) Eliminate warnings in Java tests regarding to deprecated API

[ https://issues.apache.org/jira/browse/SPARK-30412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30412. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27081

[jira] [Assigned] (SPARK-30412) Eliminate warnings in Java tests regarding to deprecated API

[ https://issues.apache.org/jira/browse/SPARK-30412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-30412: Assignee: Maxim Gekk > Eliminate warnings in Java tests regarding to deprecated API >

[jira] [Comment Edited] (SPARK-22231) Support of map, filter, withColumn, dropColumn in nested list of structures

[ https://issues.apache.org/jira/browse/SPARK-22231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17007215#comment-17007215 ] Reynold Xin edited comment on SPARK-22231 at 1/3/20 4:18 AM: - [~fqaiser94] 

[jira] [Commented] (SPARK-22231) Support of map, filter, withColumn, dropColumn in nested list of structures

[ https://issues.apache.org/jira/browse/SPARK-22231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17007215#comment-17007215 ] Reynold Xin commented on SPARK-22231: - [~fqaiser94] you convinced me with #2. It'd be very verbose

[jira] [Updated] (SPARK-29800) Rewrite non-correlated subquery use ScalaSubquery to optimize perf

[ https://issues.apache.org/jira/browse/SPARK-29800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-29800: -- Summary: Rewrite non-correlated subquery use ScalaSubquery to optimize perf (was: Plan Exists 's

[jira] [Comment Edited] (SPARK-22231) Support of map, filter, withColumn, dropColumn in nested list of structures

[ https://issues.apache.org/jira/browse/SPARK-22231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17007191#comment-17007191 ] fqaiser94 edited comment on SPARK-22231 at 1/3/20 3:28 AM: --- I feel that the 

[jira] [Comment Edited] (SPARK-22231) Support of map, filter, withColumn, dropColumn in nested list of structures

[ https://issues.apache.org/jira/browse/SPARK-22231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17007191#comment-17007191 ] fqaiser94 edited comment on SPARK-22231 at 1/3/20 3:14 AM: --- I feel that the 

[jira] [Comment Edited] (SPARK-22231) Support of map, filter, withColumn, dropColumn in nested list of structures

[ https://issues.apache.org/jira/browse/SPARK-22231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17007191#comment-17007191 ] fqaiser94 edited comment on SPARK-22231 at 1/3/20 3:07 AM: --- I feel strongly

[jira] [Comment Edited] (SPARK-22231) Support of map, filter, withColumn, dropColumn in nested list of structures

[ https://issues.apache.org/jira/browse/SPARK-22231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17007191#comment-17007191 ] fqaiser94 edited comment on SPARK-22231 at 1/3/20 3:07 AM: --- I feel strongly

[jira] [Commented] (SPARK-22231) Support of map, filter, withColumn, dropColumn in nested list of structures

[ https://issues.apache.org/jira/browse/SPARK-22231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17007191#comment-17007191 ] fqaiser94 commented on SPARK-22231: --- I feel strongly that {{withField}} and {{withFieldRenamed}} 

[jira] [Commented] (SPARK-27996) Spark UI redirect will be failed behind the https reverse proxy

[ https://issues.apache.org/jira/browse/SPARK-27996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17007173#comment-17007173 ] Igor Shikin commented on SPARK-27996: - Here is the PR for the History Server: 

[jira] [Assigned] (SPARK-29568) Add flag to stop existing stream when new copy starts

[ https://issues.apache.org/jira/browse/SPARK-29568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-29568: --- Assignee: Burak Yavuz > Add flag to stop existing stream when new copy starts >

[jira] [Resolved] (SPARK-29568) Add flag to stop existing stream when new copy starts

[ https://issues.apache.org/jira/browse/SPARK-29568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-29568. - Fix Version/s: 3.0.0 Resolution: Fixed > Add flag to stop existing stream when new copy starts >

[jira] [Updated] (SPARK-30269) Should use old partition stats to decide whether to update stats when analyzing partition

[ https://issues.apache.org/jira/browse/SPARK-30269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang updated SPARK-30269: - Fix Version/s: 3.0.0 > Should use old partition stats to decide whether to update stats when >

[jira] [Resolved] (SPARK-30373) Avoid unnecessary sort for ParquetUtils.splitFiles

[ https://issues.apache.org/jira/browse/SPARK-30373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30373. -- Resolution: Won't Fix > Avoid unnecessary sort for ParquetUtils.splitFiles >

[jira] [Assigned] (SPARK-30285) Fix deadlock between LiveListenerBus#stop and AsyncEventQueue#removeListenerOnError

[ https://issues.apache.org/jira/browse/SPARK-30285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin reassigned SPARK-30285: -- Assignee: Wang Shuo > Fix deadlock between LiveListenerBus#stop and

[jira] [Resolved] (SPARK-30285) Fix deadlock between LiveListenerBus#stop and AsyncEventQueue#removeListenerOnError

[ https://issues.apache.org/jira/browse/SPARK-30285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved SPARK-30285. Fix Version/s: 3.0.0 2.4.5 Resolution: Fixed

[jira] [Resolved] (SPARK-26173) Prior regularization for Logistic Regression

[ https://issues.apache.org/jira/browse/SPARK-26173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-26173. -- Resolution: Won't Fix > Prior regularization for Logistic Regression >

[jira] [Resolved] (SPARK-27462) Spark hive can not choose some columns in target table flexibly, when running insert into.

[ https://issues.apache.org/jira/browse/SPARK-27462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-27462. -- Resolution: Won't Fix > Spark hive can not choose some columns in target table flexibly, when

[jira] [Commented] (SPARK-22231) Support of map, filter, withColumn, dropColumn in nested list of structures

[ https://issues.apache.org/jira/browse/SPARK-22231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17007131#comment-17007131 ] Henry Davidge commented on SPARK-22231: --- I agree that these methods make more sense on DataFrame

[jira] [Comment Edited] (SPARK-22231) Support of map, filter, withColumn, dropColumn in nested list of structures

[ https://issues.apache.org/jira/browse/SPARK-22231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17007116#comment-17007116 ] fqaiser94 edited comment on SPARK-22231 at 1/2/20 11:25 PM: Hi folks, I can

[jira] [Commented] (SPARK-22231) Support of map, filter, withColumn, dropColumn in nested list of structures

[ https://issues.apache.org/jira/browse/SPARK-22231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17007117#comment-17007117 ] Reynold Xin commented on SPARK-22231: - Makes sense. One question (I've asked about this before):

[jira] [Commented] (SPARK-22231) Support of map, filter, withColumn, dropColumn in nested list of structures

[ https://issues.apache.org/jira/browse/SPARK-22231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17007116#comment-17007116 ] fqaiser94 commented on SPARK-22231: --- Hi folks, I can personally affirm that this would be a valuable

[jira] [Updated] (SPARK-27495) SPIP: Support Stage level resource configuration and scheduling

[ https://issues.apache.org/jira/browse/SPARK-27495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-27495: -- Target Version/s: 3.0.0 > SPIP: Support Stage level resource configuration and scheduling >

[jira] [Assigned] (SPARK-30387) Improve YarnClientSchedulerBackend log message

[ https://issues.apache.org/jira/browse/SPARK-30387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-30387: Assignee: jobit mathew > Improve YarnClientSchedulerBackend log message >

[jira] [Resolved] (SPARK-30387) Improve YarnClientSchedulerBackend log message

[ https://issues.apache.org/jira/browse/SPARK-30387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-30387. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27049

[jira] [Resolved] (SPARK-30341) check overflow for interval arithmetic operations

[ https://issues.apache.org/jira/browse/SPARK-30341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-30341. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26995

[jira] [Assigned] (SPARK-30341) check overflow for interval arithmetic operations

[ https://issues.apache.org/jira/browse/SPARK-30341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-30341: --- Assignee: Kent Yao > check overflow for interval arithmetic operations >

[jira] [Resolved] (SPARK-30284) CREATE VIEW should track the current catalog and namespace

[ https://issues.apache.org/jira/browse/SPARK-30284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-30284. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26923

[jira] [Created] (SPARK-30412) Eliminate warnings in Java tests regarding to deprecated API

Maxim Gekk created SPARK-30412: -- Summary: Eliminate warnings in Java tests regarding to deprecated API Key: SPARK-30412 URL: https://issues.apache.org/jira/browse/SPARK-30412 Project: Spark

[jira] [Commented] (SPARK-30174) Eliminate warnings :part 4

[ https://issues.apache.org/jira/browse/SPARK-30174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17006953#comment-17006953 ] Maxim Gekk commented on SPARK-30174: [~shivuson...@gmail.com] Are you still working on this? If so,

[jira] [Commented] (SPARK-30172) Eliminate warnings: part3

[ https://issues.apache.org/jira/browse/SPARK-30172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17006952#comment-17006952 ] Maxim Gekk commented on SPARK-30172: [~Ankitraj] Are you still working on this? > Eliminate

[jira] [Commented] (SPARK-30171) Eliminate warnings: part2

[ https://issues.apache.org/jira/browse/SPARK-30171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17006949#comment-17006949 ] Maxim Gekk commented on SPARK-30171: [~srowen] SPARK-30258 fixes warnings AvroFunctionsSuite.scala

[jira] [Issue Comment Deleted] (SPARK-29760) Document VALUES statement in SQL Reference.

[ https://issues.apache.org/jira/browse/SPARK-29760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankit Raj Boudh updated SPARK-29760: Comment: was deleted (was: @Sean R. Owen , i will raise PR for this.) > Document VALUES

[jira] [Updated] (SPARK-30411) saveAsTable does not honor spark.hadoop.hive.warehouse.subdir.inherit.perms

[ https://issues.apache.org/jira/browse/SPARK-30411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sanket Reddy updated SPARK-30411: - Description: -bash-4.2$ hdfs dfs -ls /tmp | grep my_databases drwxr-x--T - redsanket users 0

[jira] [Created] (SPARK-30411) saveAsTable does not honor spark.hadoop.hive.warehouse.subdir.inherit.perms

Sanket Reddy created SPARK-30411: Summary: saveAsTable does not honor spark.hadoop.hive.warehouse.subdir.inherit.perms Key: SPARK-30411 URL: https://issues.apache.org/jira/browse/SPARK-30411 Project:

[jira] [Commented] (SPARK-30397) [pyspark] Writer applied to custom model changes type of keys' dict from int to str

[ https://issues.apache.org/jira/browse/SPARK-30397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17006862#comment-17006862 ] Radhwane Chebaane commented on SPARK-30397: --- This issue comes from the native behaviour of

[jira] [Created] (SPARK-30410) Calculating size of table having large number of partitions causes flooding logs

Zhenhua Wang created SPARK-30410: Summary: Calculating size of table having large number of partitions causes flooding logs Key: SPARK-30410 URL: https://issues.apache.org/jira/browse/SPARK-30410

[jira] [Resolved] (SPARK-30403) Fix the NoSuchElementException exception when enable AQE with InSubquery use case

[ https://issues.apache.org/jira/browse/SPARK-30403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-30403. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27068

[jira] [Assigned] (SPARK-30403) Fix the NoSuchElementException exception when enable AQE with InSubquery use case

[ https://issues.apache.org/jira/browse/SPARK-30403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-30403: --- Assignee: Ke Jia > Fix the NoSuchElementException exception when enable AQE with

[jira] [Commented] (SPARK-27996) Spark UI redirect will be failed behind the https reverse proxy

[ https://issues.apache.org/jira/browse/SPARK-27996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17006823#comment-17006823 ] Stijn De Haes commented on SPARK-27996: --- [~ishikin] I am stumbling on this as well. Can you maybe

[jira] [Resolved] (SPARK-30407) reset the metrics info of AdaptiveSparkPlanExec plan when enable aqe

[ https://issues.apache.org/jira/browse/SPARK-30407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-30407. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27074

[jira] [Assigned] (SPARK-30407) reset the metrics info of AdaptiveSparkPlanExec plan when enable aqe

[ https://issues.apache.org/jira/browse/SPARK-30407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-30407: --- Assignee: Ke Jia > reset the metrics info of AdaptiveSparkPlanExec plan when enable aqe >

[jira] [Reopened] (SPARK-30196) Bump lz4-java version to 1.7.0

[ https://issues.apache.org/jira/browse/SPARK-30196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro reopened SPARK-30196: -- > Bump lz4-java version to 1.7.0 > -- > >

[jira] [Commented] (SPARK-30196) Bump lz4-java version to 1.7.0

[ https://issues.apache.org/jira/browse/SPARK-30196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17006819#comment-17006819 ] Takeshi Yamamuro commented on SPARK-30196: -- I'll make this open until the root cause found and

[jira] [Resolved] (SPARK-23516) I think it is unnecessary to transfer unroll memory to storage memory

[ https://issues.apache.org/jira/browse/SPARK-23516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-23516. -- Resolution: Won't Fix > I think it is unnecessary to transfer unroll memory to storage memory

[jira] [Resolved] (SPARK-17454) Use Mesos disk resources

[ https://issues.apache.org/jira/browse/SPARK-17454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-17454. -- Resolution: Won't Fix > Use Mesos disk resources > > >

[jira] [Assigned] (SPARK-26560) Repeating select on udf function throws analysis exception - function not registered

[ https://issues.apache.org/jira/browse/SPARK-26560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-26560: --- Assignee: Jungtaek Lim > Repeating select on udf function throws analysis exception -

[jira] [Resolved] (SPARK-26780) Improve shuffle read using ReadAheadInputStream

[ https://issues.apache.org/jira/browse/SPARK-26780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-26780. -- Resolution: Won't Fix > Improve shuffle read using ReadAheadInputStream >

[jira] [Resolved] (SPARK-26560) Repeating select on udf function throws analysis exception - function not registered

[ https://issues.apache.org/jira/browse/SPARK-26560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-26560. - Fix Version/s: 3.0.0 2.4.5 Resolution: Fixed Issue resolved by pull

[jira] [Created] (SPARK-30409) Use `NoOp` datasource in SQL benchmarks

Maxim Gekk created SPARK-30409: -- Summary: Use `NoOp` datasource in SQL benchmarks Key: SPARK-30409 URL: https://issues.apache.org/jira/browse/SPARK-30409 Project: Spark Issue Type: Test

[jira] [Updated] (SPARK-30408) orderBy in sortBy clause is removed by EliminateSorts

[ https://issues.apache.org/jira/browse/SPARK-30408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] APeng Zhang updated SPARK-30408: Description: OrderBy in sortBy clause will be removed by EliminateSorts. code to reproduce:

[jira] [Created] (SPARK-30408) orderBy in sortBy clause is removed by EliminateSorts

APeng Zhang created SPARK-30408: --- Summary: orderBy in sortBy clause is removed by EliminateSorts Key: SPARK-30408 URL: https://issues.apache.org/jira/browse/SPARK-30408 Project: Spark Issue

[jira] [Resolved] (SPARK-27229) GroupBy Placement in Intersect Distinct

[ https://issues.apache.org/jira/browse/SPARK-27229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-27229. -- Resolution: Won't Fix > GroupBy Placement in Intersect Distinct >

[jira] [Commented] (SPARK-27229) GroupBy Placement in Intersect Distinct

[ https://issues.apache.org/jira/browse/SPARK-27229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17006748#comment-17006748 ] Takeshi Yamamuro commented on SPARK-27229: -- I'll close this for now because the corresponding

[jira] [Commented] (SPARK-26739) Standardized Join Types for DataFrames

[ https://issues.apache.org/jira/browse/SPARK-26739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17006745#comment-17006745 ] Takeshi Yamamuro commented on SPARK-26739: -- I'll close this for now because the corresponding

[jira] [Resolved] (SPARK-26739) Standardized Join Types for DataFrames

[ https://issues.apache.org/jira/browse/SPARK-26739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-26739. -- Resolution: Won't Fix > Standardized Join Types for DataFrames >

[jira] [Commented] (SPARK-27033) Add rule to optimize binary comparisons to its push down format

[ https://issues.apache.org/jira/browse/SPARK-27033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17006742#comment-17006742 ] Takeshi Yamamuro commented on SPARK-27033: -- I'll close this for now because the corresponding

[jira] [Resolved] (SPARK-27033) Add rule to optimize binary comparisons to its push down format

[ https://issues.apache.org/jira/browse/SPARK-27033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-27033. -- Resolution: Won't Fix > Add rule to optimize binary comparisons to its push down

[jira] [Commented] (SPARK-24906) Adaptively set split size for columnar file to ensure the task read data size fit expectation

[ https://issues.apache.org/jira/browse/SPARK-24906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17006691#comment-17006691 ] Lior Chaga commented on SPARK-24906: Looking at the PR, I see only using configurable estimations

[jira] [Updated] (SPARK-30407) reset the metrics info of AdaptiveSparkPlanExec plan when enable aqe

[ https://issues.apache.org/jira/browse/SPARK-30407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ke Jia updated SPARK-30407: --- Description: Working on [https://github.com/apache/spark/pull/26813]. With on AQE, the metric info of

[jira] [Created] (SPARK-30407) reset the metrics info of AdaptiveSparkPlanExec plan when enable aqe

Ke Jia created SPARK-30407: -- Summary: reset the metrics info of AdaptiveSparkPlanExec plan when enable aqe Key: SPARK-30407 URL: https://issues.apache.org/jira/browse/SPARK-30407 Project: Spark