[GitHub] [spark] maropu commented on a change in pull request #29804: [SPARK-32859][SQL] Introduce physical rule to decide bucketing dynamically

maropu commented on a change in pull request #29804: URL: https://github.com/apache/spark/pull/29804#discussion_r497135535 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@ -951,6 +951,18 @@ object SQLConf { .checkValue(_ > 0, "th

[GitHub] [spark] maropu commented on a change in pull request #29804: [SPARK-32859][SQL] Introduce physical rule to decide bucketing dynamically

maropu commented on a change in pull request #29804: URL: https://github.com/apache/spark/pull/29804#discussion_r495886405 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/bucketing/DisableUnnecessaryBucketedScan.scala ## @@ -0,0 +1,147 @@ +/* + * Licensed

[GitHub] [spark] maropu commented on a change in pull request #29804: [SPARK-32859][SQL] Introduce physical rule to decide bucketing dynamically

maropu commented on a change in pull request #29804: URL: https://github.com/apache/spark/pull/29804#discussion_r495707732 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@ -951,6 +951,14 @@ object SQLConf { .checkValue(_ > 0, "th

[GitHub] [spark] maropu commented on a change in pull request #29804: [SPARK-32859][SQL] Introduce physical rule to decide bucketing dynamically

maropu commented on a change in pull request #29804: URL: https://github.com/apache/spark/pull/29804#discussion_r495707216 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@ -951,6 +951,14 @@ object SQLConf { .checkValue(_ > 0, "th

[GitHub] [spark] maropu commented on a change in pull request #29804: [SPARK-32859][SQL] Introduce physical rule to decide bucketing dynamically

maropu commented on a change in pull request #29804: URL: https://github.com/apache/spark/pull/29804#discussion_r495701717 ## File path: sql/core/src/test/scala/org/apache/spark/sql/sources/BucketedReadSuite.scala ## @@ -1012,4 +1014,182 @@ abstract class BucketedReadSuite ext

[GitHub] [spark] maropu commented on a change in pull request #29804: [SPARK-32859][SQL] Introduce physical rule to decide bucketing dynamically

maropu commented on a change in pull request #29804: URL: https://github.com/apache/spark/pull/29804#discussion_r495698286 ## File path: sql/core/src/test/scala/org/apache/spark/sql/sources/BucketedReadSuite.scala ## @@ -1012,4 +1014,182 @@ abstract class BucketedReadSuite ext

[GitHub] [spark] maropu commented on a change in pull request #29804: [SPARK-32859][SQL] Introduce physical rule to decide bucketing dynamically

maropu commented on a change in pull request #29804: URL: https://github.com/apache/spark/pull/29804#discussion_r493991276 ## File path: sql/core/src/test/scala/org/apache/spark/sql/sources/BucketedReadSuite.scala ## @@ -1012,4 +1014,43 @@ abstract class BucketedReadSuite exte

[GitHub] [spark] maropu commented on a change in pull request #29804: [SPARK-32859][SQL] Introduce physical rule to decide bucketing dynamically

maropu commented on a change in pull request #29804: URL: https://github.com/apache/spark/pull/29804#discussion_r493985269 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/bucketing/DisableUnnecessaryBucketedScan.scala ## @@ -0,0 +1,153 @@ +/* + * Licensed

[GitHub] [spark] maropu commented on a change in pull request #29804: [SPARK-32859][SQL] Introduce physical rule to decide bucketing dynamically

maropu commented on a change in pull request #29804: URL: https://github.com/apache/spark/pull/29804#discussion_r493984368 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala ## @@ -348,20 +352,22 @@ case class FileSourceScanExec(

[GitHub] [spark] maropu commented on a change in pull request #29804: [SPARK-32859][SQL] Introduce physical rule to decide bucketing dynamically

maropu commented on a change in pull request #29804: URL: https://github.com/apache/spark/pull/29804#discussion_r493390415 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala ## @@ -348,20 +352,22 @@ case class FileSourceScanExec(

[GitHub] [spark] maropu commented on a change in pull request #29804: [SPARK-32859][SQL] Introduce physical rule to decide bucketing dynamically

maropu commented on a change in pull request #29804: URL: https://github.com/apache/spark/pull/29804#discussion_r493097869 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala ## @@ -339,7 +343,7 @@ case class FileSourceScanExec(

[GitHub] [spark] maropu commented on a change in pull request #29804: [SPARK-32859][SQL] Introduce physical rule to decide bucketing dynamically

maropu commented on a change in pull request #29804: URL: https://github.com/apache/spark/pull/29804#discussion_r493097869 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala ## @@ -339,7 +343,7 @@ case class FileSourceScanExec(

[GitHub] [spark] maropu commented on a change in pull request #29804: [SPARK-32859][SQL] Introduce physical rule to decide bucketing dynamically

maropu commented on a change in pull request #29804: URL: https://github.com/apache/spark/pull/29804#discussion_r491243552 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@ -2764,6 +2764,14 @@ object SQLConf { .booleanConf

[GitHub] [spark] maropu commented on a change in pull request #29804: [SPARK-32859][SQL] Introduce physical rule to decide bucketing dynamically

maropu commented on a change in pull request #29804: URL: https://github.com/apache/spark/pull/29804#discussion_r491242019 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala ## @@ -165,6 +166,7 @@ case class FileSourceScanExec( p