(incubator-gluten) branch main updated: [GLUTEN-5611] [VL] Avoid trigger Spark memory listener when native memory request can be handled internally (#5631)

2024-05-06 Thread yangzy
This is an automated email from the ASF dual-hosted git repository. yangzy pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git The following commit(s) were added to refs/heads/main by this push: new 2127adcc5 [GLUTEN-5611] [VL] Avoid

Re: [I] [VL] Random memory leak [incubator-gluten]

2024-05-06 Thread via GitHub
Yohahaha closed issue #5611: [VL] Random memory leak URL: https://github.com/apache/incubator-gluten/issues/5611 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] [GLUTEN-5611] [VL] Avoid trigger Spark memory listener when native memory request can be handled internally [incubator-gluten]

2024-05-06 Thread via GitHub
Yohahaha merged PR #5631: URL: https://github.com/apache/incubator-gluten/pull/5631 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] [VL] Enable map_zip_with, zip_with functions [incubator-gluten]

2024-05-06 Thread via GitHub
github-actions[bot] commented on PR #5610: URL: https://github.com/apache/incubator-gluten/pull/5610#issuecomment-2097492490 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL] Enable map_zip_with, zip_with functions [incubator-gluten]

2024-05-06 Thread via GitHub
zhli1142015 commented on PR #5610: URL: https://github.com/apache/incubator-gluten/pull/5610#issuecomment-2097492783 ``` - map_zip_with function - map of primitive types *** FAILED *** Results do not match for query: Timezone:

Re: [PR] [VL] Enable unix_seconds Spark function [incubator-gluten]

2024-05-06 Thread via GitHub
PHILO-HE commented on PR #5602: URL: https://github.com/apache/incubator-gluten/pull/5602#issuecomment-2097483840 > Looks good. Thank you. > > @PHILO-HE can you update the function number metric @FelixYBW, Yes, I have recoded this function in internal OneNote. And will update

Re: [PR] [GLUTEN-5625][VL] Support window range frame [incubator-gluten]

2024-05-06 Thread via GitHub
PHILO-HE commented on PR #5626: URL: https://github.com/apache/incubator-gluten/pull/5626#issuecomment-2097467808 > @PHILO-HE is the PR offload SQL like below? > > ``` > sum(x) OVER ( > PARTITION BY id,sur > ORDER BY date RANGE BETWEEN 6 preceding AND CURRENT ROW

Re: [PR] [GLUTEN-5414][VL] FEAT: Support read CSV [incubator-gluten]

2024-05-06 Thread via GitHub
github-actions[bot] commented on PR #5447: URL: https://github.com/apache/incubator-gluten/pull/5447#issuecomment-2097465296 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-5622] Add new added Spark3.5 UTs in Gluten [incubator-gluten]

2024-05-06 Thread via GitHub
yma11 commented on PR #5623: URL: https://github.com/apache/incubator-gluten/pull/5623#issuecomment-2097459574 @zhouyuan Please help review this PR. Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [I] Add new added Spark3.5 UTs in Gluten [incubator-gluten]

2024-05-06 Thread via GitHub
yma11 commented on issue #5622: URL: https://github.com/apache/incubator-gluten/issues/5622#issuecomment-2097458272 Not all. There are some UTs about row index metadata column failed. This feature is under development, refer [PR](https://github.com/facebookincubator/velox/pull/9174). We

Re: [PR] [VL] Enable unix_seconds Spark function [incubator-gluten]

2024-05-06 Thread via GitHub
GlutenPerfBot commented on PR #5602: URL: https://github.com/apache/incubator-gluten/pull/5602#issuecomment-2097457488 = Performance report for TPCH SF2000 with Velox backend, for reference only query

Re: [PR] [GLUTEN-5414][VL] FEAT: Support read CSV [incubator-gluten]

2024-05-06 Thread via GitHub
jinchengchenghh commented on code in PR #5447: URL: https://github.com/apache/incubator-gluten/pull/5447#discussion_r1591799839 ## backends-velox/src/main/scala/org/apache/gluten/datasource/ArrowConvertorRule.scala: ## @@ -0,0 +1,85 @@ +/* + * Licensed to the Apache Software

Re: [PR] [GLUTEN-5414][VL] FEAT: Support read CSV [incubator-gluten]

2024-05-06 Thread via GitHub
jinchengchenghh commented on PR #5447: URL: https://github.com/apache/incubator-gluten/pull/5447#issuecomment-2097427364 > > the higher protobuf version will cause UnsatisfiedLinkError: /tmp/jnilib-4372912739792055919.tmp: /tmp/jnilib-4372912739792055919.tmp: undefined symbol:

Re: [PR] [GLUTEN-5414][VL] FEAT: Support read CSV [incubator-gluten]

2024-05-06 Thread via GitHub
jinchengchenghh commented on code in PR #5447: URL: https://github.com/apache/incubator-gluten/pull/5447#discussion_r1591796615 ## backends-velox/src/main/scala/org/apache/gluten/datasource/ArrowConvertorRule.scala: ## @@ -0,0 +1,85 @@ +/* + * Licensed to the Apache Software

Re: [PR] [GLUTEN-5414][VL] FEAT: Support read CSV [incubator-gluten]

2024-05-06 Thread via GitHub
liujiayi771 commented on PR #5447: URL: https://github.com/apache/incubator-gluten/pull/5447#issuecomment-2097422138 > the higher protobuf version will cause UnsatisfiedLinkError: /tmp/jnilib-4372912739792055919.tmp: /tmp/jnilib-4372912739792055919.tmp: undefined symbol:

Re: [PR] [VL] RAS: Include rewrite rules used by RewriteSparkPlanRulesManager in EnumeratedTransform [incubator-gluten]

2024-05-06 Thread via GitHub
github-actions[bot] commented on PR #5575: URL: https://github.com/apache/incubator-gluten/pull/5575#issuecomment-2097419847 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-5625][VL] Support window range frame [incubator-gluten]

2024-05-06 Thread via GitHub
FelixYBW commented on PR #5626: URL: https://github.com/apache/incubator-gluten/pull/5626#issuecomment-2097419175 @PHILO-HE is the PR offload SQL like below? ``` sum(x) OVER ( PARTITION BY id,sur ORDER BY date RANGE BETWEEN 6 preceding AND CURRENT ROW ) ```

Re: [PR] [VL] RAS: Include rewrite rules used by RewriteSparkPlanRulesManager in EnumeratedTransform [incubator-gluten]

2024-05-06 Thread via GitHub
github-actions[bot] commented on PR #5575: URL: https://github.com/apache/incubator-gluten/pull/5575#issuecomment-2097417101 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

(incubator-gluten) branch main updated: [VL] Daily Update Velox Version (2024_05_07) (#5628)

2024-05-06 Thread felixybw
This is an automated email from the ASF dual-hosted git repository. felixybw pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git The following commit(s) were added to refs/heads/main by this push: new ca7d3f2fe [VL] Daily Update Velox

Re: [PR] [VL] Daily Update Velox Version (2024_05_07) [incubator-gluten]

2024-05-06 Thread via GitHub
FelixYBW merged PR #5628: URL: https://github.com/apache/incubator-gluten/pull/5628 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] [VL] RAS: Include rewrite rules used by RewriteSparkPlanRulesManager in EnumeratedTransform [incubator-gluten]

2024-05-06 Thread via GitHub
github-actions[bot] commented on PR #5575: URL: https://github.com/apache/incubator-gluten/pull/5575#issuecomment-2097416457 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [I] Add new added Spark3.5 UTs in Gluten [incubator-gluten]

2024-05-06 Thread via GitHub
FelixYBW commented on issue #5622: URL: https://github.com/apache/incubator-gluten/issues/5622#issuecomment-2097415638 Are 3.5 UT all passed? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

(incubator-gluten) branch main updated: [VL] Enable unix_seconds Spark function (#5602)

2024-05-06 Thread zhli
This is an automated email from the ASF dual-hosted git repository. zhli pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git The following commit(s) were added to refs/heads/main by this push: new d18fd7ad5 [VL] Enable unix_seconds Spark

Re: [PR] [VL] Enable unix_seconds Spark function [incubator-gluten]

2024-05-06 Thread via GitHub
zhli1142015 merged PR #5602: URL: https://github.com/apache/incubator-gluten/pull/5602 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] [VL] Enable unix_seconds Spark function [incubator-gluten]

2024-05-06 Thread via GitHub
FelixYBW commented on PR #5602: URL: https://github.com/apache/incubator-gluten/pull/5602#issuecomment-2097414938 Looks good. Thank you. @PHILO-HE can you update the function number metric -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] [VL] RAS: Include rewrite rules used by RewriteSparkPlanRulesManager in EnumeratedTransform [incubator-gluten]

2024-05-06 Thread via GitHub
github-actions[bot] commented on PR #5575: URL: https://github.com/apache/incubator-gluten/pull/5575#issuecomment-2097410117 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL] Add a config to ignore fallback cost for scan [incubator-gluten]

2024-05-06 Thread via GitHub
github-actions[bot] commented on PR #5617: URL: https://github.com/apache/incubator-gluten/pull/5617#issuecomment-2097392715 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL] RAS: Include rewrite rules used by RewriteSparkPlanRulesManager in EnumeratedTransform [incubator-gluten]

2024-05-06 Thread via GitHub
github-actions[bot] commented on PR #5575: URL: https://github.com/apache/incubator-gluten/pull/5575#issuecomment-2097391634 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL] RAS: Include rewrite rules used by RewriteSparkPlanRulesManager in EnumeratedTransform [incubator-gluten]

2024-05-06 Thread via GitHub
github-actions[bot] commented on PR #5575: URL: https://github.com/apache/incubator-gluten/pull/5575#issuecomment-2097391191 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-5611] [VL] Avoid trigger Spark memory listener when native memory request can be handled internally [incubator-gluten]

2024-05-06 Thread via GitHub
github-actions[bot] commented on PR #5631: URL: https://github.com/apache/incubator-gluten/pull/5631#issuecomment-2097378132 https://github.com/apache/incubator-gluten/issues/5611 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] [GLUTEN-2874][VL] support allowDecimalPrecisionLoss [incubator-gluten]

2024-05-06 Thread via GitHub
github-actions[bot] commented on PR #2895: URL: https://github.com/apache/incubator-gluten/pull/2895#issuecomment-2097375309 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [I] [VL] Random memory leak [incubator-gluten]

2024-05-06 Thread via GitHub
Yohahaha commented on issue #5611: URL: https://github.com/apache/incubator-gluten/issues/5611#issuecomment-2097372272 > > I think this function is the issue. > >

[I] [VL] Peak memory is high than Velox's stats [incubator-gluten]

2024-05-06 Thread via GitHub
Yohahaha opened a new issue, #5630: URL: https://github.com/apache/incubator-gluten/issues/5630 ### Backend VL (Velox) ### Bug description TPCDS sf100 q67 Spark view of highest peak memory is 1.1G ``` ++---+-++

Re: [PR] [GLUTEN-5599][VL] Support json_tuple [incubator-gluten]

2024-05-06 Thread via GitHub
WangGuangxin commented on PR #5600: URL: https://github.com/apache/incubator-gluten/pull/5600#issuecomment-2097370028 @marin-ma ready to review now -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] [GLUTEN-5414][VL] FEAT: Support read CSV [incubator-gluten]

2024-05-06 Thread via GitHub
liujiayi771 commented on code in PR #5447: URL: https://github.com/apache/incubator-gluten/pull/5447#discussion_r1591761227 ## backends-velox/src/main/scala/org/apache/gluten/datasource/ArrowConvertorRule.scala: ## @@ -0,0 +1,85 @@ +/* + * Licensed to the Apache Software

Re: [PR] [GLUTEN-5599][VL] Support json_tuple [incubator-gluten]

2024-05-06 Thread via GitHub
github-actions[bot] commented on PR #5600: URL: https://github.com/apache/incubator-gluten/pull/5600#issuecomment-2097367492 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-5599][VL] Support json_tuple [incubator-gluten]

2024-05-06 Thread via GitHub
github-actions[bot] commented on PR #5600: URL: https://github.com/apache/incubator-gluten/pull/5600#issuecomment-2097367092 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-5599][VL] Support json_tuple [incubator-gluten]

2024-05-06 Thread via GitHub
github-actions[bot] commented on PR #5600: URL: https://github.com/apache/incubator-gluten/pull/5600#issuecomment-2097366461 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-5599][VL] Support json_tuple [incubator-gluten]

2024-05-06 Thread via GitHub
github-actions[bot] commented on PR #5600: URL: https://github.com/apache/incubator-gluten/pull/5600#issuecomment-2097365944 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-5599][VL] Support json_tuple [incubator-gluten]

2024-05-06 Thread via GitHub
github-actions[bot] commented on PR #5600: URL: https://github.com/apache/incubator-gluten/pull/5600#issuecomment-2097365940 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [I] [VL] Random memory leak [incubator-gluten]

2024-05-06 Thread via GitHub
zhztheplayer commented on issue #5611: URL: https://github.com/apache/incubator-gluten/issues/5611#issuecomment-2097354264 > I think this function is the issue. > >

Re: [PR] [GLUTEN-5414][VL] FEAT: Support read CSV [incubator-gluten]

2024-05-06 Thread via GitHub
github-actions[bot] commented on PR #5447: URL: https://github.com/apache/incubator-gluten/pull/5447#issuecomment-2097350683 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL] RAS: Include rewrite rules used by RewriteSparkPlanRulesManager in EnumeratedTransform [incubator-gluten]

2024-05-06 Thread via GitHub
github-actions[bot] commented on PR #5575: URL: https://github.com/apache/incubator-gluten/pull/5575#issuecomment-2097340382 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL] RAS: Include rewrite rules used by RewriteSparkPlanRulesManager in EnumeratedTransform [incubator-gluten]

2024-05-06 Thread via GitHub
github-actions[bot] commented on PR #5575: URL: https://github.com/apache/incubator-gluten/pull/5575#issuecomment-2097335480 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [CORE] Only return columns of partitions that require read for iceberg [incubator-gluten]

2024-05-06 Thread via GitHub
liujiayi771 commented on code in PR #5624: URL: https://github.com/apache/incubator-gluten/pull/5624#discussion_r1591740168 ## gluten-iceberg/src/main/scala/org/apache/iceberg/spark/source/GlutenIcebergSourceUtil.scala: ## @@ -94,15 +97,24 @@ object GlutenIcebergSourceUtil {

Re: [PR] [VL] Enable split preloading by default [incubator-gluten]

2024-05-06 Thread via GitHub
zhli1142015 commented on code in PR #5456: URL: https://github.com/apache/incubator-gluten/pull/5456#discussion_r1591740360 ## shims/common/src/main/scala/org/apache/gluten/GlutenConfig.scala: ## @@ -1230,9 +1230,10 @@ object GlutenConfig { val

(incubator-gluten) branch main updated: [CORE] Only return columns of partitions that require read for iceberg (#5624)

2024-05-06 Thread philo
This is an automated email from the ASF dual-hosted git repository. philo pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git The following commit(s) were added to refs/heads/main by this push: new 0a40b2433 [CORE] Only return columns of

Re: [PR] [CORE] Only return columns of partitions that require read for iceberg [incubator-gluten]

2024-05-06 Thread via GitHub
PHILO-HE merged PR #5624: URL: https://github.com/apache/incubator-gluten/pull/5624 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] [CORE] Only return columns of partitions that require read for iceberg [incubator-gluten]

2024-05-06 Thread via GitHub
yma11 commented on code in PR #5624: URL: https://github.com/apache/incubator-gluten/pull/5624#discussion_r1591735594 ## gluten-iceberg/src/main/scala/org/apache/iceberg/spark/source/GlutenIcebergSourceUtil.scala: ## @@ -94,15 +97,24 @@ object GlutenIcebergSourceUtil {

Re: [PR] [VL] RAS: Include rewrite rules used by RewriteSparkPlanRulesManager in EnumeratedTransform [incubator-gluten]

2024-05-06 Thread via GitHub
github-actions[bot] commented on PR #5575: URL: https://github.com/apache/incubator-gluten/pull/5575#issuecomment-2097295104 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL] CI: Gluten-it: Print planning time as well as execution time in test report [incubator-gluten]

2024-05-06 Thread via GitHub
GlutenPerfBot commented on PR #5616: URL: https://github.com/apache/incubator-gluten/pull/5616#issuecomment-2097282906 = Performance report for TPCH SF2000 with Velox backend, for reference only query

Re: [PR] [CORE] Only return columns of partitions that require read for iceberg [incubator-gluten]

2024-05-06 Thread via GitHub
Zouxxyy commented on code in PR #5624: URL: https://github.com/apache/incubator-gluten/pull/5624#discussion_r1591729432 ## gluten-iceberg/src/main/scala/org/apache/iceberg/spark/source/GlutenIcebergSourceUtil.scala: ## @@ -94,15 +97,24 @@ object GlutenIcebergSourceUtil {

Re: [PR] [CORE] Only return columns of partitions that require read for iceberg [incubator-gluten]

2024-05-06 Thread via GitHub
Zouxxyy commented on code in PR #5624: URL: https://github.com/apache/incubator-gluten/pull/5624#discussion_r1591729432 ## gluten-iceberg/src/main/scala/org/apache/iceberg/spark/source/GlutenIcebergSourceUtil.scala: ## @@ -94,15 +97,24 @@ object GlutenIcebergSourceUtil {

Re: [PR] [VL] RAS: Include rewrite rules used by RewriteSparkPlanRulesManager in EnumeratedTransform [incubator-gluten]

2024-05-06 Thread via GitHub
zhztheplayer commented on code in PR #5575: URL: https://github.com/apache/incubator-gluten/pull/5575#discussion_r1591727534 ## gluten-core/src/main/scala/org/apache/gluten/execution/HashAggregateExecBaseTransformer.scala: ## @@ -172,12 +172,32 @@ abstract class

Re: [PR] [VL] Add a config to ignore fallback cost for scan [incubator-gluten]

2024-05-06 Thread via GitHub
ulysses-you commented on code in PR #5617: URL: https://github.com/apache/incubator-gluten/pull/5617#discussion_r1591727181 ## shims/common/src/main/scala/org/apache/gluten/GlutenConfig.scala: ## @@ -208,7 +208,7 @@ class GlutenConfig(conf: SQLConf) extends Logging { def

Re: [PR] [VL] Enable split preloading by default [incubator-gluten]

2024-05-06 Thread via GitHub
ulysses-you commented on code in PR #5456: URL: https://github.com/apache/incubator-gluten/pull/5456#discussion_r1591724504 ## shims/common/src/main/scala/org/apache/gluten/GlutenConfig.scala: ## @@ -1230,9 +1230,10 @@ object GlutenConfig { val

Re: [PR] [CORE] Only return columns of partitions that require read for iceberg [incubator-gluten]

2024-05-06 Thread via GitHub
yma11 commented on code in PR #5624: URL: https://github.com/apache/incubator-gluten/pull/5624#discussion_r1591724427 ## gluten-iceberg/src/main/scala/org/apache/iceberg/spark/source/GlutenIcebergSourceUtil.scala: ## @@ -94,15 +97,24 @@ object GlutenIcebergSourceUtil {

Re: [PR] [VL] Add a config to ignore fallback cost for scan [incubator-gluten]

2024-05-06 Thread via GitHub
github-actions[bot] commented on PR #5617: URL: https://github.com/apache/incubator-gluten/pull/5617#issuecomment-2097248328 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL] RAS: Include rewrite rules used by RewriteSparkPlanRulesManager in EnumeratedTransform [incubator-gluten]

2024-05-06 Thread via GitHub
zhztheplayer commented on code in PR #5575: URL: https://github.com/apache/incubator-gluten/pull/5575#discussion_r1591722695 ## backends-velox/src/main/scala/org/apache/gluten/execution/HashAggregateExecTransformer.scala: ## @@ -60,6 +60,12 @@ abstract class

Re: [PR] [GLUTEN-5625][VL] Support window range frame [incubator-gluten]

2024-05-06 Thread via GitHub
WangGuangxin commented on PR #5626: URL: https://github.com/apache/incubator-gluten/pull/5626#issuecomment-2097247674 cc @PHILO-HE @rui-mo -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] [VL] Enable split preloading by default [incubator-gluten]

2024-05-06 Thread via GitHub
ulysses-you commented on code in PR #5456: URL: https://github.com/apache/incubator-gluten/pull/5456#discussion_r1591722008 ## shims/common/src/main/scala/org/apache/gluten/GlutenConfig.scala: ## @@ -1230,9 +1230,10 @@ object GlutenConfig { val

Re: [PR] [CORE] Only return columns of partitions that require read for iceberg [incubator-gluten]

2024-05-06 Thread via GitHub
Zouxxyy commented on code in PR #5624: URL: https://github.com/apache/incubator-gluten/pull/5624#discussion_r1591718239 ## gluten-iceberg/src/main/scala/org/apache/iceberg/spark/source/GlutenIcebergSourceUtil.scala: ## @@ -94,15 +97,24 @@ object GlutenIcebergSourceUtil {

Re: [I] [VL] Random memory leak [incubator-gluten]

2024-05-06 Thread via GitHub
Yohahaha commented on issue #5611: URL: https://github.com/apache/incubator-gluten/issues/5611#issuecomment-2097227996 I think this function is the issue.

Re: [PR] [GLUTEN-4844] [CH] support native write csv file [incubator-gluten]

2024-05-06 Thread via GitHub
github-actions[bot] commented on PR #4846: URL: https://github.com/apache/incubator-gluten/pull/4846#issuecomment-2097221647 This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days. -- This is an automated

Re: [PR] [DNM] Test Velox branch [incubator-gluten]

2024-05-06 Thread via GitHub
github-actions[bot] commented on PR #3560: URL: https://github.com/apache/incubator-gluten/pull/3560#issuecomment-2097221762 This PR was auto-closed because it has been stalled for 10 days with no activity. Please feel free to reopen if it is still valid. Thanks. -- This is an automated

Re: [PR] [DNM] Test Velox branch [incubator-gluten]

2024-05-06 Thread via GitHub
github-actions[bot] closed pull request #3560: [DNM] Test Velox branch URL: https://github.com/apache/incubator-gluten/pull/3560 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [I] TPC-DS shows bad performance on Arm server when data on HDFS [incubator-gluten]

2024-05-06 Thread via GitHub
ulysses-you commented on issue #5629: URL: https://github.com/apache/incubator-gluten/issues/5629#issuecomment-2097219589 hi @eric-yq , can you share your arm cpu information ? e.g., the cpu model, flag, etc. -- This is an automated message from the Apache Git Service. To respond to the

(incubator-gluten) branch main updated: [GLUTEN-5618][CH] Fix 'Position x is out of bound in Block' error when executing count distinct (#5619)

2024-05-06 Thread zhangzc
This is an automated email from the ASF dual-hosted git repository. zhangzc pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git The following commit(s) were added to refs/heads/main by this push: new fea47053c [GLUTEN-5618][CH] Fix

Re: [I] [CH] Fix 'Position x is out of bound in Block' error when executing count distinct [incubator-gluten]

2024-05-06 Thread via GitHub
zzcclp closed issue #5618: [CH] Fix 'Position x is out of bound in Block' error when executing count distinct URL: https://github.com/apache/incubator-gluten/issues/5618 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] [GLUTEN-5618][CH] Fix 'Position x is out of bound in Block' error when executing count distinct [incubator-gluten]

2024-05-06 Thread via GitHub
zzcclp merged PR #5619: URL: https://github.com/apache/incubator-gluten/pull/5619 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [I] [VL] add div function [incubator-gluten]

2024-05-06 Thread via GitHub
guixiaowen commented on issue #5574: URL: https://github.com/apache/incubator-gluten/issues/5574#issuecomment-2097196722 > @guixiaowen, the proposal looks good! Just left some comments on the velox PR. Thanks! OK. Thanks. -- This is an automated message from the Apache Git

Re: [I] [VL] [BUILD] ERROR: cannot verify repo.anaconda.com's certificate [incubator-gluten]

2024-05-06 Thread via GitHub
PHILO-HE commented on issue #5584: URL: https://github.com/apache/incubator-gluten/issues/5584#issuecomment-2097195697 @ychris78, you can optionally add `--no-check-certificate` for wget. BTW, we have a nightly build job that runs on pure centos/ubuntu OS. See

(incubator-gluten) branch main updated: [VL] CI: Gluten-it: Print planning time as well as execution time in test report (#5616)

2024-05-06 Thread hongze
This is an automated email from the ASF dual-hosted git repository. hongze pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git The following commit(s) were added to refs/heads/main by this push: new 717b263d0 [VL] CI: Gluten-it: Print

Re: [PR] [VL] CI: Gluten-it: Print planning time as well as execution time in test report [incubator-gluten]

2024-05-06 Thread via GitHub
zhztheplayer merged PR #5616: URL: https://github.com/apache/incubator-gluten/pull/5616 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] [VL] Add a config to ignore fallback cost for scan [incubator-gluten]

2024-05-06 Thread via GitHub
PHILO-HE commented on code in PR #5617: URL: https://github.com/apache/incubator-gluten/pull/5617#discussion_r1591705657 ## shims/common/src/main/scala/org/apache/gluten/GlutenConfig.scala: ## @@ -208,7 +208,7 @@ class GlutenConfig(conf: SQLConf) extends Logging { def

Re: [PR] [GLUTEN-5618][CH] Fix 'Position x is out of bound in Block' error when executing count distinct [incubator-gluten]

2024-05-06 Thread via GitHub
loneylee commented on PR #5619: URL: https://github.com/apache/incubator-gluten/pull/5619#issuecomment-2097187513 LGTM -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] [CORE] Only return columns of partitions that require read for iceberg [incubator-gluten]

2024-05-06 Thread via GitHub
yma11 commented on PR #5624: URL: https://github.com/apache/incubator-gluten/pull/5624#issuecomment-2097185277 @Zouxxyy Thanks for fix. We may can leave alone the velox side change as it's not quite necessary by itself. -- This is an automated message from the Apache Git Service. To

Re: [PR] [CORE] Only return columns of partitions that require read for iceberg [incubator-gluten]

2024-05-06 Thread via GitHub
yma11 commented on code in PR #5624: URL: https://github.com/apache/incubator-gluten/pull/5624#discussion_r1591701775 ## gluten-iceberg/src/main/scala/org/apache/iceberg/spark/source/GlutenIcebergSourceUtil.scala: ## @@ -94,15 +97,24 @@ object GlutenIcebergSourceUtil {

Re: [PR] [VL] Add a config to ignore fallback cost for scan [incubator-gluten]

2024-05-06 Thread via GitHub
github-actions[bot] commented on PR #5617: URL: https://github.com/apache/incubator-gluten/pull/5617#issuecomment-2097167589 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL] Daily Update Velox Version (2024_05_07) [incubator-gluten]

2024-05-06 Thread via GitHub
github-actions[bot] commented on PR #5628: URL: https://github.com/apache/incubator-gluten/pull/5628#issuecomment-2097130354 Thanks for opening a pull request! Could you open an issue for this pull request on Github Issues?

[PR] [VL] Daily Update Velox Version (2024_05_07) [incubator-gluten]

2024-05-06 Thread via GitHub
GlutenPerfBot opened a new pull request, #5628: URL: https://github.com/apache/incubator-gluten/pull/5628 Upstream Velox's New Commits: ```txt b7bacaf32 by Joe Abraham, Fix from_base64 Presto function for inputs without padding (8647) 5bf1e2770 by Bikramjeet Vig, Enhance

Re: [PR] [GLUTEN-5625][VL] Support window range frame [incubator-gluten]

2024-05-06 Thread via GitHub
github-actions[bot] commented on PR #5626: URL: https://github.com/apache/incubator-gluten/pull/5626#issuecomment-2097113029 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL] Add a config to ignore fallback cost for scan [incubator-gluten]

2024-05-06 Thread via GitHub
FelixYBW commented on code in PR #5617: URL: https://github.com/apache/incubator-gluten/pull/5617#discussion_r1591662585 ## shims/common/src/main/scala/org/apache/gluten/GlutenConfig.scala: ## @@ -208,7 +208,7 @@ class GlutenConfig(conf: SQLConf) extends Logging { def

Re: [PR] [GLUTEN-2874][VL] support allowDecimalPrecisionLoss [incubator-gluten]

2024-05-06 Thread via GitHub
github-actions[bot] commented on PR #2895: URL: https://github.com/apache/incubator-gluten/pull/2895#issuecomment-2097108061 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-1632][CH]Daily Update Clickhouse Version (20240507) [incubator-gluten]

2024-05-06 Thread via GitHub
github-actions[bot] commented on PR #5627: URL: https://github.com/apache/incubator-gluten/pull/5627#issuecomment-2097069988 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-1632][CH]Daily Update Clickhouse Version (20240507) [incubator-gluten]

2024-05-06 Thread via GitHub
github-actions[bot] commented on PR #5627: URL: https://github.com/apache/incubator-gluten/pull/5627#issuecomment-2097069907 https://github.com/apache/incubator-gluten/issues/1632 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[PR] [GLUTEN-1632][CH]Daily Update Clickhouse Version (20240507) [incubator-gluten]

2024-05-06 Thread via GitHub
kyligence-git opened a new pull request, #5627: URL: https://github.com/apache/incubator-gluten/pull/5627 Auto commit by gluten daily build, please check the build status and merge it if it's green. -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] [GLUTEN-5603] Add new added Spark3.4 UTs in Gluten [incubator-gluten]

2024-05-06 Thread via GitHub
GlutenPerfBot commented on PR #5604: URL: https://github.com/apache/incubator-gluten/pull/5604#issuecomment-2097048482 = Performance report for TPCH SF2000 with Velox backend, for reference only query

Re: [PR] [GLUTEN-5625][VL] Support window range frame [incubator-gluten]

2024-05-06 Thread via GitHub
github-actions[bot] commented on PR #5626: URL: https://github.com/apache/incubator-gluten/pull/5626#issuecomment-2096814150 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-5438] feat: Dynamically sizing off-heap memory [incubator-gluten]

2024-05-06 Thread via GitHub
github-actions[bot] commented on PR #5439: URL: https://github.com/apache/incubator-gluten/pull/5439#issuecomment-2096723564 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-5438] feat: Dynamically sizing off-heap memory [incubator-gluten]

2024-05-06 Thread via GitHub
github-actions[bot] commented on PR #5439: URL: https://github.com/apache/incubator-gluten/pull/5439#issuecomment-2096720057 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-5438] feat: Dynamically sizing off-heap memory [incubator-gluten]

2024-05-06 Thread via GitHub
supermem613 commented on code in PR #5439: URL: https://github.com/apache/incubator-gluten/pull/5439#discussion_r1591437289 ## gluten-core/src/main/java/org/apache/gluten/memory/memtarget/spark/DynamicOffHeapSizingPolicyChecker.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the

Re: [PR] [GLUTEN-5625][VL] Support window range frame [incubator-gluten]

2024-05-06 Thread via GitHub
github-actions[bot] commented on PR #5626: URL: https://github.com/apache/incubator-gluten/pull/5626#issuecomment-2096712614 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-5625][VL] Support window range frame [incubator-gluten]

2024-05-06 Thread via GitHub
github-actions[bot] commented on PR #5626: URL: https://github.com/apache/incubator-gluten/pull/5626#issuecomment-2096705171 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-5625][VL] Support window range frame [incubator-gluten]

2024-05-06 Thread via GitHub
github-actions[bot] commented on PR #5626: URL: https://github.com/apache/incubator-gluten/pull/5626#issuecomment-2096686572 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-5625][VL] Support window range frame [incubator-gluten]

2024-05-06 Thread via GitHub
github-actions[bot] commented on PR #5626: URL: https://github.com/apache/incubator-gluten/pull/5626#issuecomment-2096681462 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-5625][VL] Support window range frame [incubator-gluten]

2024-05-06 Thread via GitHub
github-actions[bot] commented on PR #5626: URL: https://github.com/apache/incubator-gluten/pull/5626#issuecomment-2096663887 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-5625][VL] Support window range frame [incubator-gluten]

2024-05-06 Thread via GitHub
github-actions[bot] commented on PR #5626: URL: https://github.com/apache/incubator-gluten/pull/5626#issuecomment-2096663524 https://github.com/apache/incubator-gluten/issues/5625 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[PR] [GLUTEN-5625][VL] Support window range frame [incubator-gluten]

2024-05-06 Thread via GitHub
WangGuangxin opened a new pull request, #5626: URL: https://github.com/apache/incubator-gluten/pull/5626 ## What changes were proposed in this pull request? Support window range frame by pre-project frame offset for non-SpecialFrameBoundary (Fixes: \#5625) ## How was

[I] [VL] Support window range frame [incubator-gluten]

2024-05-06 Thread via GitHub
WangGuangxin opened a new issue, #5625: URL: https://github.com/apache/incubator-gluten/issues/5625 ### Description Support window range frame -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

  1   2   >