Re: [PR] [GLUTEN-5341] Fix some Spark 3.5 UTs [incubator-gluten]

2024-04-18 Thread via GitHub
GlutenPerfBot commented on PR #5445: URL: https://github.com/apache/incubator-gluten/pull/5445#issuecomment-2065797345 = Performance report for TPCH SF2000 with Velox backend, for reference only query

Re: [PR] [VL] Rework co-fallback mechanism of bloom-filter might_contain/agg [incubator-gluten]

2024-04-18 Thread via GitHub
github-actions[bot] commented on PR #5435: URL: https://github.com/apache/incubator-gluten/pull/5435#issuecomment-2065778811 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL] Enable split preloading by default [incubator-gluten]

2024-04-18 Thread via GitHub
github-actions[bot] commented on PR #5456: URL: https://github.com/apache/incubator-gluten/pull/5456#issuecomment-2065777045 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL] Rework co-fallback mechanism of bloom-filter might_contain/agg [incubator-gluten]

2024-04-18 Thread via GitHub
github-actions[bot] commented on PR #5435: URL: https://github.com/apache/incubator-gluten/pull/5435#issuecomment-2065774603 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL] Daily Update Velox Version (2024_04_19) [incubator-gluten]

2024-04-18 Thread via GitHub
zhztheplayer merged PR #5452: URL: https://github.com/apache/incubator-gluten/pull/5452 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

(incubator-gluten) branch main updated: [VL] Daily Update Velox Version (2024_04_19) (#5452)

2024-04-18 Thread hongze
This is an automated email from the ASF dual-hosted git repository. hongze pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git The following commit(s) were added to refs/heads/main by this push: new c4693444e [VL] Daily Update Velox Version

Re: [PR] [VL] Support regr_sxx and regr_syy aggregate functions for Spark 3.4 [incubator-gluten]

2024-04-18 Thread via GitHub
rui-mo merged PR #5444: URL: https://github.com/apache/incubator-gluten/pull/5444 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

(incubator-gluten) branch main updated: [VL] Support regr_sxx and regr_syy aggregate functions for Spark 3.4 (#5444)

2024-04-18 Thread rui
This is an automated email from the ASF dual-hosted git repository. rui pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git The following commit(s) were added to refs/heads/main by this push: new 628763fc3 [VL] Support regr_sxx and regr_syy

Re: [PR] [VL] Daily Update Velox Version (2024_04_19) [incubator-gluten]

2024-04-18 Thread via GitHub
GlutenPerfBot commented on PR #5452: URL: https://github.com/apache/incubator-gluten/pull/5452#issuecomment-2065752406 = Performance report for TPCH SF2000 with Velox backend, for reference only query

Re: [PR] [VL] Rework co-fallback mechanism of bloom-filter might_contain/agg [incubator-gluten]

2024-04-18 Thread via GitHub
github-actions[bot] commented on PR #5435: URL: https://github.com/apache/incubator-gluten/pull/5435#issuecomment-2065747271 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-5352][GLUTEN-5459][CH]Fix and improve year function [incubator-gluten]

2024-04-18 Thread via GitHub
github-actions[bot] commented on PR #5455: URL: https://github.com/apache/incubator-gluten/pull/5455#issuecomment-2065735868 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL] Rework co-fallback mechanism of bloom-filter might_contain/agg [incubator-gluten]

2024-04-18 Thread via GitHub
zhli1142015 commented on code in PR #5435: URL: https://github.com/apache/incubator-gluten/pull/5435#discussion_r1571769493 ## backends-velox/src/main/scala/org/apache/spark/sql/catalyst/BloomFilterMightContainJointRewriteRule.scala: ## @@ -0,0 +1,48 @@ +/* + * Licensed to the

Re: [PR] [VL] Rework co-fallback mechanism of bloom-filter might_contain/agg [incubator-gluten]

2024-04-18 Thread via GitHub
zhztheplayer commented on code in PR #5435: URL: https://github.com/apache/incubator-gluten/pull/5435#discussion_r1571768311 ## backends-velox/src/main/scala/org/apache/spark/sql/catalyst/BloomFilterMightContainJointRewriteRule.scala: ## @@ -0,0 +1,48 @@ +/* + * Licensed to the

Re: [PR] [GLUTEN-5454][CH] Support delete/update/optimize/vacuum API for the MergeTree + Delta [incubator-gluten]

2024-04-18 Thread via GitHub
github-actions[bot] commented on PR #5460: URL: https://github.com/apache/incubator-gluten/pull/5460#issuecomment-2065725808 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-5454][CH] Support delete/update/optimize/vacuum API for the MergeTree + Delta [incubator-gluten]

2024-04-18 Thread via GitHub
github-actions[bot] commented on PR #5460: URL: https://github.com/apache/incubator-gluten/pull/5460#issuecomment-2065725064 https://github.com/apache/incubator-gluten/issues/5454 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] [VL] Rework co-fallback mechanism of bloom-filter might_contain/agg [incubator-gluten]

2024-04-18 Thread via GitHub
zhli1142015 commented on code in PR #5435: URL: https://github.com/apache/incubator-gluten/pull/5435#discussion_r1571748142 ## backends-velox/src/main/scala/org/apache/spark/sql/catalyst/BloomFilterMightContainJointRewriteRule.scala: ## @@ -0,0 +1,48 @@ +/* + * Licensed to the

Re: [PR] [VL] Rework co-fallback mechanism of bloom-filter might_contain/agg [incubator-gluten]

2024-04-18 Thread via GitHub
zhztheplayer commented on code in PR #5435: URL: https://github.com/apache/incubator-gluten/pull/5435#discussion_r1571761427 ## backends-velox/src/main/scala/org/apache/gluten/backendsapi/velox/ListenerApiImpl.scala: ## @@ -45,12 +46,16 @@ class ListenerApiImpl extends

Re: [PR] [VL] Rework co-fallback mechanism of bloom-filter might_contain/agg [incubator-gluten]

2024-04-18 Thread via GitHub
github-actions[bot] commented on PR #5435: URL: https://github.com/apache/incubator-gluten/pull/5435#issuecomment-2065721635 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-5352][GLUTEN-5459][CH]Fix and improve year function [incubator-gluten]

2024-04-18 Thread via GitHub
github-actions[bot] commented on PR #5455: URL: https://github.com/apache/incubator-gluten/pull/5455#issuecomment-2065721486 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [I] [VL] Issues related to Timestamp type [incubator-gluten]

2024-04-18 Thread via GitHub
mskapilks commented on issue #5364: URL: https://github.com/apache/incubator-gluten/issues/5364#issuecomment-2065712440 There is also cases when Timestamp filter predicate is present, the Scan would fallback to Spark. Although it doesn't fail but we support that in native. I will raise PR

Re: [PR] [GLUTEN-5457][CH] Fix merge cause an error log when use mergetree [incubator-gluten]

2024-04-18 Thread via GitHub
github-actions[bot] commented on PR #5458: URL: https://github.com/apache/incubator-gluten/pull/5458#issuecomment-2065712756 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-5457][CH] Fix merge cause an error log when use mergetree [incubator-gluten]

2024-04-18 Thread via GitHub
github-actions[bot] commented on PR #5458: URL: https://github.com/apache/incubator-gluten/pull/5458#issuecomment-2065712579 https://github.com/apache/incubator-gluten/issues/5457 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

(incubator-gluten) branch main updated: [GLUTEN-5341] Fix part of Spark 3.5 UTs (#5445)

2024-04-18 Thread yuanzhou
This is an automated email from the ASF dual-hosted git repository. yuanzhou pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git The following commit(s) were added to refs/heads/main by this push: new eb5b27a45 [GLUTEN-5341] Fix part of

Re: [PR] [VL] Enable split preloading by default [incubator-gluten]

2024-04-18 Thread via GitHub
github-actions[bot] commented on PR #5456: URL: https://github.com/apache/incubator-gluten/pull/5456#issuecomment-2065711007 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-5341] Fix some Spark 3.5 UTs [incubator-gluten]

2024-04-18 Thread via GitHub
zhouyuan merged PR #5445: URL: https://github.com/apache/incubator-gluten/pull/5445 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[I] [CH] Merge cause an error log when use mergetree [incubator-gluten]

2024-04-18 Thread via GitHub
loneylee opened a new issue, #5457: URL: https://github.com/apache/incubator-gluten/issues/5457 ### Backend CH (ClickHouse) ### Bug description default.lineitem_mergetree_optimize (91c217a7-5846-44e1-baf6-220e135052ba): ~DataPart() should remove part

Re: [PR] [VL] Enable split preloading by default [incubator-gluten]

2024-04-18 Thread via GitHub
github-actions[bot] commented on PR #5456: URL: https://github.com/apache/incubator-gluten/pull/5456#issuecomment-2065710851 Thanks for opening a pull request! Could you open an issue for this pull request on Github Issues?

Re: [PR] [GLUTEN-5341] Fix some Spark 3.5 UTs [incubator-gluten]

2024-04-18 Thread via GitHub
yma11 commented on code in PR #5445: URL: https://github.com/apache/incubator-gluten/pull/5445#discussion_r1571748318 ## backends-velox/src/test/scala/org/apache/gluten/execution/VeloxHashJoinSuite.scala: ## @@ -88,6 +87,8 @@ class VeloxHashJoinSuite extends

Re: [PR] [VL] Daily Update Velox Version (2024_04_19) [incubator-gluten]

2024-04-18 Thread via GitHub
zhztheplayer commented on PR #5452: URL: https://github.com/apache/incubator-gluten/pull/5452#issuecomment-2065710255 /Benchmark Velox -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] [GLUTEN-5352][CH]Fix and improve year function [incubator-gluten]

2024-04-18 Thread via GitHub
github-actions[bot] commented on PR #5455: URL: https://github.com/apache/incubator-gluten/pull/5455#issuecomment-2065709014 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-5352][CH]Fix and improve year function [incubator-gluten]

2024-04-18 Thread via GitHub
github-actions[bot] commented on PR #5455: URL: https://github.com/apache/incubator-gluten/pull/5455#issuecomment-2065708899 https://github.com/apache/incubator-gluten/issues/5352 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] [GLUTEN-5405][CH] Add rewrite todate function [incubator-gluten]

2024-04-18 Thread via GitHub
GlutenPerfBot commented on PR #5406: URL: https://github.com/apache/incubator-gluten/pull/5406#issuecomment-2065708711 = Performance report for TPCH SF2000 with Velox backend, for reference only query

Re: [PR] [GLUTEN-5438] feat: Dynamically sizing off-heap memory [incubator-gluten]

2024-04-18 Thread via GitHub
zhztheplayer commented on code in PR #5439: URL: https://github.com/apache/incubator-gluten/pull/5439#discussion_r1571726146 ## gluten-core/src/main/java/org/apache/gluten/memory/memtarget/spark/DynamicOffHeapSizingPolicyChecker.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the

Re: [PR] [GLUTEN-5438] feat: Dynamically sizing off-heap memory [incubator-gluten]

2024-04-18 Thread via GitHub
zhztheplayer commented on code in PR #5439: URL: https://github.com/apache/incubator-gluten/pull/5439#discussion_r1571725004 ## gluten-core/src/main/java/org/apache/gluten/memory/memtarget/spark/DynamicOffHeapSizingPolicyChecker.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the

Re: [PR] [GLUTEN-5438] feat: Dynamically sizing off-heap memory [incubator-gluten]

2024-04-18 Thread via GitHub
zhztheplayer commented on code in PR #5439: URL: https://github.com/apache/incubator-gluten/pull/5439#discussion_r1571716511 ## gluten-core/src/main/java/org/apache/gluten/memory/memtarget/spark/DynamicOffHeapSizingPolicyChecker.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the

Re: [PR] [VL] Support regr_sxx regr_syy aggregate functions [incubator-gluten]

2024-04-18 Thread via GitHub
liujiayi771 commented on code in PR #5444: URL: https://github.com/apache/incubator-gluten/pull/5444#discussion_r1571713055 ## backends-velox/src/test/scala/org/apache/gluten/execution/VeloxAggregateFunctionsSuite.scala: ## @@ -432,14 +432,46 @@ abstract class

Re: [PR] [VL] Support regr_sxx regr_syy aggregate functions [incubator-gluten]

2024-04-18 Thread via GitHub
rui-mo commented on code in PR #5444: URL: https://github.com/apache/incubator-gluten/pull/5444#discussion_r1571714227 ## backends-velox/src/test/scala/org/apache/gluten/execution/VeloxAggregateFunctionsSuite.scala: ## @@ -432,14 +432,46 @@ abstract class

Re: [PR] [GLUTEN-4990] [CH] fix data loss for dynamic patition [incubator-gluten]

2024-04-18 Thread via GitHub
github-actions[bot] commented on PR #4991: URL: https://github.com/apache/incubator-gluten/pull/4991#issuecomment-2065675653 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL] Support regr_sxx regr_syy aggregate functions [incubator-gluten]

2024-04-18 Thread via GitHub
liujiayi771 commented on code in PR #5444: URL: https://github.com/apache/incubator-gluten/pull/5444#discussion_r1571713055 ## backends-velox/src/test/scala/org/apache/gluten/execution/VeloxAggregateFunctionsSuite.scala: ## @@ -432,14 +432,46 @@ abstract class

Re: [PR] [VL] Support regr_sxx regr_syy aggregate functions [incubator-gluten]

2024-04-18 Thread via GitHub
rui-mo commented on code in PR #5444: URL: https://github.com/apache/incubator-gluten/pull/5444#discussion_r1571710714 ## backends-velox/src/test/scala/org/apache/gluten/execution/VeloxAggregateFunctionsSuite.scala: ## @@ -432,14 +432,46 @@ abstract class

Re: [I] [CH] Optimze todate function from bi query [incubator-gluten]

2024-04-18 Thread via GitHub
zzcclp closed issue #5405: [CH] Optimze todate function from bi query URL: https://github.com/apache/incubator-gluten/issues/5405 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] [GLUTEN-5405][CH] Add rewrite todate function [incubator-gluten]

2024-04-18 Thread via GitHub
zzcclp merged PR #5406: URL: https://github.com/apache/incubator-gluten/pull/5406 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

(incubator-gluten) branch main updated: [GLUTEN-5405][CH] Add rewrite todate function (#5406)

2024-04-18 Thread zhangzc
This is an automated email from the ASF dual-hosted git repository. zhangzc pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git The following commit(s) were added to refs/heads/main by this push: new 1dfbdb43f [GLUTEN-5405][CH] Add rewrite

Re: [PR] [VL] Remove batch size limit [incubator-gluten]

2024-04-18 Thread via GitHub
GlutenPerfBot commented on PR #5446: URL: https://github.com/apache/incubator-gluten/pull/5446#issuecomment-2065664848 = Performance report for TPCH SF2000 with Velox backend, for reference only query

Re: [PR] [VL] Support regr_sxx regr_syy aggregate functions [incubator-gluten]

2024-04-18 Thread via GitHub
liujiayi771 commented on PR #5444: URL: https://github.com/apache/incubator-gluten/pull/5444#issuecomment-2065662268 @rui-mo > Is there any unresolved issue for Spark 3.5? Currently, it's unclear to me. I will review these functions together in Spark 3.5 and will find out which

Re: [PR] [VL] Rework co-fallback mechanism of bloom-filter might_contain/agg [incubator-gluten]

2024-04-18 Thread via GitHub
github-actions[bot] commented on PR #5435: URL: https://github.com/apache/incubator-gluten/pull/5435#issuecomment-2065656609 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL] Rework co-fallback mechanism of bloom-filter might_contain/agg [incubator-gluten]

2024-04-18 Thread via GitHub
github-actions[bot] commented on PR #5435: URL: https://github.com/apache/incubator-gluten/pull/5435#issuecomment-2065657753 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL] Rework co-fallback mechanism of bloom-filter might_contain/agg [incubator-gluten]

2024-04-18 Thread via GitHub
github-actions[bot] commented on PR #5435: URL: https://github.com/apache/incubator-gluten/pull/5435#issuecomment-2065655185 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL] Daily Update Velox Version (2024_04_19) [incubator-gluten]

2024-04-18 Thread via GitHub
zhztheplayer commented on PR #5452: URL: https://github.com/apache/incubator-gluten/pull/5452#issuecomment-2065654926 /Benchmark Velox -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] [VL] Rework co-fallback mechanism of bloom-filter might_contain/agg [incubator-gluten]

2024-04-18 Thread via GitHub
zhztheplayer commented on PR #5435: URL: https://github.com/apache/incubator-gluten/pull/5435#issuecomment-2065631327 Example Velox bf with vanilla code-gen ```java public Object generate(Object[] references) { return new GeneratedIteratorForCodegenStage1(references); }

Re: [PR] [VL] Support regr_sxx regr_syy aggregate functions [incubator-gluten]

2024-04-18 Thread via GitHub
github-actions[bot] commented on PR #5444: URL: https://github.com/apache/incubator-gluten/pull/5444#issuecomment-2065630845 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL] Support regr_sxx regr_syy aggregate functions [incubator-gluten]

2024-04-18 Thread via GitHub
liujiayi771 commented on PR #5444: URL: https://github.com/apache/incubator-gluten/pull/5444#issuecomment-2065631285 cc @rui-mo @Yohahaha, thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[PR] [DNM][GLUTEN-5341] enable "test write parquet with compression codec" for Spark3.5 [incubator-gluten]

2024-04-18 Thread via GitHub
yma11 opened a new pull request, #5453: URL: https://github.com/apache/incubator-gluten/pull/5453 ## What changes were proposed in this pull request? enable "test write parquet with compression codec" for Spark3.5 ## How was this patch tested? CI -- This is an

Re: [PR] [DNM][GLUTEN-5341] enable "test write parquet with compression codec" for Spark3.5 [incubator-gluten]

2024-04-18 Thread via GitHub
github-actions[bot] commented on PR #5453: URL: https://github.com/apache/incubator-gluten/pull/5453#issuecomment-2065628349 Thanks for opening a pull request! Could you open an issue for this pull request on Github Issues?

(incubator-gluten) branch main updated (42a103b9e -> 3233ad3bb)

2024-04-18 Thread marong
This is an automated email from the ASF dual-hosted git repository. marong pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git from 42a103b9e [CH] Support expm1 function (#5422) add 3233ad3bb [VL] Remove batch size limit (#5446) No new

Re: [PR] [VL] Remove batch size limit [incubator-gluten]

2024-04-18 Thread via GitHub
marin-ma merged PR #5446: URL: https://github.com/apache/incubator-gluten/pull/5446 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] [GLUTEN-5414][VL] Support read CSV [incubator-gluten]

2024-04-18 Thread via GitHub
github-actions[bot] commented on PR #5447: URL: https://github.com/apache/incubator-gluten/pull/5447#issuecomment-2065602962 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-4039][VL] Add array forall and exists function support [incubator-gluten]

2024-04-18 Thread via GitHub
github-actions[bot] commented on PR #5420: URL: https://github.com/apache/incubator-gluten/pull/5420#issuecomment-2065601331 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-5405][CH] Add rewrite todate function [incubator-gluten]

2024-04-18 Thread via GitHub
loneylee commented on code in PR #5406: URL: https://github.com/apache/incubator-gluten/pull/5406#discussion_r1571617532 ## shims/common/src/main/scala/org/apache/gluten/GlutenConfig.scala: ## @@ -1588,6 +1591,16 @@ object GlutenConfig { .booleanConf

Re: [PR] [GLUTEN-5405][CH] Add rewrite todate function [incubator-gluten]

2024-04-18 Thread via GitHub
loneylee commented on code in PR #5406: URL: https://github.com/apache/incubator-gluten/pull/5406#discussion_r1571612142 ## backends-clickhouse/src/test/scala/org/apache/spark/sql/execution/benchmarks/CHOptimizeRuleBenchmark.scala: ## @@ -0,0 +1,96 @@ +/* + * Licensed to the

Re: [PR] [GLUTEN-5414][VL] Support read CSV [incubator-gluten]

2024-04-18 Thread via GitHub
jinchengchenghh commented on code in PR #5447: URL: https://github.com/apache/incubator-gluten/pull/5447#discussion_r1571603391 ## backends-velox/src/test/scala/org/apache/gluten/execution/TestOperator.scala: ## @@ -458,6 +458,17 @@ class TestOperator extends

Re: [PR] [GLUTEN-5414][VL] Support read CSV [incubator-gluten]

2024-04-18 Thread via GitHub
jinchengchenghh commented on code in PR #5447: URL: https://github.com/apache/incubator-gluten/pull/5447#discussion_r1571603391 ## backends-velox/src/test/scala/org/apache/gluten/execution/TestOperator.scala: ## @@ -458,6 +458,17 @@ class TestOperator extends

Re: [PR] [GLUTEN-5414][VL] Support read CSV [incubator-gluten]

2024-04-18 Thread via GitHub
leesf commented on code in PR #5447: URL: https://github.com/apache/incubator-gluten/pull/5447#discussion_r1570665866 ## backends-velox/src/test/scala/org/apache/gluten/execution/TestOperator.scala: ## @@ -458,6 +458,17 @@ class TestOperator extends

[PR] [VL] Daily Update Velox Version (2024_04_19) [incubator-gluten]

2024-04-18 Thread via GitHub
GlutenPerfBot opened a new pull request, #5452: URL: https://github.com/apache/incubator-gluten/pull/5452 Upstream Velox's New Commits: ```txt e18a4cf8f by zhli1142015, Fix stringop-overflow warning in Scratch.h (9526) 8fb0c9c45 by Christian Zentgraf, Fix MinioServer to prevent

Re: [PR] [VL] Daily Update Velox Version (2024_04_19) [incubator-gluten]

2024-04-18 Thread via GitHub
github-actions[bot] commented on PR #5452: URL: https://github.com/apache/incubator-gluten/pull/5452#issuecomment-2065514157 Thanks for opening a pull request! Could you open an issue for this pull request on Github Issues?

Re: [PR] [GLUTEN-5414][VL] Support read CSV [incubator-gluten]

2024-04-18 Thread via GitHub
jinchengchenghh commented on code in PR #5447: URL: https://github.com/apache/incubator-gluten/pull/5447#discussion_r1571515128 ## gluten-data/src/main/scala/org/apache/gluten/datasource/ArrowFileFormat.scala: ## @@ -0,0 +1,163 @@ +/* + * Licensed to the Apache Software

Re: [PR] [GLUTEN-1632][CH]Daily Update Clickhouse Version (20240419) [incubator-gluten]

2024-04-18 Thread via GitHub
github-actions[bot] commented on PR #5451: URL: https://github.com/apache/incubator-gluten/pull/5451#issuecomment-2065460216 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-1632][CH]Daily Update Clickhouse Version (20240419) [incubator-gluten]

2024-04-18 Thread via GitHub
github-actions[bot] commented on PR #5451: URL: https://github.com/apache/incubator-gluten/pull/5451#issuecomment-2065460051 https://github.com/apache/incubator-gluten/issues/1632 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[PR] [GLUTEN-1632][CH]Daily Update Clickhouse Version (20240419) [incubator-gluten]

2024-04-18 Thread via GitHub
kyligence-git opened a new pull request, #5451: URL: https://github.com/apache/incubator-gluten/pull/5451 Auto commit by gluten daily build, please check the build status and merge it if it's green. -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] [GLUTEN-5414][VL] Support read CSV [incubator-gluten]

2024-04-18 Thread via GitHub
FelixYBW commented on PR #5447: URL: https://github.com/apache/incubator-gluten/pull/5447#issuecomment-2064982504 Can you paste a UI diagram in the first comment? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [I] [VL] Some queries have performance regression on release-1.1 compared to version 20230831 [incubator-gluten]

2024-04-18 Thread via GitHub
FelixYBW commented on issue #5251: URL: https://github.com/apache/incubator-gluten/issues/5251#issuecomment-2064436425 > By the way, the effective value of spark.gluten.sql.columnar.backend.velox.maxSpillFileSize is 20MB, not 1G. #5450 will fix it. That's what I remember in Velox.

Re: [PR] [VL] Make the maxBatchSize check consistent [incubator-gluten]

2024-04-18 Thread via GitHub
FelixYBW commented on code in PR #5442: URL: https://github.com/apache/incubator-gluten/pull/5442#discussion_r1571045799 ## backends-velox/src/main/scala/org/apache/gluten/backendsapi/velox/VeloxBackend.scala: ## @@ -481,10 +481,10 @@ object BackendSettings extends

Re: [PR] [GLUTEN-5438] feat: Dynamically sizing off-heap memory [incubator-gluten]

2024-04-18 Thread via GitHub
supermem613 commented on code in PR #5439: URL: https://github.com/apache/incubator-gluten/pull/5439#discussion_r1570923153 ## gluten-core/src/main/java/org/apache/gluten/memory/memtarget/spark/DynamicOffHeapSizingPolicyChecker.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the

Re: [PR] [CH] Support expm1 function [incubator-gluten]

2024-04-18 Thread via GitHub
GlutenPerfBot commented on PR #5422: URL: https://github.com/apache/incubator-gluten/pull/5422#issuecomment-2064042159 = Performance report for TPCH SF2000 with Velox backend, for reference only query

Re: [PR] [GLUTEN-5438] feat: Dynamically sizing off-heap memory [incubator-gluten]

2024-04-18 Thread via GitHub
supermem613 commented on code in PR #5439: URL: https://github.com/apache/incubator-gluten/pull/5439#discussion_r1570853800 ## gluten-core/src/main/java/org/apache/gluten/memory/memtarget/spark/DynamicOffHeapSizingPolicyChecker.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the

Re: [PR] [GLUTEN-5438] feat: Dynamically sizing off-heap memory [incubator-gluten]

2024-04-18 Thread via GitHub
supermem613 commented on code in PR #5439: URL: https://github.com/apache/incubator-gluten/pull/5439#discussion_r1570828320 ## gluten-core/src/main/java/org/apache/gluten/memory/memtarget/spark/DynamicOffHeapSizingPolicyChecker.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the

Re: [PR] [GLUTEN-5438] feat: Dynamically sizing off-heap memory [incubator-gluten]

2024-04-18 Thread via GitHub
supermem613 commented on code in PR #5439: URL: https://github.com/apache/incubator-gluten/pull/5439#discussion_r1570827513 ## gluten-core/src/main/java/org/apache/gluten/memory/memtarget/spark/DynamicOffHeapSizingPolicyChecker.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the

Re: [PR] [CH] Support shuffle function [incubator-gluten]

2024-04-18 Thread via GitHub
GlutenPerfBot commented on PR #5432: URL: https://github.com/apache/incubator-gluten/pull/5432#issuecomment-2063918999 = Performance report for TPCH SF2000 with Velox backend, for reference only query

Re: [PR] [GLUTEN-5251][VL] Fix inconsistency of the default value for spark.gluten.sql.columnar.backend.velox.maxSpillFileSize [incubator-gluten]

2024-04-18 Thread via GitHub
GlutenPerfBot commented on PR #5450: URL: https://github.com/apache/incubator-gluten/pull/5450#issuecomment-2063820723 = Performance report for TPCH SF2000 with Velox backend, for reference only query

Re: [PR] [CH] Support expm1 function [incubator-gluten]

2024-04-18 Thread via GitHub
liuneng1994 merged PR #5422: URL: https://github.com/apache/incubator-gluten/pull/5422 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

(incubator-gluten) branch main updated: [CH] Support expm1 function (#5422)

2024-04-18 Thread liuneng
This is an automated email from the ASF dual-hosted git repository. liuneng pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git The following commit(s) were added to refs/heads/main by this push: new 42a103b9e [CH] Support expm1 function

(incubator-gluten) branch main updated: [CH] Support shuffle function (#5432)

2024-04-18 Thread liuneng
This is an automated email from the ASF dual-hosted git repository. liuneng pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git The following commit(s) were added to refs/heads/main by this push: new 65dd411e1 [CH] Support shuffle function

Re: [PR] [CH] Support shuffle function [incubator-gluten]

2024-04-18 Thread via GitHub
liuneng1994 merged PR #5432: URL: https://github.com/apache/incubator-gluten/pull/5432 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [I] [VL] Some queries have performance regression on release-1.1 compared to version 20230831 [incubator-gluten]

2024-04-18 Thread via GitHub
zhztheplayer closed issue #5251: [VL] Some queries have performance regression on release-1.1 compared to version 20230831 URL: https://github.com/apache/incubator-gluten/issues/5251 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

(incubator-gluten) branch main updated (fb987c911 -> 3e5742a6b)

2024-04-18 Thread hongze
This is an automated email from the ASF dual-hosted git repository. hongze pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git from fb987c911 [VL] Daily Update Velox Version (2024_04_18) (#5443) add 3e5742a6b [GLUTEN-5251][VL] Fix

Re: [PR] [GLUTEN-5251][VL] Fix inconsistency of the default value for spark.gluten.sql.columnar.backend.velox.maxSpillFileSize [incubator-gluten]

2024-04-18 Thread via GitHub
zhztheplayer merged PR #5450: URL: https://github.com/apache/incubator-gluten/pull/5450 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] [VL] Make the maxBatchSize check consistent [incubator-gluten]

2024-04-18 Thread via GitHub
WangGuangxin commented on code in PR #5442: URL: https://github.com/apache/incubator-gluten/pull/5442#discussion_r1570575798 ## backends-velox/src/main/scala/org/apache/gluten/backendsapi/velox/VeloxBackend.scala: ## @@ -481,10 +481,10 @@ object BackendSettings extends

Re: [PR] [VL] Make the maxBatchSize check consistent [incubator-gluten]

2024-04-18 Thread via GitHub
WangGuangxin closed pull request #5442: [VL] Make the maxBatchSize check consistent URL: https://github.com/apache/incubator-gluten/pull/5442 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] [GLUTEN-5414][VL] Support read CSV [incubator-gluten]

2024-04-18 Thread via GitHub
github-actions[bot] commented on PR #5447: URL: https://github.com/apache/incubator-gluten/pull/5447#issuecomment-2063501509 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL] Rework co-fallback mechanism of bloom-filter might_contain/agg [incubator-gluten]

2024-04-18 Thread via GitHub
github-actions[bot] commented on PR #5435: URL: https://github.com/apache/incubator-gluten/pull/5435#issuecomment-2063414318 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL] Rework co-fallback mechanism of bloom-filter might_contain/agg [incubator-gluten]

2024-04-18 Thread via GitHub
github-actions[bot] commented on PR #5435: URL: https://github.com/apache/incubator-gluten/pull/5435#issuecomment-2063412066 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-5448][VL] Fix the issue of cleaning when left with previous build in velox-backends part [incubator-gluten]

2024-04-18 Thread via GitHub
Yohahaha commented on code in PR #5449: URL: https://github.com/apache/incubator-gluten/pull/5449#discussion_r1570335030 ## dev/builddeps-veloxbe.sh: ## @@ -189,7 +189,7 @@ function build_velox { function build_gluten_cpp { echo "Start to Gluten CPP" cd $GLUTEN_DIR/cpp -

Re: [PR] [GLUTEN-5414][VL] Support read CSV [incubator-gluten]

2024-04-18 Thread via GitHub
liujiayi771 commented on code in PR #5447: URL: https://github.com/apache/incubator-gluten/pull/5447#discussion_r1570334204 ## gluten-data/src/main/scala/org/apache/gluten/datasource/ArrowFileFormat.scala: ## @@ -0,0 +1,163 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] [GLUTEN-5414][VL] Support read CSV [incubator-gluten]

2024-04-18 Thread via GitHub
github-actions[bot] commented on PR #5447: URL: https://github.com/apache/incubator-gluten/pull/5447#issuecomment-2063405653 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [I] [VL] Some queries have performance regression on release-1.1 compared to version 20230831 [incubator-gluten]

2024-04-18 Thread via GitHub
kecookier commented on issue #5251: URL: https://github.com/apache/incubator-gluten/issues/5251#issuecomment-2063401576 I have test the config `spark.gluten.sql.columnar.backend.velox.maxSpillFileSize=1G` , and the number of spill files has reduce from 1.3 million to 40. Looks good, Thank

Re: [PR] [VL] Support array transform function [incubator-gluten]

2024-04-18 Thread via GitHub
Yohahaha commented on PR #5410: URL: https://github.com/apache/incubator-gluten/pull/5410#issuecomment-2063391051 @PHILO-HE would you help take a look? thank you! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] [GLUTEN-4836][VL]Add support for WindowGroupLimitExec in gluten [incubator-gluten]

2024-04-18 Thread via GitHub
JkSelf commented on code in PR #5398: URL: https://github.com/apache/incubator-gluten/pull/5398#discussion_r1570319986 ## gluten-core/src/main/scala/org/apache/gluten/execution/WindowGroupLimitExecTransformer.scala: ## @@ -0,0 +1,158 @@ +/* + * Licensed to the Apache Software

Re: [PR] [GLUTEN-4836][VL]Add support for WindowGroupLimitExec in gluten [incubator-gluten]

2024-04-18 Thread via GitHub
JkSelf commented on code in PR #5398: URL: https://github.com/apache/incubator-gluten/pull/5398#discussion_r1570319408 ## gluten-core/src/main/resources/substrait/proto/substrait/algebra.proto: ## @@ -319,6 +319,15 @@ message WindowRel { } } +message WindowGroupLimitRel {

Re: [PR] [GLUTEN-5414][VL] Support read CSV [incubator-gluten]

2024-04-18 Thread via GitHub
github-actions[bot] commented on PR #5447: URL: https://github.com/apache/incubator-gluten/pull/5447#issuecomment-2063332740 https://github.com/apache/incubator-gluten/issues/5414 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] [GLUTEN-5251][VL] Fix inconsistency of the default value for spark.gluten.sql.columnar.backend.velox.maxSpillFileSize [incubator-gluten]

2024-04-18 Thread via GitHub
github-actions[bot] commented on PR #5450: URL: https://github.com/apache/incubator-gluten/pull/5450#issuecomment-2063311030 https://github.com/apache/incubator-gluten/issues/5251 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

  1   2   >