Re: [PR] [VL] Daily Update Velox Version (2024_04_18) [incubator-gluten]

2024-04-17 Thread via GitHub
zhztheplayer commented on PR #5443: URL: https://github.com/apache/incubator-gluten/pull/5443#issuecomment-2063053177 Error ``` - Gluten - SPARK-29544: adaptive skew join with different join types 2024-04-18T01:36:24.2479088Z - Gluten - SPARK-34682: AQEShuffleReadExec

Re: [PR] [VL] Daily Update Velox Version (2024_04_18) [incubator-gluten]

2024-04-17 Thread via GitHub
zhouyuan commented on PR #5443: URL: https://github.com/apache/incubator-gluten/pull/5443#issuecomment-2063042378 /Benchmark Velox -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] [VL] Support regr_sxx regr_syy aggregate functions [incubator-gluten]

2024-04-17 Thread via GitHub
liujiayi771 commented on PR #5444: URL: https://github.com/apache/incubator-gluten/pull/5444#issuecomment-2063035775 cc @PHILO-HE, thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [I] A tool to indicate perf gain of Gluten vs. vanilla Spark [incubator-gluten]

2024-04-17 Thread via GitHub
ivoson commented on issue #4470: URL: https://github.com/apache/incubator-gluten/issues/4470#issuecomment-2063012980 > @zhouyuan Thank you for submitting the issue. This issue will allow us to implement gluten faster instead of passing A/B Test. > > For EventLog analysis, we are

Re: [PR] [GLUTEN-5307][VL] Fix Potential Overflow Issue in VeloxShuffleWriter Due to Mismatched Data Types of RowNumber [incubator-gluten]

2024-04-17 Thread via GitHub
GlutenPerfBot commented on PR #5326: URL: https://github.com/apache/incubator-gluten/pull/5326#issuecomment-2062983956 = Performance report for TPCH SF2000 with Velox backend, for reference only query

Re: [PR] [VL] Remove batch size limit [incubator-gluten]

2024-04-17 Thread via GitHub
github-actions[bot] commented on PR #5446: URL: https://github.com/apache/incubator-gluten/pull/5446#issuecomment-2062951040 Thanks for opening a pull request! Could you open an issue for this pull request on Github Issues?

Re: [PR] [VL] Make the maxBatchSize check consistent [incubator-gluten]

2024-04-17 Thread via GitHub
marin-ma commented on code in PR #5442: URL: https://github.com/apache/incubator-gluten/pull/5442#discussion_r1569912522 ## backends-velox/src/main/scala/org/apache/gluten/backendsapi/velox/VeloxBackend.scala: ## @@ -481,10 +481,10 @@ object BackendSettings extends

Re: [PR] [GLUTEN-5341] Fix some Spark 3.5 UTs [incubator-gluten]

2024-04-17 Thread via GitHub
github-actions[bot] commented on PR #5445: URL: https://github.com/apache/incubator-gluten/pull/5445#issuecomment-2062945897 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [CORE] Fix negative buffer size [incubator-gluten]

2024-04-17 Thread via GitHub
GlutenPerfBot commented on PR #5441: URL: https://github.com/apache/incubator-gluten/pull/5441#issuecomment-2062945664 = Performance report for TPCH SF2000 with Velox backend, for reference only query

[PR] [GLUTEN-5341] Fix some Spark 3.5 UTs [incubator-gluten]

2024-04-17 Thread via GitHub
yma11 opened a new pull request, #5445: URL: https://github.com/apache/incubator-gluten/pull/5445 ## What changes were proposed in this pull request? Fix some Spark3.5 UTs ## How was this patch tested? CI -- This is an automated message from the Apache Git

Re: [PR] [GLUTEN-5341] Fix some Spark 3.5 UTs [incubator-gluten]

2024-04-17 Thread via GitHub
github-actions[bot] commented on PR #5445: URL: https://github.com/apache/incubator-gluten/pull/5445#issuecomment-2062945718 https://github.com/apache/incubator-gluten/issues/5341 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] [GLUTEN-4990] [CH] fix data loss for dynamic patition [incubator-gluten]

2024-04-17 Thread via GitHub
github-actions[bot] commented on PR #4991: URL: https://github.com/apache/incubator-gluten/pull/4991#issuecomment-2062941664 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-5438] feat: Dynamically sizing off-heap memory [incubator-gluten]

2024-04-17 Thread via GitHub
zhztheplayer commented on code in PR #5439: URL: https://github.com/apache/incubator-gluten/pull/5439#discussion_r1569852415 ## gluten-core/src/main/java/org/apache/gluten/memory/memtarget/spark/DynamicOffHeapSizingPolicyChecker.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the

Re: [PR] [GLUTEN-5405][CH] Add rewrite todate function [incubator-gluten]

2024-04-17 Thread via GitHub
github-actions[bot] commented on PR #5406: URL: https://github.com/apache/incubator-gluten/pull/5406#issuecomment-2062925363 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-5438] feat: Dynamically sizing off-heap memory [incubator-gluten]

2024-04-17 Thread via GitHub
zhztheplayer commented on code in PR #5439: URL: https://github.com/apache/incubator-gluten/pull/5439#discussion_r1569852415 ## gluten-core/src/main/java/org/apache/gluten/memory/memtarget/spark/DynamicOffHeapSizingPolicyChecker.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the

Re: [PR] [VL] Support regr_sxx regr_syy aggregate functions [incubator-gluten]

2024-04-17 Thread via GitHub
liujiayi771 commented on PR #5444: URL: https://github.com/apache/incubator-gluten/pull/5444#issuecomment-2062924800 I will try to fix all these regr_xx aggregate functions for spark 3.5 in another PR. -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] [VL] Support regr_sxx regr_syy aggregate functions [incubator-gluten]

2024-04-17 Thread via GitHub
github-actions[bot] commented on PR #5444: URL: https://github.com/apache/incubator-gluten/pull/5444#issuecomment-2062924416 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[PR] [VL] Support regr_sxx regr_syy aggregate functions [incubator-gluten]

2024-04-17 Thread via GitHub
liujiayi771 opened a new pull request, #5444: URL: https://github.com/apache/incubator-gluten/pull/5444 ## What changes were proposed in this pull request? Support regr_sxx regr_syy aggregate functions. ## How was this patch tested? Add new test cases. -- This

Re: [PR] [VL] Support regr_sxx regr_syy aggregate functions [incubator-gluten]

2024-04-17 Thread via GitHub
github-actions[bot] commented on PR #5444: URL: https://github.com/apache/incubator-gluten/pull/5444#issuecomment-2062924261 Thanks for opening a pull request! Could you open an issue for this pull request on Github Issues?

Re: [PR] [GLUTEN-5341] Fix VeloxParquetWriteForHiveSuite.scala [incubator-gluten]

2024-04-17 Thread via GitHub
GlutenPerfBot commented on PR #5426: URL: https://github.com/apache/incubator-gluten/pull/5426#issuecomment-2062914594 = Performance report for TPCH SF2000 with Velox backend, for reference only query

Re: [PR] [GLUTEN-5438] feat: Dynamically sizing off-heap memory [incubator-gluten]

2024-04-17 Thread via GitHub
FelixYBW commented on PR #5439: URL: https://github.com/apache/incubator-gluten/pull/5439#issuecomment-2062904153 Thank you for your PR! @zhanglistar may do a test quickly -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] [GLUTEN-5307][VL] Fix Potential Overflow Issue in VeloxShuffleWriter Due to Mismatched Data Types of RowNumber [incubator-gluten]

2024-04-17 Thread via GitHub
FelixYBW commented on PR #5326: URL: https://github.com/apache/incubator-gluten/pull/5326#issuecomment-2062898554 @yangzhg Did you create an issue in Velox community? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] [VL] Make the maxBatchSize check consistent [incubator-gluten]

2024-04-17 Thread via GitHub
marin-ma commented on PR #5442: URL: https://github.com/apache/incubator-gluten/pull/5442#issuecomment-2062897985 @FelixYBW Should we completely remove the batch size limitation on both java and native code as https://github.com/apache/incubator-gluten/pull/5326 has been merged? --

Re: [PR] [GLUTEN-5307][VL] Fix Potential Overflow Issue in VeloxShuffleWriter Due to Mismatched Data Types of RowNumber [incubator-gluten]

2024-04-17 Thread via GitHub
marin-ma commented on PR #5326: URL: https://github.com/apache/incubator-gluten/pull/5326#issuecomment-2062896416 @yangzhg Sorry that I didn't notice this PR wasn't got merged. There's no performance impact. Thanks for your fix! -- This is an automated message from the Apache Git

(incubator-gluten) branch main updated: [GLUTEN-5307][VL] Fix Potential Overflow Issue in VeloxShuffleWriter Due to Mismatched Data Types of RowNumber (#5326)

2024-04-17 Thread marong
This is an automated email from the ASF dual-hosted git repository. marong pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git The following commit(s) were added to refs/heads/main by this push: new 2d0b6e1da [GLUTEN-5307][VL] Fix Potential

Re: [PR] [GLUTEN-5307][VL] Fix Potential Overflow Issue in VeloxShuffleWriter Due to Mismatched Data Types of RowNumber [incubator-gluten]

2024-04-17 Thread via GitHub
marin-ma merged PR #5326: URL: https://github.com/apache/incubator-gluten/pull/5326 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [I] [VL] Potential Overflow Issue in `VeloxShuffleWriter` Due to Mismatched Data Types of RowNumber [incubator-gluten]

2024-04-17 Thread via GitHub
marin-ma closed issue #5307: [VL] Potential Overflow Issue in `VeloxShuffleWriter` Due to Mismatched Data Types of RowNumber URL: https://github.com/apache/incubator-gluten/issues/5307 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

(incubator-gluten) branch main updated: [VL] Fix negative buffer size (#5441)

2024-04-17 Thread rui
This is an automated email from the ASF dual-hosted git repository. rui pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git The following commit(s) were added to refs/heads/main by this push: new 74c54f39d [VL] Fix negative buffer size

Re: [PR] [CORE] Fix negative buffer size [incubator-gluten]

2024-04-17 Thread via GitHub
rui-mo merged PR #5441: URL: https://github.com/apache/incubator-gluten/pull/5441 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] [VL] Upgrade Arrow java version to 15.0.0 [incubator-gluten]

2024-04-17 Thread via GitHub
jinchengchenghh commented on PR #5430: URL: https://github.com/apache/incubator-gluten/pull/5430#issuecomment-2062882751 ok, I wil close this one -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] [VL] Upgrade Arrow java version to 15.0.0 [incubator-gluten]

2024-04-17 Thread via GitHub
jinchengchenghh closed pull request #5430: [VL] Upgrade Arrow java version to 15.0.0 URL: https://github.com/apache/incubator-gluten/pull/5430 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] [GLUTEN-5405][CH] Add rewrite todate function [incubator-gluten]

2024-04-17 Thread via GitHub
taiyang-li commented on code in PR #5406: URL: https://github.com/apache/incubator-gluten/pull/5406#discussion_r1569816446 ## shims/common/src/main/scala/org/apache/gluten/GlutenConfig.scala: ## @@ -1588,6 +1591,16 @@ object GlutenConfig { .booleanConf

Re: [PR] [GLUTEN-5405][CH] Add rewrite todate function [incubator-gluten]

2024-04-17 Thread via GitHub
taiyang-li commented on code in PR #5406: URL: https://github.com/apache/incubator-gluten/pull/5406#discussion_r1569817562 ## backends-clickhouse/src/test/scala/org/apache/spark/sql/execution/benchmarks/CHOptimizeRuleBenchmark.scala: ## @@ -0,0 +1,96 @@ +/* + * Licensed to the

Re: [PR] [GLUTEN-5307][VL] Fix Potential Overflow Issue in VeloxShuffleWriter Due to Mismatched Data Types of RowNumber [incubator-gluten]

2024-04-17 Thread via GitHub
GlutenPerfBot commented on PR #5326: URL: https://github.com/apache/incubator-gluten/pull/5326#issuecomment-2062877040 = Performance report for TPCH SF2000 with Velox backend, for reference only query

(incubator-gluten) branch main updated: [GLUTEN-5341] Enable VeloxParquetWriteForHiveSuite.scala (#5426)

2024-04-17 Thread kejia
This is an automated email from the ASF dual-hosted git repository. kejia pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git The following commit(s) were added to refs/heads/main by this push: new 64f79b65a [GLUTEN-5341] Enable

Re: [PR] [GLUTEN-5341] Fix VeloxParquetWriteForHiveSuite.scala [incubator-gluten]

2024-04-17 Thread via GitHub
JkSelf merged PR #5426: URL: https://github.com/apache/incubator-gluten/pull/5426 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] [VL] Rework co-fallback mechanism of bloom-filter might_contain/agg [incubator-gluten]

2024-04-17 Thread via GitHub
github-actions[bot] commented on PR #5435: URL: https://github.com/apache/incubator-gluten/pull/5435#issuecomment-2062854932 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL] Rework co-fallback mechanism of bloom-filter might_contain/agg [incubator-gluten]

2024-04-17 Thread via GitHub
github-actions[bot] commented on PR #5435: URL: https://github.com/apache/incubator-gluten/pull/5435#issuecomment-2062850085 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-4058][CORE] Support JDK 11 and 17 for Gluten [incubator-gluten]

2024-04-17 Thread via GitHub
github-actions[bot] commented on PR #4059: URL: https://github.com/apache/incubator-gluten/pull/4059#issuecomment-2062842569 This PR was auto-closed because it has been stalled for 10 days with no activity. Please feel free to reopen if it is still valid. Thanks. -- This is an automated

Re: [PR] [GLUTEN-4058][CORE] Support JDK 11 and 17 for Gluten [incubator-gluten]

2024-04-17 Thread via GitHub
github-actions[bot] closed pull request #4059: [GLUTEN-4058][CORE] Support JDK 11 and 17 for Gluten URL: https://github.com/apache/incubator-gluten/pull/4059 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] [GLUTEN-5307][VL] Fix Potential Overflow Issue in VeloxShuffleWriter Due to Mismatched Data Types of RowNumber [incubator-gluten]

2024-04-17 Thread via GitHub
marin-ma commented on PR #5326: URL: https://github.com/apache/incubator-gluten/pull/5326#issuecomment-2062837862 /Benchmark Velox -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] [CORE] Fix negative buffer size [incubator-gluten]

2024-04-17 Thread via GitHub
WangGuangxin commented on PR #5441: URL: https://github.com/apache/incubator-gluten/pull/5441#issuecomment-2062832037 cc @rui-mo @PHILO-HE -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] [VL][UT] Fix scalar-subquery-select.sql in spark35 [incubator-gluten]

2024-04-17 Thread via GitHub
liujiayi771 commented on PR #5425: URL: https://github.com/apache/incubator-gluten/pull/5425#issuecomment-2062831759 @zhouyuan For the overwrite sql, this file is the only change. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] [VL] Make the maxBatchSize check consistent [incubator-gluten]

2024-04-17 Thread via GitHub
WangGuangxin commented on PR #5442: URL: https://github.com/apache/incubator-gluten/pull/5442#issuecomment-2062831381 cc @marin-ma -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] [GLUTEN-5434] Add script for parquet read fuzzer test [incubator-gluten]

2024-04-17 Thread via GitHub
marin-ma commented on PR #5437: URL: https://github.com/apache/incubator-gluten/pull/5437#issuecomment-2062810538 > For 1), we can use script in this PR as it will generate files with different schemas by different parquet version used. Thanks for your explanation! Just one more

Re: [PR] [GLUTEN-5434] Add script for parquet read fuzzer test [incubator-gluten]

2024-04-17 Thread via GitHub
yma11 commented on PR #5437: URL: https://github.com/apache/incubator-gluten/pull/5437#issuecomment-2062794394 > Thanks! Not sure if this fuzzer test can cover many cases. Looks like the data generator only generates some simple data types

Re: [PR] [VL][UT] Fix scalar-subquery-select.sql in spark35 [incubator-gluten]

2024-04-17 Thread via GitHub
zhouyuan commented on PR #5425: URL: https://github.com/apache/incubator-gluten/pull/5425#issuecomment-2062790379 @liujiayi771 thanks for taking care of this, did you check other files? Is this the only change in SQLQuerySuite? thanks, -yuan -- This is an automated message

Re: [PR] [VL] Add a bad test case when bloom_filter_agg is fallen back while might_contain is not [incubator-gluten]

2024-04-17 Thread via GitHub
zhli1142015 commented on code in PR #5433: URL: https://github.com/apache/incubator-gluten/pull/5433#discussion_r1569728838 ## gluten-ut/spark33/src/test/scala/org/apache/spark/sql/GlutenBloomFilterAggregateQuerySuite.scala: ## @@ -113,4 +113,37 @@ class

Re: [PR] [VL] Daily Update Velox Version (2024_04_18) [incubator-gluten]

2024-04-17 Thread via GitHub
github-actions[bot] commented on PR #5443: URL: https://github.com/apache/incubator-gluten/pull/5443#issuecomment-2062755139 Thanks for opening a pull request! Could you open an issue for this pull request on Github Issues?

[PR] [VL] Daily Update Velox Version (2024_04_18) [incubator-gluten]

2024-04-17 Thread via GitHub
GlutenPerfBot opened a new pull request, #5443: URL: https://github.com/apache/incubator-gluten/pull/5443 Upstream Velox's New Commits: ```txt 910b2008f by Jimmy Lu, Document KLL serialization format compatibility guarantee (9518) a830ffb29 by zhli1142015, Add benchmark for

Re: [PR] [VL] Make the maxBatchSize check consistent [incubator-gluten]

2024-04-17 Thread via GitHub
github-actions[bot] commented on PR #5442: URL: https://github.com/apache/incubator-gluten/pull/5442#issuecomment-2062710897 Thanks for opening a pull request! Could you open an issue for this pull request on Github Issues?

[PR] [VL] Make the maxBatchSize check consistent [incubator-gluten]

2024-04-17 Thread via GitHub
WangGuangxin opened a new pull request, #5442: URL: https://github.com/apache/incubator-gluten/pull/5442 ## What changes were proposed in this pull request? A minor change to make the `maxBatchSize` consistent with the check in `VeloxShuffleWriter`

Re: [PR] [CORE] Fix negative buffer size [incubator-gluten]

2024-04-17 Thread via GitHub
github-actions[bot] commented on PR #5441: URL: https://github.com/apache/incubator-gluten/pull/5441#issuecomment-2062676346 Thanks for opening a pull request! Could you open an issue for this pull request on Github Issues?

[PR] [CORE] Fix negative buffer size [incubator-gluten]

2024-04-17 Thread via GitHub
WangGuangxin opened a new pull request, #5441: URL: https://github.com/apache/incubator-gluten/pull/5441 ## What changes were proposed in this pull request? The buffer size may overflow before convert to Long. We got an error in `RowToVeloxColumnarExec`, ```

Re: [PR] [GLUTEN-1632][CH]Daily Update Clickhouse Version (20240418) [incubator-gluten]

2024-04-17 Thread via GitHub
github-actions[bot] commented on PR #5440: URL: https://github.com/apache/incubator-gluten/pull/5440#issuecomment-2062623295 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-1632][CH]Daily Update Clickhouse Version (20240418) [incubator-gluten]

2024-04-17 Thread via GitHub
github-actions[bot] commented on PR #5440: URL: https://github.com/apache/incubator-gluten/pull/5440#issuecomment-2062623076 https://github.com/apache/incubator-gluten/issues/1632 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[PR] [GLUTEN-1632][CH]Daily Update Clickhouse Version (20240418) [incubator-gluten]

2024-04-17 Thread via GitHub
kyligence-git opened a new pull request, #5440: URL: https://github.com/apache/incubator-gluten/pull/5440 Auto commit by gluten daily build, please check the build status and merge it if it's green. -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] [VL][UT] Fix scalar-subquery-select.sql in spark35 [incubator-gluten]

2024-04-17 Thread via GitHub
GlutenPerfBot commented on PR #5425: URL: https://github.com/apache/incubator-gluten/pull/5425#issuecomment-2062580149 = Performance report for TPCH SF2000 with Velox backend, for reference only query

Re: [PR] [GLUTEN-5341] Fix VeloxParquetWriteForHiveSuite.scala [incubator-gluten]

2024-04-17 Thread via GitHub
ayushi-agarwal commented on PR #5426: URL: https://github.com/apache/incubator-gluten/pull/5426#issuecomment-2061919213 @yma11 @zhouyuan thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] [VL] Extend adapter setup to support Debian distributions [incubator-gluten]

2024-04-17 Thread via GitHub
PHILO-HE commented on PR #5415: URL: https://github.com/apache/incubator-gluten/pull/5415#issuecomment-2061512570 Hi @Preetesh2110, thanks for your work! Is it possible to propose this change in upstream Velox? -- This is an automated message from the Apache Git Service. To respond to

Re: [PR] [GLUTEN-5434] Add script for parquet read fuzzer test [incubator-gluten]

2024-04-17 Thread via GitHub
marin-ma commented on PR #5437: URL: https://github.com/apache/incubator-gluten/pull/5437#issuecomment-2061487322 Thanks! Not sure if this fuzzer test can cover many cases. Looks like the data generator only generates some simple data types

Re: [PR] Dynamically sizing off-heap memory [incubator-gluten]

2024-04-17 Thread via GitHub
github-actions[bot] commented on PR #5439: URL: https://github.com/apache/incubator-gluten/pull/5439#issuecomment-2061466188 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] Dynamically sizing off-heap memory [incubator-gluten]

2024-04-17 Thread via GitHub
github-actions[bot] commented on PR #5439: URL: https://github.com/apache/incubator-gluten/pull/5439#issuecomment-2061465341 Thanks for opening a pull request! Could you open an issue for this pull request on Github Issues?

[PR] Dynamically sizing off-heap memory [incubator-gluten]

2024-04-17 Thread via GitHub
supermem613 opened a new pull request, #5439: URL: https://github.com/apache/incubator-gluten/pull/5439 ## What changes were proposed in this pull request? Today, in Spark we specify the on-heap and off-heap memory sizes as a configuration value read at the beginning of executing a

Re: [PR] [GLUTEN-5438] feat: Dynamically sizing off-heap memory [incubator-gluten]

2024-04-17 Thread via GitHub
github-actions[bot] commented on PR #5439: URL: https://github.com/apache/incubator-gluten/pull/5439#issuecomment-2061473106 https://github.com/apache/incubator-gluten/issues/5438 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [I] UT failures for Spark3.5 [incubator-gluten]

2024-04-17 Thread via GitHub
liujiayi771 commented on issue #5341: URL: https://github.com/apache/incubator-gluten/issues/5341#issuecomment-2061400678 @yma11 I will help to check the regr related UTs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] [VL] Extend adapter setup to support Debian distributions [incubator-gluten]

2024-04-17 Thread via GitHub
surnaik commented on PR #5415: URL: https://github.com/apache/incubator-gluten/pull/5415#issuecomment-2061383048 @PHILO-HE please take a look. Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL][UT] Fix scalar-subquery-select.sql in spark35 [incubator-gluten]

2024-04-17 Thread via GitHub
GlutenPerfBot commented on PR #5425: URL: https://github.com/apache/incubator-gluten/pull/5425#issuecomment-2061344632 = Performance report for TPCH SF2000 with Velox backend, for reference only query

Re: [I] Dynamically sizing off-heap memory [incubator-gluten]

2024-04-17 Thread via GitHub
zhli1142015 commented on issue #5438: URL: https://github.com/apache/incubator-gluten/issues/5438#issuecomment-2061259707 cc @FelixYBW , thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[I] Dynamically sizing off-heap memory [incubator-gluten]

2024-04-17 Thread via GitHub
supermem613 opened a new issue, #5438: URL: https://github.com/apache/incubator-gluten/issues/5438 ### Description When using Gluten with Velox and Spark, today we specify the off-heap memory size and accordingly adjust the on-heap memory as well. In practice, this means that the

(incubator-gluten) branch main updated: [VL][UT] Fix scalar-subquery-select.sql in spark35(#5425)

2024-04-17 Thread yuanzhou
This is an automated email from the ASF dual-hosted git repository. yuanzhou pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git The following commit(s) were added to refs/heads/main by this push: new 9b3f59a1c [VL][UT] Fix

Re: [PR] [GLUTEN-5434] Add script for parquet read fuzzer test [incubator-gluten]

2024-04-17 Thread via GitHub
zhouyuan commented on PR #5437: URL: https://github.com/apache/incubator-gluten/pull/5437#issuecomment-2061242267 CC @marin-ma -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] [VL][UT] Fix scalar-subquery-select.sql in spark35 [incubator-gluten]

2024-04-17 Thread via GitHub
zhouyuan merged PR #5425: URL: https://github.com/apache/incubator-gluten/pull/5425 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] [VL][UT] Fix scalar-subquery-select.sql in spark35 [incubator-gluten]

2024-04-17 Thread via GitHub
zhouyuan commented on PR #5425: URL: https://github.com/apache/incubator-gluten/pull/5425#issuecomment-2061238239 related: https://github.com/apache/incubator-gluten/issues/5341 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] [GLUTEN-5341][VL] Enable UT of GlutenExpressionMappingSuite [incubator-gluten]

2024-04-17 Thread via GitHub
GlutenPerfBot commented on PR #5423: URL: https://github.com/apache/incubator-gluten/pull/5423#issuecomment-2061237249 = Performance report for TPCH SF2000 with Velox backend, for reference only query

Re: [PR] [WIP][DNM][Core] Spark input_file_name support [incubator-gluten]

2024-04-17 Thread via GitHub
github-actions[bot] commented on PR #5106: URL: https://github.com/apache/incubator-gluten/pull/5106#issuecomment-2061163041 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [WIP][DNM][Core] Spark input_file_name support [incubator-gluten]

2024-04-17 Thread via GitHub
github-actions[bot] commented on PR #5106: URL: https://github.com/apache/incubator-gluten/pull/5106#issuecomment-2061175379 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-5341]Fix test write parquet with compression codec [incubator-gluten]

2024-04-17 Thread via GitHub
GlutenPerfBot commented on PR #5424: URL: https://github.com/apache/incubator-gluten/pull/5424#issuecomment-2061138798 = Performance report for TPCH SF2000 with Velox backend, for reference only query

Re: [PR] [GLUTEN-5341][VL] Enable UT of GlutenExpressionMappingSuite [incubator-gluten]

2024-04-17 Thread via GitHub
zhli1142015 merged PR #5423: URL: https://github.com/apache/incubator-gluten/pull/5423 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

(incubator-gluten) branch main updated: [GLUTEN-5341]Fix test write parquet with compression codec (#5424)

2024-04-17 Thread zhli
This is an automated email from the ASF dual-hosted git repository. zhli pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git The following commit(s) were added to refs/heads/main by this push: new bd2ca459c [GLUTEN-5341]Fix test write

Re: [PR] [GLUTEN-5341]Fix test write parquet with compression codec [incubator-gluten]

2024-04-17 Thread via GitHub
zhli1142015 merged PR #5424: URL: https://github.com/apache/incubator-gluten/pull/5424 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[PR] [GLUTEN-5434] Add script for parquet read fuzzer test [incubator-gluten]

2024-04-17 Thread via GitHub
yma11 opened a new pull request, #5437: URL: https://github.com/apache/incubator-gluten/pull/5437 ## What changes were proposed in this pull request? Leverage different version of parquet-mr to generate parquet files with random data and then verify the data using Gluten UT ##

(incubator-gluten) branch main updated: [GLUTEN-5341][VL] Enable UT of GlutenExpressionMappingSuite (#5423)

2024-04-17 Thread zhli
This is an automated email from the ASF dual-hosted git repository. zhli pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git The following commit(s) were added to refs/heads/main by this push: new 5574938a6 [GLUTEN-5341][VL] Enable UT of

Re: [PR] [GLUTEN-5341]Fix test write parquet with compression codec [incubator-gluten]

2024-04-17 Thread via GitHub
ayushi-agarwal commented on PR #5424: URL: https://github.com/apache/incubator-gluten/pull/5424#issuecomment-2061052715 @JkSelf @Yohahaha Thanks for reviewing. Could you please merge it if everything looks good. -- This is an automated message from the Apache Git Service. To respond to

Re: [I] UT failures for Spark3.5 [incubator-gluten]

2024-04-17 Thread via GitHub
zhli1142015 closed issue #5341: UT failures for Spark3.5 URL: https://github.com/apache/incubator-gluten/issues/5341 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] [GLUTEN-5434] Add script for parquet read fuzzer test [incubator-gluten]

2024-04-17 Thread via GitHub
github-actions[bot] commented on PR #5437: URL: https://github.com/apache/incubator-gluten/pull/5437#issuecomment-2061037287 https://github.com/apache/incubator-gluten/issues/5434 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

(incubator-gluten) branch main updated: Make gluten_golden_file_upload.png size small (#5436)

2024-04-17 Thread yao
This is an automated email from the ASF dual-hosted git repository. yao pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git The following commit(s) were added to refs/heads/main by this push: new 129127ed2 Make gluten_golden_file_upload.png

Re: [PR] [GLUTEN-4990] [CH] fix data loss for dynamic patition [incubator-gluten]

2024-04-17 Thread via GitHub
github-actions[bot] commented on PR #4991: URL: https://github.com/apache/incubator-gluten/pull/4991#issuecomment-2060831385 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [DOCS] Make gluten_golden_file_upload.png size small [incubator-gluten]

2024-04-17 Thread via GitHub
yaooqinn merged PR #5436: URL: https://github.com/apache/incubator-gluten/pull/5436 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] [CH] Support shuffle function [incubator-gluten]

2024-04-17 Thread via GitHub
exmy commented on PR #5432: URL: https://github.com/apache/incubator-gluten/pull/5432#issuecomment-2060802199 @PHILO-HE Could you help review these changes about velox ut disable. I excluded these uts about shuffle function due to it's a non deterministic function. Thanks! -- This is

[PR] [DOCS] Make gluten_golden_file_upload.png size small [incubator-gluten]

2024-04-17 Thread via GitHub
ulysses-you opened a new pull request, #5436: URL: https://github.com/apache/incubator-gluten/pull/5436 ## What changes were proposed in this pull request? 27.1MB -> 176KB ## How was this patch tested? N/A -- This is an automated message from the Apache Git

Re: [PR] [DOCS] Make gluten_golden_file_upload.png size small [incubator-gluten]

2024-04-17 Thread via GitHub
ulysses-you commented on PR #5436: URL: https://github.com/apache/incubator-gluten/pull/5436#issuecomment-2060733490 cc @PHILO-HE @zwangsheng thank you -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [DOCS] Make gluten_golden_file_upload.png size small [incubator-gluten]

2024-04-17 Thread via GitHub
github-actions[bot] commented on PR #5436: URL: https://github.com/apache/incubator-gluten/pull/5436#issuecomment-2060733044 Thanks for opening a pull request! Could you open an issue for this pull request on Github Issues?

Re: [PR] [GLUTEN-5405][CH] Add rewrite todate function [incubator-gluten]

2024-04-17 Thread via GitHub
loneylee commented on PR #5406: URL: https://github.com/apache/incubator-gluten/pull/5406#issuecomment-2060728050 > similar Not the same. This pr is to cancel two conversions of timestamp. -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] [GLUTEN-5405][CH] Add rewrite todate function [incubator-gluten]

2024-04-17 Thread via GitHub
github-actions[bot] commented on PR #5406: URL: https://github.com/apache/incubator-gluten/pull/5406#issuecomment-2060725019 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL] Add a bad test case when bloom_filter_agg is fallen back while might_contain is not [incubator-gluten]

2024-04-17 Thread via GitHub
GlutenPerfBot commented on PR #5433: URL: https://github.com/apache/incubator-gluten/pull/5433#issuecomment-2060708410 = Performance report for TPCH SF2000 with Velox backend, for reference only query

Re: [PR] [VL] Rework co-fallback mechanism of bloom-filter might_contain/agg [incubator-gluten]

2024-04-17 Thread via GitHub
github-actions[bot] commented on PR #5435: URL: https://github.com/apache/incubator-gluten/pull/5435#issuecomment-2060651599 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL] Rework co-fallback mechanism of bloom-filter might_contain/agg [incubator-gluten]

2024-04-17 Thread via GitHub
github-actions[bot] commented on PR #5435: URL: https://github.com/apache/incubator-gluten/pull/5435#issuecomment-2060651178 Thanks for opening a pull request! Could you open an issue for this pull request on Github Issues?

[PR] [VL] Rework co-fallback mechanism of bloom-filter might_contain/agg [incubator-gluten]

2024-04-17 Thread via GitHub
zhztheplayer opened a new pull request, #5435: URL: https://github.com/apache/incubator-gluten/pull/5435 Velox's bloom-filter agg/filter functions are logically different with Spark's version. This makes their resident Gluten/Spark Filter/Aggregate operators logically different with

(incubator-gluten) branch main updated: [VL] Add a bad test case when bloom_filter_agg is fallen back while might_contain is not

2024-04-17 Thread hongze
This is an automated email from the ASF dual-hosted git repository. hongze pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git The following commit(s) were added to refs/heads/main by this push: new a1db28ddf [VL] Add a bad test case when

  1   2   >