(incubator-gluten) branch main updated: [CORE] Add a compilation-time check to forbid case-class inheritance (#5729)

2024-05-13 Thread hongze
This is an automated email from the ASF dual-hosted git repository. hongze pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git The following commit(s) were added to refs/heads/main by this push: new ecf61ee2f [CORE] Add a compilation-time

Re: [PR] [CORE] Add a compilation-time check to forbid case-class inheritance [incubator-gluten]

2024-05-13 Thread via GitHub
zhztheplayer merged PR #5729: URL: https://github.com/apache/incubator-gluten/pull/5729 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] [VL] Drop the test table after all tests in FallbackSuite [incubator-gluten]

2024-05-13 Thread via GitHub
ivoson commented on PR #5737: URL: https://github.com/apache/incubator-gluten/pull/5737#issuecomment-2109287064 cc @zhztheplayer can you please take a look? Thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] [VL] Drop the test table after all tests in FallbackSuite [incubator-gluten]

2024-05-13 Thread via GitHub
github-actions[bot] commented on PR #5737: URL: https://github.com/apache/incubator-gluten/pull/5737#issuecomment-2109287011 Thanks for opening a pull request! Could you open an issue for this pull request on Github Issues?

Re: [I] Add supported backend information in Sig (signature for a expression) [incubator-gluten]

2024-05-13 Thread via GitHub
zhztheplayer commented on issue #5700: URL: https://github.com/apache/incubator-gluten/issues/5700#issuecomment-2109286962 > When do plan validation, it can be eagerly detected if the function is not supported by the backend; We had this API

[PR] [VL] Drop the test table after all tests in FallbackSuite [incubator-gluten]

2024-05-13 Thread via GitHub
ivoson opened a new pull request, #5737: URL: https://github.com/apache/incubator-gluten/pull/5737 ## What changes were proposed in this pull request? Follow up of https://github.com/apache/incubator-gluten/pull/5655 to drop new added test table after all. ## How was this

Re: [PR] [GLUTEN-5668][CH] Support mixed inequal conditions in hash join [incubator-gluten]

2024-05-13 Thread via GitHub
github-actions[bot] commented on PR #5735: URL: https://github.com/apache/incubator-gluten/pull/5735#issuecomment-2109251543 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-5668][CH] Support mixed inequal conditions in hash join [incubator-gluten]

2024-05-13 Thread via GitHub
github-actions[bot] commented on PR #5735: URL: https://github.com/apache/incubator-gluten/pull/5735#issuecomment-2109237816 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [I] Operator::getOutput failed for [operator: ValueStream, plan node ID: 0]:IOError: ZSTD decompression failed: Unknown frame descriptor [incubator-gluten]

2024-05-13 Thread via GitHub
zhouyuan commented on issue #5736: URL: https://github.com/apache/incubator-gluten/issues/5736#issuecomment-2109230024 @zhztheplayer is this related with some bugs on c2r case? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] [GLUTEN-5668][CH] Support mixed inequal conditions in hash join [incubator-gluten]

2024-05-13 Thread via GitHub
github-actions[bot] commented on PR #5735: URL: https://github.com/apache/incubator-gluten/pull/5735#issuecomment-2109214269 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-5668][CH] Support mixed inequal conditions in hash join [incubator-gluten]

2024-05-13 Thread via GitHub
github-actions[bot] commented on PR #5735: URL: https://github.com/apache/incubator-gluten/pull/5735#issuecomment-2109213288 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[I] Operator::getOutput failed for [operator: ValueStream, plan node ID: 0]:IOError: ZSTD decompression failed: Unknown frame descriptor [incubator-gluten]

2024-05-13 Thread via GitHub
boneanxs opened a new issue, #5736: URL: https://github.com/apache/incubator-gluten/issues/5736 ### Backend VL (Velox) ### Bug description Meet this error when running with velox backend: ```java Reason: Operator::getOutput failed for [operator: ValueStream,

Re: [I] Add supported backend information in Sig (signature for a expression) [incubator-gluten]

2024-05-13 Thread via GitHub
ivoson commented on issue #5700: URL: https://github.com/apache/incubator-gluten/issues/5700#issuecomment-2109203576 > In our plan, the validation should all happens in native library. native library should have all the info it supports the expression/operators or not. But Velox itself

Re: [PR] [GLUTEN-5668][CH] Support mixed inequal conditions in hash join [incubator-gluten]

2024-05-13 Thread via GitHub
github-actions[bot] commented on PR #5735: URL: https://github.com/apache/incubator-gluten/pull/5735#issuecomment-2109200654 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [I] [VL] Unsupported spark function list [please leave a comment if you plan to pick some] [incubator-gluten]

2024-05-13 Thread via GitHub
marin-ma commented on issue #4039: URL: https://github.com/apache/incubator-gluten/issues/4039#issuecomment-2109187441 > I'd like to take a look at `stack` function, it seems like a `Generator`, meaning one row of input might return multiple rows of output, does Velox has this generator

Re: [PR] [GLUTEN-5414] [VL] Support datasource v2 scan csv [incubator-gluten]

2024-05-13 Thread via GitHub
github-actions[bot] commented on PR #5717: URL: https://github.com/apache/incubator-gluten/pull/5717#issuecomment-2109185724 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [I] Shuffle write time metric is wrong [incubator-gluten]

2024-05-13 Thread via GitHub
hahazyb201 commented on issue #5731: URL: https://github.com/apache/incubator-gluten/issues/5731#issuecomment-2109185313 Hi, I think Payload#writeTime_ is actually counted in RssPartitionWriter::stop at this

Re: [PR] [VL] Refine memory changes logic when growCapacity [incubator-gluten]

2024-05-13 Thread via GitHub
GlutenPerfBot commented on PR #5690: URL: https://github.com/apache/incubator-gluten/pull/5690#issuecomment-2109178794 = Performance report for TPCH SF2000 with Velox backend, for reference only query

Re: [PR] [GLUTEN-5668][CH] Support mixed inequal conditions in hash join [incubator-gluten]

2024-05-13 Thread via GitHub
github-actions[bot] commented on PR #5735: URL: https://github.com/apache/incubator-gluten/pull/5735#issuecomment-2109171538 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [WIP][VL] Support celeborn sort based shuffle [incubator-gluten]

2024-05-13 Thread via GitHub
github-actions[bot] commented on PR #5675: URL: https://github.com/apache/incubator-gluten/pull/5675#issuecomment-2109169780 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-5668][CH] Support mixed inequal conditions in hash join [incubator-gluten]

2024-05-13 Thread via GitHub
github-actions[bot] commented on PR #5735: URL: https://github.com/apache/incubator-gluten/pull/5735#issuecomment-2109164238 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-5668][CH] Support mixed inequal conditions in hash join [incubator-gluten]

2024-05-13 Thread via GitHub
github-actions[bot] commented on PR #5735: URL: https://github.com/apache/incubator-gluten/pull/5735#issuecomment-2109163246 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [I] [VL] Unsupported spark function list [please leave a comment if you plan to pick some] [incubator-gluten]

2024-05-13 Thread via GitHub
xumingming commented on issue #4039: URL: https://github.com/apache/incubator-gluten/issues/4039#issuecomment-2109163864 I'd like to take a look at `stack` function, it seems like a `Generator`, meaning one row of input might return multiple rows of output, does Velox has this generator

Re: [PR] [WIP][VL] Support celeborn sort based shuffle [incubator-gluten]

2024-05-13 Thread via GitHub
kerwin-zk commented on code in PR #5675: URL: https://github.com/apache/incubator-gluten/pull/5675#discussion_r1599300738 ## cpp/core/jni/JniWrapper.cc: ## @@ -148,6 +149,69 @@ class JavaInputStreamAdaptor final : public arrow::io::InputStream { bool closed_ = false; };

Re: [PR] [GLUTEN-5668][CH] Support mixed inequal conditions in hash join [incubator-gluten]

2024-05-13 Thread via GitHub
github-actions[bot] commented on PR #5735: URL: https://github.com/apache/incubator-gluten/pull/5735#issuecomment-2109159126 https://github.com/apache/incubator-gluten/issues/5668 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] [GLUTEN-5668][CH] Support mixed inequal conditions in hash join [incubator-gluten]

2024-05-13 Thread via GitHub
github-actions[bot] commented on PR #5735: URL: https://github.com/apache/incubator-gluten/pull/5735#issuecomment-2109159381 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [CORE] Add a compilation-time check to forbid case-class inheritance [incubator-gluten]

2024-05-13 Thread via GitHub
github-actions[bot] commented on PR #5729: URL: https://github.com/apache/incubator-gluten/pull/5729#issuecomment-2109157420 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [WIP][VL] Support celeborn sort based shuffle [incubator-gluten]

2024-05-13 Thread via GitHub
marin-ma commented on code in PR #5675: URL: https://github.com/apache/incubator-gluten/pull/5675#discussion_r1599290918 ## cpp/core/jni/JniWrapper.cc: ## @@ -148,6 +149,69 @@ class JavaInputStreamAdaptor final : public arrow::io::InputStream { bool closed_ = false; };

Re: [PR] [WIP][VL] Support celeborn sort based shuffle [incubator-gluten]

2024-05-13 Thread via GitHub
kerwin-zk commented on code in PR #5675: URL: https://github.com/apache/incubator-gluten/pull/5675#discussion_r1599288631 ## cpp/core/jni/JniWrapper.cc: ## @@ -148,6 +149,69 @@ class JavaInputStreamAdaptor final : public arrow::io::InputStream { bool closed_ = false; };

Re: [PR] [WIP][VL] Support celeborn sort based shuffle [incubator-gluten]

2024-05-13 Thread via GitHub
github-actions[bot] commented on PR #5675: URL: https://github.com/apache/incubator-gluten/pull/5675#issuecomment-2109140852 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [I] Fix some typos in NewToGluten.md [incubator-gluten]

2024-05-13 Thread via GitHub
Yohahaha closed issue #5708: Fix some typos in NewToGluten.md URL: https://github.com/apache/incubator-gluten/issues/5708 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [I] spillTime_ is been calculated twice? [incubator-gluten]

2024-05-13 Thread via GitHub
Yohahaha closed issue #5724: spillTime_ is been calculated twice? URL: https://github.com/apache/incubator-gluten/issues/5724 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] [CORE] Add a compilation-time check to forbid case-class inheritance [incubator-gluten]

2024-05-13 Thread via GitHub
github-actions[bot] commented on PR #5729: URL: https://github.com/apache/incubator-gluten/pull/5729#issuecomment-2109130427 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL] Refine memory changes logic when growCapacity [incubator-gluten]

2024-05-13 Thread via GitHub
Yohahaha commented on PR #5690: URL: https://github.com/apache/incubator-gluten/pull/5690#issuecomment-2109123992 /Benchmark Velox -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [I] [CH]Invalid result from converting `x in (1, 2)` in [incubator-gluten]

2024-05-13 Thread via GitHub
lgbo-ustc closed issue #5734: [CH]Invalid result from converting `x in (1, 2)` in URL: https://github.com/apache/incubator-gluten/issues/5734 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] [VL] Add Support for tencentos 2.4 [incubator-gluten]

2024-05-13 Thread via GitHub
github-actions[bot] commented on PR #5207: URL: https://github.com/apache/incubator-gluten/pull/5207#issuecomment-2109121374 This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days. -- This is an automated

(incubator-gluten) branch main updated: [GLUTEN-5724][VL] Remove redundant counter for calculating VeloxShuffleWriter spill time (#5725)

2024-05-13 Thread marong
This is an automated email from the ASF dual-hosted git repository. marong pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git The following commit(s) were added to refs/heads/main by this push: new 182a02999 [GLUTEN-5724][VL] Remove

Re: [PR] [GLUTEN-5724][VL] Remove redundant counter for calculating VeloxShuffleWriter spill time [incubator-gluten]

2024-05-13 Thread via GitHub
marin-ma merged PR #5725: URL: https://github.com/apache/incubator-gluten/pull/5725 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [I] Run `jstack` against Spark Driver process failed on MacOS M1 [incubator-gluten]

2024-05-13 Thread via GitHub
xumingming commented on issue #5702: URL: https://github.com/apache/incubator-gluten/issues/5702#issuecomment-2109116977 I am using macOS(Apple Silicon), JDK: ``` openjdk version "1.8.0_402" OpenJDK Runtime Environment (Zulu 8.76.0.17-CA-macos-aarch64) (build 1.8.0_402-b06)

[I] [CH]Invalid result from convert `x in (1, 2)` in [incubator-gluten]

2024-05-13 Thread via GitHub
lgbo-ustc opened a new issue, #5734: URL: https://github.com/apache/incubator-gluten/issues/5734 ### Backend CH (ClickHouse) ### Bug description [Expected behavior] and [actual behavior]. For a expression like `x in (1, 2)`, the following two ways have different

Re: [PR] [VL] Fallback Window operator if there is a mismatch in output type [incubator-gluten]

2024-05-13 Thread via GitHub
github-actions[bot] commented on PR #5554: URL: https://github.com/apache/incubator-gluten/pull/5554#issuecomment-2109093713 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-5620][CORE] Remove check_overflow and refactor code [incubator-gluten]

2024-05-13 Thread via GitHub
jinchengchenghh commented on code in PR #5654: URL: https://github.com/apache/incubator-gluten/pull/5654#discussion_r1599242846 ## gluten-core/src/main/scala/org/apache/gluten/expression/ExpressionConverter.scala: ## @@ -492,7 +514,6 @@ object ExpressionConverter extends

Re: [PR] [VL] Daily Update Velox Version (2024_05_14) [incubator-gluten]

2024-05-13 Thread via GitHub
github-actions[bot] commented on PR #5733: URL: https://github.com/apache/incubator-gluten/pull/5733#issuecomment-2109033205 Thanks for opening a pull request! Could you open an issue for this pull request on Github Issues?

[PR] [VL] Daily Update Velox Version (2024_05_14) [incubator-gluten]

2024-05-13 Thread via GitHub
GlutenPerfBot opened a new pull request, #5733: URL: https://github.com/apache/incubator-gluten/pull/5733 Upstream Velox's New Commits: ```txt b29d93374 by Wei He, Add overwrite flag to simple function registration API (9158) a54929be5 by xiaoxmeng, Re-enable spill in

Re: [PR] [CORE] Add a compilation-time check to forbid case-class inheritance [incubator-gluten]

2024-05-13 Thread via GitHub
zhztheplayer commented on code in PR #5729: URL: https://github.com/apache/incubator-gluten/pull/5729#discussion_r1599227484 ## pom.xml: ## @@ -615,13 +615,28 @@ scala-maven-plugin ${scala.compiler.version} + + +

Re: [PR] [CORE] Add a compilation-time check to forbid case-class inheritance [incubator-gluten]

2024-05-13 Thread via GitHub
github-actions[bot] commented on PR #5729: URL: https://github.com/apache/incubator-gluten/pull/5729#issuecomment-2109031542 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-1632][CH]Daily Update Clickhouse Version (20240514) [incubator-gluten]

2024-05-13 Thread via GitHub
github-actions[bot] commented on PR #5732: URL: https://github.com/apache/incubator-gluten/pull/5732#issuecomment-2108947482 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-1632][CH]Daily Update Clickhouse Version (20240514) [incubator-gluten]

2024-05-13 Thread via GitHub
github-actions[bot] commented on PR #5732: URL: https://github.com/apache/incubator-gluten/pull/5732#issuecomment-2108947259 https://github.com/apache/incubator-gluten/issues/1632 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[PR] [GLUTEN-1632][CH]Daily Update Clickhouse Version (20240514) [incubator-gluten]

2024-05-13 Thread via GitHub
kyligence-git opened a new pull request, #5732: URL: https://github.com/apache/incubator-gluten/pull/5732 Auto commit by gluten daily build, please check the build status and merge it if it's green. -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] [GLUTEN-4652][VL] Fix min_by/max_by result mismatch when RDD partition num > 1 [incubator-gluten]

2024-05-13 Thread via GitHub
GlutenPerfBot commented on PR #5711: URL: https://github.com/apache/incubator-gluten/pull/5711#issuecomment-2108637796 = Performance report for TPCH SF2000 with Velox backend, for reference only query

Re: [PR] [GLUTEN-4652][VL] Fix min_by/max_by result mismatch when RDD partition num > 1 [incubator-gluten]

2024-05-13 Thread via GitHub
GlutenPerfBot commented on PR #5711: URL: https://github.com/apache/incubator-gluten/pull/5711#issuecomment-2108529275 = Performance report for TPCDS SF2000 with Velox backend, for reference only query

Re: [PR] [WIP][VL] Support celeborn sort based shuffle [incubator-gluten]

2024-05-13 Thread via GitHub
github-actions[bot] commented on PR #5675: URL: https://github.com/apache/incubator-gluten/pull/5675#issuecomment-2108108869 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [I] Shuffle write time metric is wrong [incubator-gluten]

2024-05-13 Thread via GitHub
Yohahaha commented on issue #5731: URL: https://github.com/apache/incubator-gluten/issues/5731#issuecomment-2107857019 `RssPartitionWriter::stop` does not count `Payload#writeTime_`, so I think write time does not count twice in celeborn. -- This is an automated message from the Apache

Re: [PR] [VL] Refine memory changes logic when growCapacity [incubator-gluten]

2024-05-13 Thread via GitHub
github-actions[bot] commented on PR #5690: URL: https://github.com/apache/incubator-gluten/pull/5690#issuecomment-2107768946 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [WIP][VL] Support celeborn sort based shuffle [incubator-gluten]

2024-05-13 Thread via GitHub
github-actions[bot] commented on PR #5675: URL: https://github.com/apache/incubator-gluten/pull/5675#issuecomment-2107516246 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL] Refine memory changes logic when growCapacity [incubator-gluten]

2024-05-13 Thread via GitHub
github-actions[bot] commented on PR #5690: URL: https://github.com/apache/incubator-gluten/pull/5690#issuecomment-2107438951 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL]: Fix VeloxColumnarWriteFilesExecwithNewChildren doesn't replace the dummy child [incubator-gluten]

2024-05-13 Thread via GitHub
GlutenPerfBot commented on PR #5726: URL: https://github.com/apache/incubator-gluten/pull/5726#issuecomment-2107308119 = Performance report for TPCH SF2000 with Velox backend, for reference only query

[I] Shuffle write time metric is wrong [incubator-gluten]

2024-05-13 Thread via GitHub
hahazyb201 opened a new issue, #5731: URL: https://github.com/apache/incubator-gluten/issues/5731 ### Backend VL (Velox) ### Bug description In the DAG, when I observe the "shuffle write time total" metric, I found it was much bigger than I expected. So I dive deep into

(incubator-gluten) branch main updated: [GLUTEN-4652][VL] Fix min_by/max_by result mismatch when RDD partition num > 1 (#5711)

2024-05-13 Thread ulyssesyou
This is an automated email from the ASF dual-hosted git repository. ulyssesyou pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git The following commit(s) were added to refs/heads/main by this push: new 33f993554 [GLUTEN-4652][VL] Fix

Re: [PR] [GLUTEN-4652][VL] Fix min_by/max_by result mismatch when RDD partition num > 1 [incubator-gluten]

2024-05-13 Thread via GitHub
ulysses-you merged PR #5711: URL: https://github.com/apache/incubator-gluten/pull/5711 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] [GLUTEN-4652][VL] Fix min_by/max_by result mismatch when RDD partition num > 1 [incubator-gluten]

2024-05-13 Thread via GitHub
zhouyifan279 commented on PR #5711: URL: https://github.com/apache/incubator-gluten/pull/5711#issuecomment-2107249584 @yma11 CI `run-spark-test-spark34`'s failure seems irrelevant with this PR. Can you rerun it? -- This is an automated message from the Apache Git Service. To respond to

(incubator-gluten) branch main updated: [VL]: Fix VeloxColumnarWriteFilesExecwithNewChildren doesn't replace the dummy child (#5726)

2024-05-13 Thread hongze
This is an automated email from the ASF dual-hosted git repository. hongze pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git The following commit(s) were added to refs/heads/main by this push: new 600a5eeb9 [VL]: Fix

Re: [PR] [VL]: Fix VeloxColumnarWriteFilesExecwithNewChildren doesn't replace the dummy child [incubator-gluten]

2024-05-13 Thread via GitHub
zhztheplayer merged PR #5726: URL: https://github.com/apache/incubator-gluten/pull/5726 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] [GLUTEN-5691][CH] Enable merge on local disk first after insert into mergetree [incubator-gluten]

2024-05-13 Thread via GitHub
github-actions[bot] commented on PR #5692: URL: https://github.com/apache/incubator-gluten/pull/5692#issuecomment-2107139071 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL][DOC] Suggest enable vcpkg for docker build in centos [incubator-gluten]

2024-05-13 Thread via GitHub
zml1206 closed pull request #5641: [VL][DOC] Suggest enable vcpkg for docker build in centos URL: https://github.com/apache/incubator-gluten/pull/5641 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] [VL][DOC] Suggest enable vcpkg for docker build in centos [incubator-gluten]

2024-05-13 Thread via GitHub
zml1206 commented on PR #5641: URL: https://github.com/apache/incubator-gluten/pull/5641#issuecomment-2107124748 Enable vcpkg=on as default is going on, so close it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] [VL] Refine memory changes logic when growCapacity [incubator-gluten]

2024-05-13 Thread via GitHub
github-actions[bot] commented on PR #5690: URL: https://github.com/apache/incubator-gluten/pull/5690#issuecomment-2107119930 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL] Add test for shuffle function [incubator-gluten]

2024-05-13 Thread via GitHub
exmy commented on code in PR #5722: URL: https://github.com/apache/incubator-gluten/pull/5722#discussion_r1598167365 ## gluten-core/src/main/scala/org/apache/gluten/backendsapi/SparkPlanExecApi.scala: ## @@ -211,6 +211,13 @@ trait SparkPlanExecApi {

Re: [PR] [CORE] Remove wrong comment for JoinSelectionOverrides [incubator-gluten]

2024-05-13 Thread via GitHub
github-actions[bot] commented on PR #5730: URL: https://github.com/apache/incubator-gluten/pull/5730#issuecomment-2107093942 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [CORE] Remove wrong comment for JoinSelectionOverrides [incubator-gluten]

2024-05-13 Thread via GitHub
github-actions[bot] commented on PR #5730: URL: https://github.com/apache/incubator-gluten/pull/5730#issuecomment-2107093510 Thanks for opening a pull request! Could you open an issue for this pull request on Github Issues?

[PR] [CORE] Remove wrong comment for JoinSelectionOverrides [incubator-gluten]

2024-05-13 Thread via GitHub
zml1206 opened a new pull request, #5730: URL: https://github.com/apache/incubator-gluten/pull/5730 ## What changes were proposed in this pull request? `ExtractEquiJoinKeysShim.unapply` calls `ExtractEquiJoinKeys`, `ExtractEquiJoinKeys` determines that there must have an equal condition.

Re: [PR] [CORE] Add a compilation-time check to forbid case-class inheritance [incubator-gluten]

2024-05-13 Thread via GitHub
PHILO-HE commented on code in PR #5729: URL: https://github.com/apache/incubator-gluten/pull/5729#discussion_r1598150719 ## pom.xml: ## @@ -615,13 +615,28 @@ scala-maven-plugin ${scala.compiler.version} + + +

Re: [I] Can I install Gluten in AWS EMR? Unsupport linux distribution: amzn [incubator-gluten]

2024-05-13 Thread via GitHub
PHILO-HE commented on issue #5689: URL: https://github.com/apache/incubator-gluten/issues/5689#issuecomment-2107014113 @astarrr, I guess it can work on your centos-like or ubuntu-like OS after a few code changes are made. Currently, we hard-coded the supported OS. To let the script for

Re: [PR] [CORE] Add a compilation-time check to forbid case-class inheritance [incubator-gluten]

2024-05-13 Thread via GitHub
github-actions[bot] commented on PR #5729: URL: https://github.com/apache/incubator-gluten/pull/5729#issuecomment-2107003165 Thanks for opening a pull request! Could you open an issue for this pull request on Github Issues?

Re: [PR] [VL] Add test for shuffle function [incubator-gluten]

2024-05-13 Thread via GitHub
GlutenPerfBot commented on PR #5722: URL: https://github.com/apache/incubator-gluten/pull/5722#issuecomment-2106986817 = Performance report for TPCH SF2000 with Velox backend, for reference only query

Re: [PR] [CORE] Add a compilation-time check to forbid case-class inheritance [incubator-gluten]

2024-05-13 Thread via GitHub
github-actions[bot] commented on PR #5729: URL: https://github.com/apache/incubator-gluten/pull/5729#issuecomment-2106950701 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [CORE] Add a compilation-time check to forbid case-class inheritance [incubator-gluten]

2024-05-13 Thread via GitHub
github-actions[bot] commented on PR #5729: URL: https://github.com/apache/incubator-gluten/pull/5729#issuecomment-2106950319 Thanks for opening a pull request! Could you open an issue for this pull request on Github Issues?

[PR] [CORE] Add a compilation-time check to forbid case-class inheritance [incubator-gluten]

2024-05-13 Thread via GitHub
zhztheplayer opened a new pull request, #5729: URL: https://github.com/apache/incubator-gluten/pull/5729 This reopens https://github.com/apache/incubator-gluten/pull/5723 with a fix for building with Intellij Idea. -- This is an automated message from the Apache Git Service. To respond

Re: [PR] [GLUTEN-5414] [VL] Support datasource v2 scan csv [incubator-gluten]

2024-05-13 Thread via GitHub
github-actions[bot] commented on PR #5717: URL: https://github.com/apache/incubator-gluten/pull/5717#issuecomment-2106930432 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL] [Test] test dense rank support in RowsStreamingWindow [incubator-gluten]

2024-05-13 Thread via GitHub
github-actions[bot] commented on PR #5728: URL: https://github.com/apache/incubator-gluten/pull/5728#issuecomment-2106899056 Thanks for opening a pull request! Could you open an issue for this pull request on Github Issues?

Re: [PR] [GLUTEN-5620][CORE] Remove check_overflow and refactor code [incubator-gluten]

2024-05-13 Thread via GitHub
baibaichen commented on code in PR #5654: URL: https://github.com/apache/incubator-gluten/pull/5654#discussion_r1598026841 ## gluten-core/src/main/scala/org/apache/gluten/expression/ExpressionConverter.scala: ## @@ -492,7 +514,6 @@ object ExpressionConverter extends

Re: [PR] Revert "[CORE] Add a compilation-time check to forbid case-class inheritance " [incubator-gluten]

2024-05-13 Thread via GitHub
github-actions[bot] commented on PR #5727: URL: https://github.com/apache/incubator-gluten/pull/5727#issuecomment-2106882729 Thanks for opening a pull request! Could you open an issue for this pull request on Github Issues?

(incubator-gluten) 01/01: Revert "[CORE] Add a compilation-time check to forbid case-class inheritance"

2024-05-13 Thread hongze
This is an automated email from the ASF dual-hosted git repository. hongze pushed a commit to branch revert-5723-wip-forbid-inherit-caseclass in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git commit e33432bba6a2ffe48fbc8267001e660c2c18d407 Author: Hongze Zhang AuthorDate:

[PR] Revert "[CORE] Add a compilation-time check to forbid case-class inheritance " [incubator-gluten]

2024-05-13 Thread via GitHub
zhztheplayer opened a new pull request, #5727: URL: https://github.com/apache/incubator-gluten/pull/5727 Reverts apache/incubator-gluten#5723 Intellij doesn't pick the custom wart's code. Investigating. -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] Revert "[CORE] Add a compilation-time check to forbid case-class inheritance " [incubator-gluten]

2024-05-13 Thread via GitHub
github-actions[bot] commented on PR #5727: URL: https://github.com/apache/incubator-gluten/pull/5727#issuecomment-2106883136 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] Revert "[CORE] Add a compilation-time check to forbid case-class inheritance " [incubator-gluten]

2024-05-13 Thread via GitHub
zhztheplayer merged PR #5727: URL: https://github.com/apache/incubator-gluten/pull/5727 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

(incubator-gluten) branch main updated: Revert "[CORE] Add a compilation-time check to forbid case-class inheritance" (#5727)

2024-05-13 Thread hongze
This is an automated email from the ASF dual-hosted git repository. hongze pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git The following commit(s) were added to refs/heads/main by this push: new 512f4e150 Revert "[CORE] Add a

(incubator-gluten) branch revert-5723-wip-forbid-inherit-caseclass created (now e33432bba)

2024-05-13 Thread hongze
This is an automated email from the ASF dual-hosted git repository. hongze pushed a change to branch revert-5723-wip-forbid-inherit-caseclass in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git at e33432bba Revert "[CORE] Add a compilation-time check to forbid

Re: [PR] [VL] Add test for shuffle function [incubator-gluten]

2024-05-13 Thread via GitHub
PHILO-HE commented on code in PR #5722: URL: https://github.com/apache/incubator-gluten/pull/5722#discussion_r1598022007 ## gluten-core/src/main/scala/org/apache/gluten/backendsapi/SparkPlanExecApi.scala: ## @@ -211,6 +211,13 @@ trait SparkPlanExecApi {

Re: [PR] [VL]: Fix VeloxColumnarWriteFilesExecwithNewChildren doesn't replace the dummy child [incubator-gluten]

2024-05-13 Thread via GitHub
github-actions[bot] commented on PR #5726: URL: https://github.com/apache/incubator-gluten/pull/5726#issuecomment-2106872755 Thanks for opening a pull request! Could you open an issue for this pull request on Github Issues?

[PR] [VL]: Fix VeloxColumnarWriteFilesExecwithNewChildren doesn't replace the dummy child [incubator-gluten]

2024-05-13 Thread via GitHub
zhztheplayer opened a new pull request, #5726: URL: https://github.com/apache/incubator-gluten/pull/5726 This is a bugfix extracted from PR https://github.com/apache/incubator-gluten/pull/5698. -- This is an automated message from the Apache Git Service. To respond to the message,

(incubator-gluten) branch main updated: [VL] Add test for shuffle function (#5722)

2024-05-13 Thread zhli
This is an automated email from the ASF dual-hosted git repository. zhli pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git The following commit(s) were added to refs/heads/main by this push: new 6db8920e6 [VL] Add test for shuffle

Re: [PR] [VL] Add test for shuffle function [incubator-gluten]

2024-05-13 Thread via GitHub
zhli1142015 merged PR #5722: URL: https://github.com/apache/incubator-gluten/pull/5722 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [I] [VL] Add SampleExec operator support [incubator-gluten]

2024-05-13 Thread via GitHub
gaoyangxiaozhu commented on issue #5315: URL: https://github.com/apache/incubator-gluten/issues/5315#issuecomment-2106856251 super thanks @rui-mo / @PHILO-HE for the confirm. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

(incubator-gluten) branch main updated: [CORE] Add a compilation-time check to forbid case-class inheritance

2024-05-13 Thread hongze
This is an automated email from the ASF dual-hosted git repository. hongze pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git The following commit(s) were added to refs/heads/main by this push: new f5a6c98bb [CORE] Add a compilation-time

Re: [PR] [CORE] Add a compilation-time check to forbid case-class inheritance [incubator-gluten]

2024-05-13 Thread via GitHub
zhztheplayer merged PR #5723: URL: https://github.com/apache/incubator-gluten/pull/5723 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] [GLUTEN-4652][VL] Fix min_by/max_by result mismatch when RDD partition num > 1 [incubator-gluten]

2024-05-13 Thread via GitHub
yma11 commented on PR #5711: URL: https://github.com/apache/incubator-gluten/pull/5711#issuecomment-2106811681 > > > @yma11 Did you reproduced the problem I mentioned above based on the master gluten branch? That is what I'm trying to fix. > > > > > > I mean which part of your

Re: [PR] [GLUTEN-4652][VL] Fix min_by/max_by result mismatch when RDD partition num > 1 [incubator-gluten]

2024-05-13 Thread via GitHub
zhouyifan279 commented on PR #5711: URL: https://github.com/apache/incubator-gluten/pull/5711#issuecomment-2106806866 > > @yma11 Did you reproduced the problem I mentioned above based on the master gluten branch? That is what I'm trying to fix. > > I mean which part of your code

Re: [PR] [VL] Add test for shuffle function [incubator-gluten]

2024-05-13 Thread via GitHub
zhli1142015 commented on PR #5722: URL: https://github.com/apache/incubator-gluten/pull/5722#issuecomment-2106806635 cc @PHILO-HE and @rui-mo , thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [WIP][VL] Support celeborn sort based shuffle [incubator-gluten]

2024-05-13 Thread via GitHub
marin-ma commented on code in PR #5675: URL: https://github.com/apache/incubator-gluten/pull/5675#discussion_r1597965285 ## cpp/velox/shuffle/VeloxShuffleWriter.h: ## @@ -132,6 +132,8 @@ class VeloxShuffleWriter final : public ShuffleWriter { arrow::Status

  1   2   >