Re: [PR] [VL] Support posexplode function and code refactoring on GenerateExecTransformer [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #4901: URL: https://github.com/apache/incubator-gluten/pull/4901#issuecomment-1990870848 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [I] [VL] Results are mismatch with the vanilla Spark when cast(sum(decmial(20,4)), float) [incubator-gluten]

2024-03-12 Thread via GitHub
kecookier commented on issue #4891: URL: https://github.com/apache/incubator-gluten/issues/4891#issuecomment-1990911877 > I've identified the error code, and I will submit a velox PR as soon as possible. cc @rui-mo -- This is an automated message from the Apache Git Service. To

Re: [PR] [VL] Daily Update Velox Version (2024_03_12) [incubator-gluten]

2024-03-12 Thread via GitHub
GlutenPerfBot commented on PR #4923: URL: https://github.com/apache/incubator-gluten/pull/4923#issuecomment-1990927461 = Performance report for TPCH SF2000 with Velox backend, for reference only query

Re: [PR] [GLUTEN-4745][CH] support Sort Merge Join [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #4812: URL: https://github.com/apache/incubator-gluten/pull/4812#issuecomment-1990773004 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL] Fix wrong plan equality due to case class inheritance [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #4893: URL: https://github.com/apache/incubator-gluten/pull/4893#issuecomment-1990839852 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-4745][CH] support Sort Merge Join [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #4812: URL: https://github.com/apache/incubator-gluten/pull/4812#issuecomment-1990841409 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-4796][VL] Add orc char type validation [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #4797: URL: https://github.com/apache/incubator-gluten/pull/4797#issuecomment-1990718346 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL] Daily Update Velox Version (2024_03_12) [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #4923: URL: https://github.com/apache/incubator-gluten/pull/4923#issuecomment-1990855788 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [CORE] Prior to #4893, add vanilla Spark's original scan source code to keep git history [incubator-gluten]

2024-03-12 Thread via GitHub
Yohahaha commented on PR #4931: URL: https://github.com/apache/incubator-gluten/pull/4931#issuecomment-1990906639 > > could you add Spark commit id as well? > > It should be from Spark GitHub repo's version tags, but I am not 100% sure. Some of them were fetched from their released

Re: [PR] [GLUTEN-4796][VL] Add orc char type validation [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #4797: URL: https://github.com/apache/incubator-gluten/pull/4797#issuecomment-1990727003 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [DNM] Velox test [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #4929: URL: https://github.com/apache/incubator-gluten/pull/4929#issuecomment-1990886520 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-4745][CH] support Sort Merge Join [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #4812: URL: https://github.com/apache/incubator-gluten/pull/4812#issuecomment-1990901831 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL] Support posexplode function and code refactoring on GenerateExecTransformer [incubator-gluten]

2024-03-12 Thread via GitHub
liujiayi771 commented on code in PR #4901: URL: https://github.com/apache/incubator-gluten/pull/4901#discussion_r1520944825 ## backends-velox/src/main/scala/io/glutenproject/execution/GenerateExecTransformer.scala: ## @@ -0,0 +1,232 @@ +/* + * Licensed to the Apache Software

Re: [PR] [CORE] Prior to #4893, add vanilla Spark's original scan source code to keep git history [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #4931: URL: https://github.com/apache/incubator-gluten/pull/4931#issuecomment-1990830817 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-4745][CH] support Sort Merge Join [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #4812: URL: https://github.com/apache/incubator-gluten/pull/4812#issuecomment-1990824258 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [CORE] Prior to #4893, add vanilla Spark's original scan source code to keep git history [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #4931: URL: https://github.com/apache/incubator-gluten/pull/4931#issuecomment-1990830168 Thanks for opening a pull request! Could you open an issue for this pull request on Github Issues?

[PR] [CORE] Prior to #4893, add vanilla Spark's original scan source code to keep git history [incubator-gluten]

2024-03-12 Thread via GitHub
zhztheplayer opened a new pull request, #4931: URL: https://github.com/apache/incubator-gluten/pull/4931 This should be merged as an individual commit before merging https://github.com/apache/incubator-gluten/pull/4893. To keep the git history then further developers would know how

Re: [PR] [CORE] Prior to #4893, add vanilla Spark's original scan source code to keep git history [incubator-gluten]

2024-03-12 Thread via GitHub
zhztheplayer commented on PR #4931: URL: https://github.com/apache/incubator-gluten/pull/4931#issuecomment-1990904386 > could you add Spark commit id as well? It should be from Spark GitHub repo's version tags, but I am not 100% sure. Some of them were fetched from their released

Re: [PR] [VL] Fix wrong plan equality due to case class inheritance [incubator-gluten]

2024-03-12 Thread via GitHub
Yohahaha commented on PR #4893: URL: https://github.com/apache/incubator-gluten/pull/4893#issuecomment-1990903745 seems copy Apache Spark source file into Gluten brings a limitation that require user must use same commit id or tag of these copied source file, if user has modified these

Re: [PR] [VL] Daily Update Velox Version (2024_03_12) [incubator-gluten]

2024-03-12 Thread via GitHub
yma11 commented on PR #4923: URL: https://github.com/apache/incubator-gluten/pull/4923#issuecomment-1990810271 /Benchmark Velox -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] [VL] Support posexplode function and code refactoring on GenerateExecTransformer [incubator-gluten]

2024-03-12 Thread via GitHub
liujiayi771 commented on code in PR #4901: URL: https://github.com/apache/incubator-gluten/pull/4901#discussion_r1520944825 ## backends-velox/src/main/scala/io/glutenproject/execution/GenerateExecTransformer.scala: ## @@ -0,0 +1,232 @@ +/* + * Licensed to the Apache Software

Re: [PR] [VL] Support posexplode function and code refactoring on GenerateExecTransformer [incubator-gluten]

2024-03-12 Thread via GitHub
liujiayi771 commented on code in PR #4901: URL: https://github.com/apache/incubator-gluten/pull/4901#discussion_r1520945524 ## backends-velox/src/main/scala/io/glutenproject/execution/GenerateExecTransformer.scala: ## @@ -0,0 +1,232 @@ +/* + * Licensed to the Apache Software

[I] Besides Centos and Ubuntu systems, does the gluten project support the Maipo system? [incubator-gluten]

2024-03-12 Thread via GitHub
bigmancomeon opened a new issue, #4932: URL: https://github.com/apache/incubator-gluten/issues/4932 ### Description ![image](https://github.com/apache/incubator-gluten/assets/129055247/2b2b15b7-dea9-4e9c-a2d5-fbcbc3a7a446) As shown in the figure, my Linux machine is a Maipo

Re: [PR] [CORE] Prior to #4893, add vanilla Spark's original scan source code to keep git history [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #4931: URL: https://github.com/apache/incubator-gluten/pull/4931#issuecomment-1990836084 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL] Support posexplode function and code refactoring on GenerateExecTransformer [incubator-gluten]

2024-03-12 Thread via GitHub
marin-ma commented on code in PR #4901: URL: https://github.com/apache/incubator-gluten/pull/4901#discussion_r1520931741 ## backends-velox/src/main/scala/io/glutenproject/execution/GenerateExecTransformer.scala: ## @@ -0,0 +1,240 @@ +/* + * Licensed to the Apache Software

Re: [PR] [CORE] Prior to #4893, add vanilla Spark's original scan source code to keep git history [incubator-gluten]

2024-03-12 Thread via GitHub
Yohahaha commented on PR #4931: URL: https://github.com/apache/incubator-gluten/pull/4931#issuecomment-1990882392 could you add Spark commit id as well? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] [VL] Add uniffle integration [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #3767: URL: https://github.com/apache/incubator-gluten/pull/3767#issuecomment-1990970382 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

(incubator-gluten) branch main updated: [Gluten-4706] Add a mode to execute count distinct directly instead of Expand+Count (#4708)

2024-03-12 Thread mahongbin
This is an automated email from the ASF dual-hosted git repository. mahongbin pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git The following commit(s) were added to refs/heads/main by this push: new b07b36960 [Gluten-4706] Add a mode to

Re: [PR] [Gluten-4706] [CH][CORE] Add a mode to execute count distinct directly instead o… [incubator-gluten]

2024-03-12 Thread via GitHub
binmahone merged PR #4708: URL: https://github.com/apache/incubator-gluten/pull/4708 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [I] Add a mode to execute count distinct directly instead of Expand+Count [incubator-gluten]

2024-03-12 Thread via GitHub
binmahone closed issue #4706: Add a mode to execute count distinct directly instead of Expand+Count URL: https://github.com/apache/incubator-gluten/issues/4706 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] [VL] parquet file metadata columns support in velox [incubator-gluten]

2024-03-12 Thread via GitHub
zhouyuan commented on PR #3870: URL: https://github.com/apache/incubator-gluten/pull/3870#issuecomment-1991044843 CC: @yma11 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] [VL] Fix wrong plan equality due to case class inheritance [incubator-gluten]

2024-03-12 Thread via GitHub
Yohahaha commented on PR #4893: URL: https://github.com/apache/incubator-gluten/pull/4893#issuecomment-1991044848 > > f user has modified these source file in their own Spark > > Thanks for the comment here. It sounds like a very valid use case we should consider about. Do you

Re: [PR] [VL] Daily Update Velox Version (2024_03_12) [incubator-gluten]

2024-03-12 Thread via GitHub
PHILO-HE commented on code in PR #4923: URL: https://github.com/apache/incubator-gluten/pull/4923#discussion_r1521071799 ## gluten-ut/spark34/src/test/scala/io/glutenproject/utils/velox/VeloxTestSettings.scala: ## @@ -130,8 +130,6 @@ class VeloxTestSettings extends

Re: [PR] [VL] parquet file metadata columns support in velox [incubator-gluten]

2024-03-12 Thread via GitHub
yma11 commented on code in PR #3870: URL: https://github.com/apache/incubator-gluten/pull/3870#discussion_r1521186042 ## gluten-ut/spark33/src/test/scala/org/apache/spark/sql/execution/datasources/GlutenFileMetadataStructSuite.scala: ## @@ -16,6 +16,156 @@ */ package

Re: [PR] [VL] parquet file metadata columns support in velox [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #3870: URL: https://github.com/apache/incubator-gluten/pull/3870#issuecomment-1991300996 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL] Daily Update Velox Version (2024_03_12) [incubator-gluten]

2024-03-12 Thread via GitHub
GlutenPerfBot commented on PR #4923: URL: https://github.com/apache/incubator-gluten/pull/4923#issuecomment-1991348456 = Performance report for TPCH SF2000 with Velox backend, for reference only query

Re: [PR] [VL] Fix wrong plan equality due to case class inheritance [incubator-gluten]

2024-03-12 Thread via GitHub
zhztheplayer commented on PR #4893: URL: https://github.com/apache/incubator-gluten/pull/4893#issuecomment-1991001143 > f user has modified these source file in their own Spark Thanks for the comment here. It sounds like a very valid use case we should consider about. Do you already

Re: [PR] [GLUTEN-4903][CELEBORN][WIP] Support multiple versions of Celeborn [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #4913: URL: https://github.com/apache/incubator-gluten/pull/4913#issuecomment-1991009107 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL] parquet file metadata columns support in velox [incubator-gluten]

2024-03-12 Thread via GitHub
yma11 commented on code in PR #3870: URL: https://github.com/apache/incubator-gluten/pull/3870#discussion_r1521062380 ## cpp/velox/compute/WholeStageResultIterator.cc: ## @@ -145,15 +145,31 @@ WholeStageResultIterator::WholeStageResultIterator( const auto& lengths =

Re: [PR] [VL] parquet file metadata columns support in velox [incubator-gluten]

2024-03-12 Thread via GitHub
zhouyuan commented on code in PR #3870: URL: https://github.com/apache/incubator-gluten/pull/3870#discussion_r1521101767 ## gluten-ut/spark33/src/test/scala/org/apache/spark/sql/execution/datasources/GlutenFileMetadataStructSuite.scala: ## @@ -16,6 +16,156 @@ */ package

Re: [PR] [VL] Daily Update Velox Version (2024_03_12) [incubator-gluten]

2024-03-12 Thread via GitHub
PHILO-HE merged PR #4923: URL: https://github.com/apache/incubator-gluten/pull/4923 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] [GLUTEN-4903][CELEBORN][WIP] Support multiple versions of Celeborn [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #4913: URL: https://github.com/apache/incubator-gluten/pull/4913#issuecomment-1990993641 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [Gluten-4912][CH]Support Specifying columns in clickhouse tables to b… [incubator-gluten]

2024-03-12 Thread via GitHub
GlutenPerfBot commented on PR #4925: URL: https://github.com/apache/incubator-gluten/pull/4925#issuecomment-1991076575 = Performance report for TPCH SF2000 with Velox backend, for reference only query

Re: [PR] [VL] Fix wrong plan equality due to case class inheritance [incubator-gluten]

2024-03-12 Thread via GitHub
zhztheplayer commented on PR #4893: URL: https://github.com/apache/incubator-gluten/pull/4893#issuecomment-1991077056 > shim layer always complex, but copy source code is risky and hard to maintain, Once the abstract classes are added, I actually don't want them to be maintained

Re: [PR] [GLUTEN-4926][CELEBORN] `CelebornShuffleManager` should remove `shuffleId` from `columnarShuffleIds` after unregistering shuffle [incubator-gluten]

2024-03-12 Thread via GitHub
ulysses-you merged PR #4927: URL: https://github.com/apache/incubator-gluten/pull/4927 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [I] CelebornShuffleManager should remove shuffleId from columnarShuffleIds after unregistering shuffle [incubator-gluten]

2024-03-12 Thread via GitHub
ulysses-you closed issue #4926: CelebornShuffleManager should remove shuffleId from columnarShuffleIds after unregistering shuffle URL: https://github.com/apache/incubator-gluten/issues/4926 -- This is an automated message from the Apache Git Service. To respond to the message, please log on

(incubator-gluten) branch main updated: [GLUTEN-4926][CELEBORN] CelebornShuffleManager should remove shuffleId from columnarShuffleIds after unregistering shuffle (#4927)

2024-03-12 Thread ulyssesyou
This is an automated email from the ASF dual-hosted git repository. ulyssesyou pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git The following commit(s) were added to refs/heads/main by this push: new 494692d3f [GLUTEN-4926][CELEBORN]

[I] Bump Iceberg version [incubator-gluten]

2024-03-12 Thread via GitHub
SteNicholas opened a new issue, #4933: URL: https://github.com/apache/incubator-gluten/issues/4933 ### Description Iceberg has already released [1.4.0](https://github.com/apache/iceberg/releases/tag/apache-iceberg-1.4.0),

[I] Besides Centos and Ubuntu systems, does the gluten project support the RedHat 7 system? [incubator-gluten]

2024-03-12 Thread via GitHub
bigmancomeon opened a new issue, #4935: URL: https://github.com/apache/incubator-gluten/issues/4935 ### Backend VL (Velox) ### Bug description [Expected behavior] and [actual behavior].

Re: [PR] [VL] Refine log plan/split json into one line [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #4934: URL: https://github.com/apache/incubator-gluten/pull/4934#issuecomment-1990973526 Thanks for opening a pull request! Could you open an issue for this pull request on Github Issues?

Re: [I] Website not accessible? [incubator-gluten]

2024-03-12 Thread via GitHub
weiting-chen closed issue #4921: Website not accessible? URL: https://github.com/apache/incubator-gluten/issues/4921 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [I] Website not accessible? [incubator-gluten]

2024-03-12 Thread via GitHub
weiting-chen commented on issue #4921: URL: https://github.com/apache/incubator-gluten/issues/4921#issuecomment-1990991067 The website(https://gluten.apache.org/) is online now. I will close this issue. -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] [GLUTEN-3559][VL] enable more sql query tests for Spark34 [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #4880: URL: https://github.com/apache/incubator-gluten/pull/4880#issuecomment-1991019844 https://github.com/apache/incubator-gluten/issues/3559 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] [VL] Fix wrong plan equality due to case class inheritance [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #4893: URL: https://github.com/apache/incubator-gluten/pull/4893#issuecomment-1991062490 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL] parquet file metadata columns support in velox [incubator-gluten]

2024-03-12 Thread via GitHub
yma11 commented on code in PR #3870: URL: https://github.com/apache/incubator-gluten/pull/3870#discussion_r1521090112 ## cpp/velox/substrait/SubstraitParser.cc: ## @@ -104,31 +104,41 @@ std::vector SubstraitParser::parseNamedStruct(const ::substrait::NamedS return typeList;

(incubator-gluten) branch main updated (b07b36960 -> 4ee2ddb1d)

2024-03-12 Thread philo
This is an automated email from the ASF dual-hosted git repository. philo pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git from b07b36960 [Gluten-4706] Add a mode to execute count distinct directly instead of Expand+Count (#4708)

Re: [PR] [VL] parquet file metadata columns support in velox [incubator-gluten]

2024-03-12 Thread via GitHub
yma11 commented on code in PR #3870: URL: https://github.com/apache/incubator-gluten/pull/3870#discussion_r1521102914 ## gluten-core/src/main/scala/io/glutenproject/execution/BasicScanExecTransformer.scala: ## @@ -37,12 +37,15 @@ import com.google.protobuf.StringValue import

Re: [PR] [VL] parquet file metadata columns support in velox [incubator-gluten]

2024-03-12 Thread via GitHub
gaoyangxiaozhu commented on code in PR #3870: URL: https://github.com/apache/incubator-gluten/pull/3870#discussion_r1521142100 ## cpp/velox/substrait/SubstraitParser.cc: ## @@ -104,31 +104,41 @@ std::vector SubstraitParser::parseNamedStruct(const ::substrait::NamedS return

Re: [PR] [VL] parquet file metadata columns support in velox [incubator-gluten]

2024-03-12 Thread via GitHub
yma11 commented on code in PR #3870: URL: https://github.com/apache/incubator-gluten/pull/3870#discussion_r1521062380 ## cpp/velox/compute/WholeStageResultIterator.cc: ## @@ -145,15 +145,31 @@ WholeStageResultIterator::WholeStageResultIterator( const auto& lengths =

Re: [PR] [VL] parquet file metadata columns support in velox [incubator-gluten]

2024-03-12 Thread via GitHub
yma11 commented on code in PR #3870: URL: https://github.com/apache/incubator-gluten/pull/3870#discussion_r1521185608 ## cpp/velox/substrait/SubstraitParser.cc: ## @@ -104,31 +104,41 @@ std::vector SubstraitParser::parseNamedStruct(const ::substrait::NamedS return typeList;

Re: [PR] [GLUTEN-4914][CH] Fix exceptions in ASTParser [incubator-gluten]

2024-03-12 Thread via GitHub
liuneng1994 merged PR #4916: URL: https://github.com/apache/incubator-gluten/pull/4916 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [I] [CH] unsupported function "unix_timestamp" [incubator-gluten]

2024-03-12 Thread via GitHub
liuneng1994 closed issue #4914: [CH] unsupported function "unix_timestamp" URL: https://github.com/apache/incubator-gluten/issues/4914 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

(incubator-gluten) branch main updated: [GLUTEN-4914][CH] Fix exceptions in ASTParser #4916

2024-03-12 Thread liuneng
This is an automated email from the ASF dual-hosted git repository. liuneng pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git The following commit(s) were added to refs/heads/main by this push: new 8b43ac728 [GLUTEN-4914][CH] Fix

Re: [PR] [GLUTEN-4241][VL] Add plan node to convert Vanilla spark columnar format data to Velox columnar format data [incubator-gluten]

2024-03-12 Thread via GitHub
boneanxs commented on PR #4818: URL: https://github.com/apache/incubator-gluten/pull/4818#issuecomment-1991360394 Hey @FelixYBW `ArrowFieldWriter` calls arrow `ValueVector`, which is internally uses `ArrowBuf` to store values, so it should be offheap memory.

Re: [PR] [Gluten-4912][CH]Support Specifying columns in clickhouse tables to b… [incubator-gluten]

2024-03-12 Thread via GitHub
binmahone merged PR #4925: URL: https://github.com/apache/incubator-gluten/pull/4925 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [I] Support Specifying columns in clickhouse tables to be Low Cardinality [incubator-gluten]

2024-03-12 Thread via GitHub
binmahone closed issue #4912: Support Specifying columns in clickhouse tables to be Low Cardinality URL: https://github.com/apache/incubator-gluten/issues/4912 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

(incubator-gluten) branch main updated: [Gluten-4912][CH]Support Specifying columns in clickhouse tables to be Low Cardinality (#4925)

2024-03-12 Thread mahongbin
This is an automated email from the ASF dual-hosted git repository. mahongbin pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git The following commit(s) were added to refs/heads/main by this push: new 7ff18ee85 [Gluten-4912][CH]Support

Re: [PR] [VL] Support posexplode function and code refactoring on GenerateExecTransformer [incubator-gluten]

2024-03-12 Thread via GitHub
marin-ma commented on code in PR #4901: URL: https://github.com/apache/incubator-gluten/pull/4901#discussion_r1521018352 ## backends-velox/src/main/scala/io/glutenproject/execution/GenerateExecTransformer.scala: ## @@ -0,0 +1,232 @@ +/* + * Licensed to the Apache Software

Re: [PR] [GLUTEN-3559][VL] enable more sql query tests for Spark34 [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #4880: URL: https://github.com/apache/incubator-gluten/pull/4880#issuecomment-1991023581 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL] Fix wrong plan equality due to case class inheritance [incubator-gluten]

2024-03-12 Thread via GitHub
zhztheplayer commented on PR #4893: URL: https://github.com/apache/incubator-gluten/pull/4893#issuecomment-1991040313 > I believe the way below is better than copy more and more Spark source files into Gluten. In case of ambiguity, the files are pasted to Gluten with 'Abstract'

Re: [PR] [VL] Fix wrong plan equality due to case class inheritance [incubator-gluten]

2024-03-12 Thread via GitHub
Yohahaha commented on PR #4893: URL: https://github.com/apache/incubator-gluten/pull/4893#issuecomment-1991055517 > However it's not about directly overriding the same class with same name in Gluten which may lead to a bunch of problems like class loadings. Are we aligned here?

Re: [PR] [VL] Daily Update Velox Version (2024_03_12) [incubator-gluten]

2024-03-12 Thread via GitHub
Yohahaha commented on code in PR #4923: URL: https://github.com/apache/incubator-gluten/pull/4923#discussion_r1521095512 ## gluten-ut/spark34/src/test/scala/io/glutenproject/utils/velox/VeloxTestSettings.scala: ## @@ -130,8 +130,6 @@ class VeloxTestSettings extends

Re: [PR] [DNM][VL] check ci apache jenkins [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #4936: URL: https://github.com/apache/incubator-gluten/pull/4936#issuecomment-1991138405 Thanks for opening a pull request! Could you open an issue for this pull request on Github Issues?

Re: [PR] [DNM][VL] check ci apache jenkins [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #4936: URL: https://github.com/apache/incubator-gluten/pull/4936#issuecomment-1991138843 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL] parquet file metadata columns support in velox [incubator-gluten]

2024-03-12 Thread via GitHub
gaoyangxiaozhu commented on code in PR #3870: URL: https://github.com/apache/incubator-gluten/pull/3870#discussion_r1521209340 ## gluten-core/src/main/scala/io/glutenproject/execution/BasicScanExecTransformer.scala: ## @@ -37,12 +37,15 @@ import com.google.protobuf.StringValue

Re: [PR] [VL] parquet file metadata columns support in velox [incubator-gluten]

2024-03-12 Thread via GitHub
gaoyangxiaozhu commented on code in PR #3870: URL: https://github.com/apache/incubator-gluten/pull/3870#discussion_r1521217320 ## cpp/velox/substrait/SubstraitParser.cc: ## @@ -104,31 +104,41 @@ std::vector SubstraitParser::parseNamedStruct(const ::substrait::NamedS return

Re: [PR] [VL] Fix wrong plan equality due to case class inheritance [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #4893: URL: https://github.com/apache/incubator-gluten/pull/4893#issuecomment-1991341063 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-3283][VL] Upgrade arrow version to 14.0.1 and add compile arrow java module. [incubator-gluten]

2024-03-12 Thread via GitHub
fyp711 commented on PR #4206: URL: https://github.com/apache/incubator-gluten/pull/4206#issuecomment-1991381290 > I'm not quite sure what this compilation want to do.Is this compilation intended to output gluten-velox-bundle-spark3.2_2.12-* -1.1.0. jar ? Yes, If you run into this

Re: [I] Bump Iceberg version [incubator-gluten]

2024-03-12 Thread via GitHub
SteNicholas commented on issue #4933: URL: https://github.com/apache/incubator-gluten/issues/4933#issuecomment-1990958086 cc @liujiayi771. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[PR] [VL] Refine log plan/split json into one line [incubator-gluten]

2024-03-12 Thread via GitHub
Yohahaha opened a new pull request, #4934: URL: https://github.com/apache/incubator-gluten/pull/4934 ## What changes were proposed in this pull request? Refine log plan/split json into one line, it helps filter json output quickly when multi task running. Before ```

[PR] edit jekyll setting [incubator-gluten-site]

2024-03-12 Thread via GitHub
weiting-chen opened a new pull request, #4: URL: https://github.com/apache/incubator-gluten-site/pull/4 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

(incubator-gluten-site) branch main updated: edit jekyll setting (#4)

2024-03-12 Thread weitingchen
This is an automated email from the ASF dual-hosted git repository. weitingchen pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/incubator-gluten-site.git The following commit(s) were added to refs/heads/main by this push: new 2298d30 edit jekyll setting

(incubator-gluten-site) branch asf-site created (now b0a0b88)

2024-03-12 Thread git-site-role
This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a change to branch asf-site in repository https://gitbox.apache.org/repos/asf/incubator-gluten-site.git at b0a0b88 Automatic Site Publish by Buildbot This branch includes the following new commits:

Re: [PR] edit jekyll setting [incubator-gluten-site]

2024-03-12 Thread via GitHub
weiting-chen merged PR #4: URL: https://github.com/apache/incubator-gluten-site/pull/4 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Error while running github feature from .asf.yaml in incubator-gluten-site!

2024-03-12 Thread Apache Infrastructure
An error occurred while running github feature in .asf.yaml!: .asf.yaml: Invalid GitHub Pages branch ''asf-site'' - must be default branch or gh-pages! - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For

Re: [I] Bump Iceberg version [incubator-gluten]

2024-03-12 Thread via GitHub
liujiayi771 commented on issue #4933: URL: https://github.com/apache/incubator-gluten/issues/4933#issuecomment-1990993157 The current implementation has compatibility issues with the latest version of Iceberg that need to be resolved. -- This is an automated message from the Apache Git

Re: [PR] [GLUTEN-4903][CELEBORN][WIP] Support multiple versions of Celeborn [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #4913: URL: https://github.com/apache/incubator-gluten/pull/4913#issuecomment-1990993198 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL] parquet file metadata columns support in velox [incubator-gluten]

2024-03-12 Thread via GitHub
gaoyangxiaozhu commented on PR #3870: URL: https://github.com/apache/incubator-gluten/pull/3870#issuecomment-1991024316 @zhli1142015 and @zhouyuan could you help review ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] [CH][Just Test]240305 support delta ctas [incubator-gluten]

2024-03-12 Thread via GitHub
binmahone closed pull request #4851: [CH][Just Test]240305 support delta ctas URL: https://github.com/apache/incubator-gluten/pull/4851 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] [VL] parquet file metadata columns support in velox [incubator-gluten]

2024-03-12 Thread via GitHub
zhouyuan commented on PR #3870: URL: https://github.com/apache/incubator-gluten/pull/3870#issuecomment-1991116962 Quick note: This patch adds the native impl for parquet metadata read support which is a requirement from delta lake connector. The general idea is to add a metadata list

Re: [PR] [VL] parquet file metadata columns support in velox [incubator-gluten]

2024-03-12 Thread via GitHub
yma11 commented on code in PR #3870: URL: https://github.com/apache/incubator-gluten/pull/3870#discussion_r1521135199 ## shims/spark34/src/main/scala/io/glutenproject/sql/shims/spark34/Spark34Shims.scala: ## @@ -153,6 +157,34 @@ class Spark34Shims extends SparkShims { }

Re: [PR] [DNM][VL] check ci apache jenkins [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #4936: URL: https://github.com/apache/incubator-gluten/pull/4936#issuecomment-1991180231 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [Gluten-4706] [CH][CORE] Add a mode to execute count distinct directly instead o… [incubator-gluten]

2024-03-12 Thread via GitHub
GlutenPerfBot commented on PR #4708: URL: https://github.com/apache/incubator-gluten/pull/4708#issuecomment-1991218764 = Performance report for TPCH SF2000 with Velox backend, for reference only query

Re: [PR] [VL] parquet file metadata columns support in velox [incubator-gluten]

2024-03-12 Thread via GitHub
gaoyangxiaozhu commented on code in PR #3870: URL: https://github.com/apache/incubator-gluten/pull/3870#discussion_r1521217320 ## cpp/velox/substrait/SubstraitParser.cc: ## @@ -104,31 +104,41 @@ std::vector SubstraitParser::parseNamedStruct(const ::substrait::NamedS return

Re: [PR] [GLUTEN-4903][CELEBORN][WIP] Support multiple versions of Celeborn [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #4913: URL: https://github.com/apache/incubator-gluten/pull/4913#issuecomment-1991450804 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL] parquet file metadata columns support in velox [incubator-gluten]

2024-03-12 Thread via GitHub
yma11 commented on code in PR #3870: URL: https://github.com/apache/incubator-gluten/pull/3870#discussion_r1521381033 ## cpp/velox/substrait/SubstraitParser.cc: ## @@ -104,31 +104,41 @@ std::vector SubstraitParser::parseNamedStruct(const ::substrait::NamedS return typeList;

Re: [PR] [DOC] Minor fix for wrong gluten folder used in doc [incubator-gluten]

2024-03-12 Thread via GitHub
zhouyuan merged PR #4938: URL: https://github.com/apache/incubator-gluten/pull/4938 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

(incubator-gluten) branch main updated: [DOC] Minor fix for wrong gluten folder used in doc (#4938)

2024-03-12 Thread yuanzhou
This is an automated email from the ASF dual-hosted git repository. yuanzhou pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git The following commit(s) were added to refs/heads/main by this push: new 7481c8c78 [DOC] Minor fix for wrong

Re: [PR] [VL] Fix wrong plan equality due to case class inheritance [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #4893: URL: https://github.com/apache/incubator-gluten/pull/4893#issuecomment-199148 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL] Support read iceberg mor table for Velox backend [incubator-gluten]

2024-03-12 Thread via GitHub
yma11 commented on code in PR #4779: URL: https://github.com/apache/incubator-gluten/pull/4779#discussion_r1521505386 ## cpp/velox/compute/WholeStageResultIterator.cc: ## @@ -152,8 +152,28 @@ WholeStageResultIterator::WholeStageResultIterator( auto partitionColumn =

  1   2   >