Re: [PR] [VL] Add uniffle integration [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #3767: URL: https://github.com/apache/incubator-gluten/pull/3767#issuecomment-1993626916 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL] Support skewness aggregate function [incubator-gluten]

2024-03-12 Thread via GitHub
PHILO-HE commented on code in PR #4939: URL: https://github.com/apache/incubator-gluten/pull/4939#discussion_r1522564366 ## backends-velox/src/main/scala/io/glutenproject/execution/HashAggregateExecTransformer.scala: ## @@ -386,23 +386,33 @@ abstract class

Re: [PR] [VL] Support skewness aggregate function [incubator-gluten]

2024-03-12 Thread via GitHub
liujiayi771 commented on code in PR #4939: URL: https://github.com/apache/incubator-gluten/pull/4939#discussion_r1522561050 ## backends-velox/src/main/scala/io/glutenproject/execution/HashAggregateExecTransformer.scala: ## @@ -386,23 +386,33 @@ abstract class

Re: [PR] [VL] Support skewness aggregate function [incubator-gluten]

2024-03-12 Thread via GitHub
liujiayi771 commented on code in PR #4939: URL: https://github.com/apache/incubator-gluten/pull/4939#discussion_r1522561050 ## backends-velox/src/main/scala/io/glutenproject/execution/HashAggregateExecTransformer.scala: ## @@ -386,23 +386,33 @@ abstract class

Re: [PR] [VL] Add uniffle integration [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #3767: URL: https://github.com/apache/incubator-gluten/pull/3767#issuecomment-1993614839 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-4796][VL] Force fallback for orc char type scan [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #4797: URL: https://github.com/apache/incubator-gluten/pull/4797#issuecomment-1993576318 https://github.com/apache/incubator-gluten/issues/4796 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] [VL] Support posexplode function and code refactoring on GenerateExecTransformer [incubator-gluten]

2024-03-12 Thread via GitHub
zhouyuan merged PR #4901: URL: https://github.com/apache/incubator-gluten/pull/4901 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

(incubator-gluten) branch main updated: [VL] Support posexplode function and code refactoring on GenerateExecTransformer (#4901)

2024-03-12 Thread yuanzhou
This is an automated email from the ASF dual-hosted git repository. yuanzhou pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git The following commit(s) were added to refs/heads/main by this push: new b1fe7a33d [VL] Support posexplode

Re: [PR] [VL] Daily Update Velox Version (2024_03_13) [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #4944: URL: https://github.com/apache/incubator-gluten/pull/4944#issuecomment-1993451412 Thanks for opening a pull request! Could you open an issue for this pull request on Github Issues?

[PR] [VL] Daily Update Velox Version (2024_03_13) [incubator-gluten]

2024-03-12 Thread via GitHub
GlutenPerfBot opened a new pull request, #4944: URL: https://github.com/apache/incubator-gluten/pull/4944 Velox Upstream New Commits: ```txt 85f39732b by hengjiang.ly, Add prefix-sort with support for fixed width sorting keys (8146) de54d1e18 by Ma, Rong, Allow binding fixed

Re: [PR] [VL] Support skewness aggregate function [incubator-gluten]

2024-03-12 Thread via GitHub
PHILO-HE commented on code in PR #4939: URL: https://github.com/apache/incubator-gluten/pull/4939#discussion_r1522442068 ## backends-clickhouse/src/main/scala/io/glutenproject/utils/CHExpressionUtil.scala: ## @@ -174,6 +174,7 @@ object CHExpressionUtil { ENCODE ->

Re: [PR] [VL] Refine log plan/split json into one line [incubator-gluten]

2024-03-12 Thread via GitHub
GlutenPerfBot commented on PR #4934: URL: https://github.com/apache/incubator-gluten/pull/4934#issuecomment-1993401479 = Performance report for TPCH SF2000 with Velox backend, for reference only query

Re: [PR] [VL] Add uniffle integration [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #3767: URL: https://github.com/apache/incubator-gluten/pull/3767#issuecomment-1993383531 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [I] [CH] New byte buffer takes most of time in SourceFromJavalter::generate [incubator-gluten]

2024-03-12 Thread via GitHub
zhanglistar commented on issue #4943: URL: https://github.com/apache/incubator-gluten/issues/4943#issuecomment-1993376883 optoruntime::new_array_c可能是传入的`memory.m_capacity`过大,另外jdk中会对内存进行memset,导致该函数占用过多的时间。 -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] [VL] Fix wrong plan equality due to case class inheritance [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #4893: URL: https://github.com/apache/incubator-gluten/pull/4893#issuecomment-1993349772 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[I] [CH] New byte buffer takes most of time in SourceFromJavalter::generate [incubator-gluten]

2024-03-12 Thread via GitHub
taiyang-li opened a new issue, #4943: URL: https://github.com/apache/incubator-gluten/issues/4943 ### Description ![d722f3fabeb6881fe8b49f58cf0eb6c](https://github.com/apache/incubator-gluten/assets/8181003/8244ef97-fd00-4838-a341-adcb669847ec) ``` bool

Re: [PR] [GLUTEN-4827][UT] Add Golden Files for TPC-H Spark34 + Gluten Execution Plan [incubator-gluten]

2024-03-12 Thread via GitHub
zwangsheng commented on PR #4828: URL: https://github.com/apache/incubator-gluten/pull/4828#issuecomment-1993329577 Thanks for both @ulysses-you @PHILO-HE, i will revert this commit to focus on Spark 34 Golden Files, after some test, will turn this PR ready. -- This is an automated

Re: [PR] [GLUTEN-4827][UT] Add Golden Files for TPC-H Spark34 + Gluten Execution Plan [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #4828: URL: https://github.com/apache/incubator-gluten/pull/4828#issuecomment-1993323947 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-4827][UT] Add Golden Files for TPC-H Spark34 + Gluten Execution Plan [incubator-gluten]

2024-03-12 Thread via GitHub
ulysses-you commented on PR #4828: URL: https://github.com/apache/incubator-gluten/pull/4828#issuecomment-1993313046 I'm fine to seperate pr. @PHILO-HE IIUC the current action should work, just some code cleanup leave to another pr. -- This is an automated message from the Apache

Re: [I] [VL] Results are mismatch with vanilla Spark, it could be get_json_object() causing the issue. [incubator-gluten]

2024-03-12 Thread via GitHub
kecookier commented on issue #4928: URL: https://github.com/apache/incubator-gluten/issues/4928#issuecomment-1993302558 The following unit test case can reproduce the issue. I'm sure that got wrong value while parsing double in function `SIMDGetJsonObjectFunction::extractStringResult()`.

Re: [PR] [GLUTEN-4827][UT] Add Golden Files for TPC-H Spark34 + Gluten Execution Plan [incubator-gluten]

2024-03-12 Thread via GitHub
PHILO-HE commented on PR #4828: URL: https://github.com/apache/incubator-gluten/pull/4828#issuecomment-1993302181 > > @zwangsheng is there any block on this pr ? > > I'm still testing merge upload step. But IMO, we can leave merge step job in the following PR. WDYT @ulysses-you

Re: [PR] [VL] Add uniffle integration [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #3767: URL: https://github.com/apache/incubator-gluten/pull/3767#issuecomment-1993290413 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

(incubator-gluten) branch main updated: [VL] Refine log plan/split json into one line

2024-03-12 Thread yangzy
This is an automated email from the ASF dual-hosted git repository. yangzy pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git The following commit(s) were added to refs/heads/main by this push: new fd8ff2dc4 [VL] Refine log plan/split json

Re: [PR] [VL] Refine log plan/split json into one line [incubator-gluten]

2024-03-12 Thread via GitHub
Yohahaha merged PR #4934: URL: https://github.com/apache/incubator-gluten/pull/4934 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] [VL] Add uniffle integration [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #3767: URL: https://github.com/apache/incubator-gluten/pull/3767#issuecomment-1993257271 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL] Refine log plan/split json into one line [incubator-gluten]

2024-03-12 Thread via GitHub
marin-ma commented on PR #4934: URL: https://github.com/apache/incubator-gluten/pull/4934#issuecomment-1993236382 Thanks for the cleanup! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] [GLUTEN-4827][UT] Add Golden Files for TPC-H Spark34 + Gluten Execution Plan [incubator-gluten]

2024-03-12 Thread via GitHub
zwangsheng commented on PR #4828: URL: https://github.com/apache/incubator-gluten/pull/4828#issuecomment-1993232921 > @zwangsheng is there any block on this pr ? I'm still testing merge upload step. But IMO, we can leave merge step job in the following PR. WDYT @ulysses-you

Re: [PR] [VL] parquet file metadata columns support in velox [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #3870: URL: https://github.com/apache/incubator-gluten/pull/3870#issuecomment-1993211907 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-4827][UT] Add Golden Files for TPC-H Spark34 + Gluten Execution Plan [incubator-gluten]

2024-03-12 Thread via GitHub
ulysses-you commented on PR #4828: URL: https://github.com/apache/incubator-gluten/pull/4828#issuecomment-1993175055 @zwangsheng is there any block on this pr ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] [VL] Add uniffle integration [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #3767: URL: https://github.com/apache/incubator-gluten/pull/3767#issuecomment-1993139416 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL] Support read iceberg mor table for Velox backend [incubator-gluten]

2024-03-12 Thread via GitHub
liujiayi771 commented on PR #4779: URL: https://github.com/apache/incubator-gluten/pull/4779#issuecomment-1993089055 @yma11 I have modified the map in `SplitInfo` to a two-dimensional vector. -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] [VL] Support read iceberg mor table for Velox backend [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #4779: URL: https://github.com/apache/incubator-gluten/pull/4779#issuecomment-1993087336 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-3582] Support PageIndex [incubator-gluten]

2024-03-12 Thread via GitHub
lgbo-ustc commented on code in PR #4634: URL: https://github.com/apache/incubator-gluten/pull/4634#discussion_r1522363619 ## cpp-ch/local-engine/Storages/Parquet/VectorizedParquetRecordReader.cpp: ## @@ -0,0 +1,523 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] [VL] Refine log plan/split json into one line [incubator-gluten]

2024-03-12 Thread via GitHub
Yohahaha commented on PR #4934: URL: https://github.com/apache/incubator-gluten/pull/4934#issuecomment-1993067646 @marin-ma @PHILO-HE please help review this minor patch, thank you! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] [GLUTEN-3582] Support PageIndex [incubator-gluten]

2024-03-12 Thread via GitHub
lgbo-ustc commented on code in PR #4634: URL: https://github.com/apache/incubator-gluten/pull/4634#discussion_r1522361645 ## cpp-ch/local-engine/Storages/Parquet/RowRanges.h: ## @@ -0,0 +1,208 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + *

Re: [PR] [GLUTEN-3582] Support PageIndex [incubator-gluten]

2024-03-12 Thread via GitHub
lgbo-ustc commented on code in PR #4634: URL: https://github.com/apache/incubator-gluten/pull/4634#discussion_r1522353767 ## cpp-ch/local-engine/Storages/Parquet/ColumnIndexFilter.cpp: ## @@ -0,0 +1,982 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or

[I] [VL] remove dynamic packaging and use static packaing [incubator-gluten]

2024-03-12 Thread via GitHub
zhouyuan opened a new issue, #4942: URL: https://github.com/apache/incubator-gluten/issues/4942 ### Description The dynamic packaging will make a jar with all necessary dependencies for shared libs. On gluten start it will try to extract the libs and then load into JVM. This

Re: [PR] [GLUTEN-3582] Support PageIndex [incubator-gluten]

2024-03-12 Thread via GitHub
lgbo-ustc commented on code in PR #4634: URL: https://github.com/apache/incubator-gluten/pull/4634#discussion_r1522350915 ## cpp-ch/local-engine/Storages/Parquet/ColumnIndexFilter.h: ## @@ -0,0 +1,207 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more

Re: [PR] [VL] Support skewness aggregate function [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #4939: URL: https://github.com/apache/incubator-gluten/pull/4939#issuecomment-1992951474 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL] Fix wrong plan equality due to case class inheritance [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #4893: URL: https://github.com/apache/incubator-gluten/pull/4893#issuecomment-1992930874 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-4903][CELEBORN][WIP] Support multiple versions of Celeborn [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #4913: URL: https://github.com/apache/incubator-gluten/pull/4913#issuecomment-1992916044 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL] Support read iceberg mor table for Velox backend [incubator-gluten]

2024-03-12 Thread via GitHub
yma11 commented on PR #4779: URL: https://github.com/apache/incubator-gluten/pull/4779#issuecomment-1992901840 @liujiayi771 Seems code has scala style violations. Please update. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] [VL] Support read iceberg mor table for Velox backend [incubator-gluten]

2024-03-12 Thread via GitHub
yma11 commented on code in PR #4779: URL: https://github.com/apache/incubator-gluten/pull/4779#discussion_r1522324972 ## cpp/velox/compute/WholeStageResultIterator.cc: ## @@ -152,8 +152,28 @@ WholeStageResultIterator::WholeStageResultIterator( auto partitionColumn =

Re: [I] Besides Centos and Ubuntu systems, does the gluten project support the RedHat 7 system? [incubator-gluten]

2024-03-12 Thread via GitHub
PHILO-HE commented on issue #4935: URL: https://github.com/apache/incubator-gluten/issues/4935#issuecomment-1992859638 Assume it has been fixed by https://github.com/apache/incubator-gluten/pull/4206, which requires re-building gluten with `--compile_arrow_java=ON` -- This is an

Re: [PR] [DOC] Minor fix for wrong gluten folder used in doc [incubator-gluten]

2024-03-12 Thread via GitHub
GlutenPerfBot commented on PR #4938: URL: https://github.com/apache/incubator-gluten/pull/4938#issuecomment-1992795881 = Performance report for TPCH SF2000 with Velox backend, for reference only query

Re: [PR] [VL] Fix wrong plan equality due to case class inheritance [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #4893: URL: https://github.com/apache/incubator-gluten/pull/4893#issuecomment-1992771964 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL] Support skewness aggregate function [incubator-gluten]

2024-03-12 Thread via GitHub
liujiayi771 commented on PR #4939: URL: https://github.com/apache/incubator-gluten/pull/4939#issuecomment-1992743075 cc @rui-mo @PHILO-HE, thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] [GLUTEN-1632][CH]Daily Update Clickhouse Version (20240313) [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #4941: URL: https://github.com/apache/incubator-gluten/pull/4941#issuecomment-1992716396 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-1632][CH]Daily Update Clickhouse Version (20240313) [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #4941: URL: https://github.com/apache/incubator-gluten/pull/4941#issuecomment-1992716177 https://github.com/apache/incubator-gluten/issues/1632 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[PR] [GLUTEN-1632][CH]Daily Update Clickhouse Version (20240313) [incubator-gluten]

2024-03-12 Thread via GitHub
lwz9103 opened a new pull request, #4941: URL: https://github.com/apache/incubator-gluten/pull/4941 Auto commit by gluten daily build, please check the build status and merge it if it's green. -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [I] master --yarn on EMR with OSS Spark fails with java.lang.ClassNotFoundException: org.apache.spark.shuffle.sort.ColumnarShuffleManager [incubator-gluten]

2024-03-12 Thread via GitHub
sagarlakshmipathy commented on issue #4924: URL: https://github.com/apache/incubator-gluten/issues/4924#issuecomment-1992591955 moved the gluten jar to `SPARK_HOME/jars` folder and got it to work ``` ./spark-3.4.1-bin-hadoop3/bin/spark-shell \ --master yarn     \

Re: [I] master --yarn on EMR with OSS Spark fails with java.lang.ClassNotFoundException: org.apache.spark.shuffle.sort.ColumnarShuffleManager [incubator-gluten]

2024-03-12 Thread via GitHub
sagarlakshmipathy closed issue #4924: master --yarn on EMR with OSS Spark fails with java.lang.ClassNotFoundException: org.apache.spark.shuffle.sort.ColumnarShuffleManager URL: https://github.com/apache/incubator-gluten/issues/4924 -- This is an automated message from the Apache Git

Re: [PR] [VL]Bucket join support for Iceberg tables [incubator-gluten]

2024-03-12 Thread via GitHub
SinghAsDev commented on code in PR #4859: URL: https://github.com/apache/incubator-gluten/pull/4859#discussion_r1521849064 ## gluten-iceberg/src/test/scala/io/glutenproject/execution/VeloxIcebergSuite.scala: ## @@ -56,44 +71,246 @@ class VeloxIcebergSuite extends

Re: [PR] [VL] parquet file metadata columns support in velox [incubator-gluten]

2024-03-12 Thread via GitHub
gaoyangxiaozhu commented on PR #3870: URL: https://github.com/apache/incubator-gluten/pull/3870#issuecomment-1992105649 various build issue , can you help re-trigger @yma11 / @zhouyuan / @zhli1142015

Re: [PR] [VL] parquet file metadata columns support in velox [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #3870: URL: https://github.com/apache/incubator-gluten/pull/3870#issuecomment-1992018595 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-3559][VL] enable more sql query tests for Spark34 [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #4880: URL: https://github.com/apache/incubator-gluten/pull/4880#issuecomment-1991893449 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-4903][CELEBORN][WIP] Support multiple versions of Celeborn [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #4913: URL: https://github.com/apache/incubator-gluten/pull/4913#issuecomment-1991871011 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL] Support read iceberg mor table for Velox backend [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #4779: URL: https://github.com/apache/incubator-gluten/pull/4779#issuecomment-1991857238 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL] Support read iceberg mor table for Velox backend [incubator-gluten]

2024-03-12 Thread via GitHub
liujiayi771 commented on code in PR #4779: URL: https://github.com/apache/incubator-gluten/pull/4779#discussion_r1521626189 ## cpp/velox/compute/WholeStageResultIterator.cc: ## @@ -152,8 +152,28 @@ WholeStageResultIterator::WholeStageResultIterator( auto partitionColumn

Re: [PR] [GLUTEN-4903][CELEBORN][WIP] Support multiple versions of Celeborn [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #4913: URL: https://github.com/apache/incubator-gluten/pull/4913#issuecomment-1991772237 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL] Support skewness aggregate function [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #4939: URL: https://github.com/apache/incubator-gluten/pull/4939#issuecomment-1991728807 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL] Support skewness aggregate function [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #4939: URL: https://github.com/apache/incubator-gluten/pull/4939#issuecomment-1991728318 Thanks for opening a pull request! Could you open an issue for this pull request on Github Issues?

[PR] [VL] Support skewness aggregate function [incubator-gluten]

2024-03-12 Thread via GitHub
liujiayi771 opened a new pull request, #4939: URL: https://github.com/apache/incubator-gluten/pull/4939 ## What changes were proposed in this pull request? Support skewness aggregate function for Velox backend. ## How was this patch tested? Add skewness test case.

Re: [PR] [VL] Support read iceberg mor table for Velox backend [incubator-gluten]

2024-03-12 Thread via GitHub
yma11 commented on code in PR #4779: URL: https://github.com/apache/incubator-gluten/pull/4779#discussion_r1521505386 ## cpp/velox/compute/WholeStageResultIterator.cc: ## @@ -152,8 +152,28 @@ WholeStageResultIterator::WholeStageResultIterator( auto partitionColumn =

Re: [PR] [GLUTEN-3559][VL] enable more sql query tests for Spark34 [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #4880: URL: https://github.com/apache/incubator-gluten/pull/4880#issuecomment-1991667434 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL] Fix wrong plan equality due to case class inheritance [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #4893: URL: https://github.com/apache/incubator-gluten/pull/4893#issuecomment-199148 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-4903][CELEBORN][WIP] Support multiple versions of Celeborn [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #4913: URL: https://github.com/apache/incubator-gluten/pull/4913#issuecomment-1991586394 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

(incubator-gluten) branch main updated: [DOC] Minor fix for wrong gluten folder used in doc (#4938)

2024-03-12 Thread yuanzhou
This is an automated email from the ASF dual-hosted git repository. yuanzhou pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git The following commit(s) were added to refs/heads/main by this push: new 7481c8c78 [DOC] Minor fix for wrong

Re: [PR] [DOC] Minor fix for wrong gluten folder used in doc [incubator-gluten]

2024-03-12 Thread via GitHub
zhouyuan merged PR #4938: URL: https://github.com/apache/incubator-gluten/pull/4938 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] [VL] parquet file metadata columns support in velox [incubator-gluten]

2024-03-12 Thread via GitHub
yma11 commented on code in PR #3870: URL: https://github.com/apache/incubator-gluten/pull/3870#discussion_r1521381033 ## cpp/velox/substrait/SubstraitParser.cc: ## @@ -104,31 +104,41 @@ std::vector SubstraitParser::parseNamedStruct(const ::substrait::NamedS return typeList;

Re: [PR] [DOC] Minor fix for docker_centos7.md [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #4938: URL: https://github.com/apache/incubator-gluten/pull/4938#issuecomment-1991534488 Thanks for opening a pull request! Could you open an issue for this pull request on Github Issues?

Re: [PR] [VL] Verify unhex has been offloaded to native successfully [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #4937: URL: https://github.com/apache/incubator-gluten/pull/4937#issuecomment-1991500082 Thanks for opening a pull request! Could you open an issue for this pull request on Github Issues?

[PR] [VL] Verify unhex has been offloaded to native successfully [incubator-gluten]

2024-03-12 Thread via GitHub
Yohahaha opened a new pull request, #4937: URL: https://github.com/apache/incubator-gluten/pull/4937 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] [VL] parquet file metadata columns support in velox [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #3870: URL: https://github.com/apache/incubator-gluten/pull/3870#issuecomment-1991460438 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-4903][CELEBORN][WIP] Support multiple versions of Celeborn [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #4913: URL: https://github.com/apache/incubator-gluten/pull/4913#issuecomment-1991450804 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-3283][VL] Upgrade arrow version to 14.0.1 and add compile arrow java module. [incubator-gluten]

2024-03-12 Thread via GitHub
fyp711 commented on PR #4206: URL: https://github.com/apache/incubator-gluten/pull/4206#issuecomment-1991381290 > I'm not quite sure what this compilation want to do.Is this compilation intended to output gluten-velox-bundle-spark3.2_2.12-* -1.1.0. jar ? Yes, If you run into this

Re: [PR] [GLUTEN-4241][VL] Add plan node to convert Vanilla spark columnar format data to Velox columnar format data [incubator-gluten]

2024-03-12 Thread via GitHub
boneanxs commented on PR #4818: URL: https://github.com/apache/incubator-gluten/pull/4818#issuecomment-1991360394 Hey @FelixYBW `ArrowFieldWriter` calls arrow `ValueVector`, which is internally uses `ArrowBuf` to store values, so it should be offheap memory.

Re: [PR] [VL] Daily Update Velox Version (2024_03_12) [incubator-gluten]

2024-03-12 Thread via GitHub
GlutenPerfBot commented on PR #4923: URL: https://github.com/apache/incubator-gluten/pull/4923#issuecomment-1991348456 = Performance report for TPCH SF2000 with Velox backend, for reference only query

Re: [PR] [VL] Fix wrong plan equality due to case class inheritance [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #4893: URL: https://github.com/apache/incubator-gluten/pull/4893#issuecomment-1991341063 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL] parquet file metadata columns support in velox [incubator-gluten]

2024-03-12 Thread via GitHub
gaoyangxiaozhu commented on code in PR #3870: URL: https://github.com/apache/incubator-gluten/pull/3870#discussion_r1521217320 ## cpp/velox/substrait/SubstraitParser.cc: ## @@ -104,31 +104,41 @@ std::vector SubstraitParser::parseNamedStruct(const ::substrait::NamedS return

Re: [PR] [VL] parquet file metadata columns support in velox [incubator-gluten]

2024-03-12 Thread via GitHub
gaoyangxiaozhu commented on code in PR #3870: URL: https://github.com/apache/incubator-gluten/pull/3870#discussion_r1521217320 ## cpp/velox/substrait/SubstraitParser.cc: ## @@ -104,31 +104,41 @@ std::vector SubstraitParser::parseNamedStruct(const ::substrait::NamedS return

Re: [PR] [VL] parquet file metadata columns support in velox [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #3870: URL: https://github.com/apache/incubator-gluten/pull/3870#issuecomment-1991300996 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL] parquet file metadata columns support in velox [incubator-gluten]

2024-03-12 Thread via GitHub
gaoyangxiaozhu commented on code in PR #3870: URL: https://github.com/apache/incubator-gluten/pull/3870#discussion_r1521209340 ## gluten-core/src/main/scala/io/glutenproject/execution/BasicScanExecTransformer.scala: ## @@ -37,12 +37,15 @@ import com.google.protobuf.StringValue

Re: [I] [CH] unsupported function "unix_timestamp" [incubator-gluten]

2024-03-12 Thread via GitHub
liuneng1994 closed issue #4914: [CH] unsupported function "unix_timestamp" URL: https://github.com/apache/incubator-gluten/issues/4914 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

(incubator-gluten) branch main updated: [GLUTEN-4914][CH] Fix exceptions in ASTParser #4916

2024-03-12 Thread liuneng
This is an automated email from the ASF dual-hosted git repository. liuneng pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git The following commit(s) were added to refs/heads/main by this push: new 8b43ac728 [GLUTEN-4914][CH] Fix

Re: [PR] [GLUTEN-4914][CH] Fix exceptions in ASTParser [incubator-gluten]

2024-03-12 Thread via GitHub
liuneng1994 merged PR #4916: URL: https://github.com/apache/incubator-gluten/pull/4916 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] [VL] parquet file metadata columns support in velox [incubator-gluten]

2024-03-12 Thread via GitHub
yma11 commented on code in PR #3870: URL: https://github.com/apache/incubator-gluten/pull/3870#discussion_r1521186042 ## gluten-ut/spark33/src/test/scala/org/apache/spark/sql/execution/datasources/GlutenFileMetadataStructSuite.scala: ## @@ -16,6 +16,156 @@ */ package

Re: [PR] [VL] parquet file metadata columns support in velox [incubator-gluten]

2024-03-12 Thread via GitHub
yma11 commented on code in PR #3870: URL: https://github.com/apache/incubator-gluten/pull/3870#discussion_r1521185608 ## cpp/velox/substrait/SubstraitParser.cc: ## @@ -104,31 +104,41 @@ std::vector SubstraitParser::parseNamedStruct(const ::substrait::NamedS return typeList;

Re: [PR] [VL] parquet file metadata columns support in velox [incubator-gluten]

2024-03-12 Thread via GitHub
yma11 commented on code in PR #3870: URL: https://github.com/apache/incubator-gluten/pull/3870#discussion_r1521062380 ## cpp/velox/compute/WholeStageResultIterator.cc: ## @@ -145,15 +145,31 @@ WholeStageResultIterator::WholeStageResultIterator( const auto& lengths =

Re: [PR] [Gluten-4706] [CH][CORE] Add a mode to execute count distinct directly instead o… [incubator-gluten]

2024-03-12 Thread via GitHub
GlutenPerfBot commented on PR #4708: URL: https://github.com/apache/incubator-gluten/pull/4708#issuecomment-1991218764 = Performance report for TPCH SF2000 with Velox backend, for reference only query

Re: [PR] [DNM][VL] check ci apache jenkins [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #4936: URL: https://github.com/apache/incubator-gluten/pull/4936#issuecomment-1991180231 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL] parquet file metadata columns support in velox [incubator-gluten]

2024-03-12 Thread via GitHub
gaoyangxiaozhu commented on code in PR #3870: URL: https://github.com/apache/incubator-gluten/pull/3870#discussion_r1521142100 ## cpp/velox/substrait/SubstraitParser.cc: ## @@ -104,31 +104,41 @@ std::vector SubstraitParser::parseNamedStruct(const ::substrait::NamedS return

Re: [PR] [VL] parquet file metadata columns support in velox [incubator-gluten]

2024-03-12 Thread via GitHub
yma11 commented on code in PR #3870: URL: https://github.com/apache/incubator-gluten/pull/3870#discussion_r1521135199 ## shims/spark34/src/main/scala/io/glutenproject/sql/shims/spark34/Spark34Shims.scala: ## @@ -153,6 +157,34 @@ class Spark34Shims extends SparkShims { }

Re: [PR] [DNM][VL] check ci apache jenkins [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #4936: URL: https://github.com/apache/incubator-gluten/pull/4936#issuecomment-1991138843 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [DNM][VL] check ci apache jenkins [incubator-gluten]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #4936: URL: https://github.com/apache/incubator-gluten/pull/4936#issuecomment-1991138405 Thanks for opening a pull request! Could you open an issue for this pull request on Github Issues?

Re: [PR] [VL] parquet file metadata columns support in velox [incubator-gluten]

2024-03-12 Thread via GitHub
zhouyuan commented on PR #3870: URL: https://github.com/apache/incubator-gluten/pull/3870#issuecomment-1991116962 Quick note: This patch adds the native impl for parquet metadata read support which is a requirement from delta lake connector. The general idea is to add a metadata list

Re: [PR] [VL] parquet file metadata columns support in velox [incubator-gluten]

2024-03-12 Thread via GitHub
yma11 commented on code in PR #3870: URL: https://github.com/apache/incubator-gluten/pull/3870#discussion_r1521102914 ## gluten-core/src/main/scala/io/glutenproject/execution/BasicScanExecTransformer.scala: ## @@ -37,12 +37,15 @@ import com.google.protobuf.StringValue import

(incubator-gluten) branch main updated (b07b36960 -> 4ee2ddb1d)

2024-03-12 Thread philo
This is an automated email from the ASF dual-hosted git repository. philo pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git from b07b36960 [Gluten-4706] Add a mode to execute count distinct directly instead of Expand+Count (#4708)

Re: [PR] [VL] Daily Update Velox Version (2024_03_12) [incubator-gluten]

2024-03-12 Thread via GitHub
PHILO-HE merged PR #4923: URL: https://github.com/apache/incubator-gluten/pull/4923 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] [VL] parquet file metadata columns support in velox [incubator-gluten]

2024-03-12 Thread via GitHub
zhouyuan commented on code in PR #3870: URL: https://github.com/apache/incubator-gluten/pull/3870#discussion_r1521101767 ## gluten-ut/spark33/src/test/scala/org/apache/spark/sql/execution/datasources/GlutenFileMetadataStructSuite.scala: ## @@ -16,6 +16,156 @@ */ package

  1   2   >