Re: [PR] [GLUTEN-5547][VL] Add config to force fallback for parquet timestamp type scan [incubator-gluten]

2024-04-25 Thread via GitHub
github-actions[bot] commented on PR #5546: URL: https://github.com/apache/incubator-gluten/pull/5546#issuecomment-2078673582 https://github.com/apache/incubator-gluten/issues/5547 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] [CH] [Minor] fix ut due to https://github.com/ClickHouse/ClickHouse/pull/61216 [incubator-gluten]

2024-04-25 Thread via GitHub
ulysses-you commented on PR #5388: URL: https://github.com/apache/incubator-gluten/pull/5388#issuecomment-2078667174 cc @zhztheplayer @PHILO-HE @zhouyuan it seems the prs in 4.12 have regression, I'm not sure which commit cause it since we are doing benchmark each day rather than each

Re: [PR] [VL] Add config to force fallback for parquet timestamp type scan [incubator-gluten]

2024-04-25 Thread via GitHub
github-actions[bot] commented on PR #5546: URL: https://github.com/apache/incubator-gluten/pull/5546#issuecomment-2078663316 Thanks for opening a pull request! Could you open an issue for this pull request on Github Issues?

Re: [PR] [VL] Add config to force fallback for parquet timestamp type scan [incubator-gluten]

2024-04-25 Thread via GitHub
github-actions[bot] commented on PR #5546: URL: https://github.com/apache/incubator-gluten/pull/5546#issuecomment-2078663511 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] Make the hasNext method can be called multi times [incubator-gluten]

2024-04-25 Thread via GitHub
zhztheplayer commented on PR #5545: URL: https://github.com/apache/incubator-gluten/pull/5545#issuecomment-2078607024 Please don't forget adding a prefix to PR title and commit message. -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [PR] [GLUTEN-5476][CH] Triger merge on insert task [incubator-gluten]

2024-04-25 Thread via GitHub
github-actions[bot] commented on PR #5529: URL: https://github.com/apache/incubator-gluten/pull/5529#issuecomment-2078598221 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-5461] FEAT: ColumnarArrowPythonEvalExec support for Velox backend [incubator-gluten]

2024-04-25 Thread via GitHub
GlutenPerfBot commented on PR #5462: URL: https://github.com/apache/incubator-gluten/pull/5462#issuecomment-2078585388 = Performance report for TPCH SF2000 with Velox backend, for reference only query

Re: [PR] [VL] Fix case-class inheritance for VeloxColumnarWriteFilesExec [incubator-gluten]

2024-04-25 Thread via GitHub
ulysses-you commented on code in PR #5480: URL: https://github.com/apache/incubator-gluten/pull/5480#discussion_r1580445934 ## backends-velox/src/main/scala/org/apache/spark/sql/execution/VeloxColumnarWriteFilesExec.scala: ## @@ -308,12 +321,39 @@ class

Re: [PR] [CORE] Upgrade Arrow to 15.0.0 [incubator-gluten]

2024-04-25 Thread via GitHub
github-actions[bot] commented on PR #5174: URL: https://github.com/apache/incubator-gluten/pull/5174#issuecomment-2078573676 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL] Abfs FileSink Onboard [incubator-gluten]

2024-04-25 Thread via GitHub
gaoyangxiaozhu commented on PR #5527: URL: https://github.com/apache/incubator-gluten/pull/5527#issuecomment-2078556210 @JkSelf -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

(incubator-gluten) branch main updated: [GLUTEN-5461] FEAT: ColumnarArrowPythonEvalExec support for Velox backend (#5462)

2024-04-25 Thread yuanzhou
This is an automated email from the ASF dual-hosted git repository. yuanzhou pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git The following commit(s) were added to refs/heads/main by this push: new 44111b32c [GLUTEN-5461] FEAT:

Re: [PR] [GLUTEN-5461] FEAT: ColumnarArrowPythonEvalExec support for Velox backend [incubator-gluten]

2024-04-25 Thread via GitHub
zhouyuan merged PR #5462: URL: https://github.com/apache/incubator-gluten/pull/5462 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Make the hasNext method can be called multi times [incubator-gluten]

2024-04-25 Thread via GitHub
github-actions[bot] commented on PR #5545: URL: https://github.com/apache/incubator-gluten/pull/5545#issuecomment-2078552861 Thanks for opening a pull request! Could you open an issue for this pull request on Github Issues?

Re: [PR] Make the hasNext method can be called multi times [incubator-gluten]

2024-04-25 Thread via GitHub
JkSelf commented on PR #5545: URL: https://github.com/apache/incubator-gluten/pull/5545#issuecomment-2078552907 @FelixYBW @zhztheplayer Can you help to review? Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[PR] Make the hasNext method can be called multi times [incubator-gluten]

2024-04-25 Thread via GitHub
JkSelf opened a new pull request, #5545: URL: https://github.com/apache/incubator-gluten/pull/5545 ## What changes were proposed in this pull request? Currently, the iterator is designed to close once the hasNext() method returns false. Therefore, if `hasNext()` is called a second time,

Re: [PR] [GLUTEN-4652] Fix min_by/max_by result mismatch [incubator-gluten]

2024-04-25 Thread via GitHub
github-actions[bot] commented on PR #5544: URL: https://github.com/apache/incubator-gluten/pull/5544#issuecomment-2078548450 https://github.com/apache/incubator-gluten/issues/4652 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[PR] [GLUTEN-4652] Fix min_by/max_by result mismatch [incubator-gluten]

2024-04-25 Thread via GitHub
yma11 opened a new pull request, #5544: URL: https://github.com/apache/incubator-gluten/pull/5544 ## What changes were proposed in this pull request? Fix `min_by`/`max_by` result mismatch. Take `max_by` for example, we need to keep intermediate result row like `` which will be

Re: [PR] [GLUTEN-5532] Clean up some dead code in GlutenPlugin [incubator-gluten]

2024-04-25 Thread via GitHub
ivoson commented on PR #5533: URL: https://github.com/apache/incubator-gluten/pull/5533#issuecomment-2078545389 Thanks for the review @rui-mo -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] fix shuffle OOM if input batch is extremely large [incubator-gluten]

2024-04-25 Thread via GitHub
guhaiyan0221 commented on code in PR #5536: URL: https://github.com/apache/incubator-gluten/pull/5536#discussion_r1580421897 ## cpp/velox/shuffle/VeloxShuffleWriter.cc: ## @@ -892,6 +909,8 @@ uint32_t VeloxShuffleWriter::calculatePartitionBufferSize(const facebook::velox:

Re: [I] [VL] Unsupported spark function list [please leave a comment if you plan to pick some] [incubator-gluten]

2024-04-25 Thread via GitHub
zhli1142015 commented on issue #4039: URL: https://github.com/apache/incubator-gluten/issues/4039#issuecomment-2078523538 I'd like to take `soundex` and `levenshtein`, thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] [CORE] Upgrade Arrow to 15.0.0 [incubator-gluten]

2024-04-25 Thread via GitHub
github-actions[bot] commented on PR #5174: URL: https://github.com/apache/incubator-gluten/pull/5174#issuecomment-2078523983 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL] Improve checkNativeWrite in VeloxParquetWriteForHiveSuite [incubator-gluten]

2024-04-25 Thread via GitHub
ulysses-you commented on code in PR #5496: URL: https://github.com/apache/incubator-gluten/pull/5496#discussion_r1580346752 ## backends-velox/src/test/scala/org/apache/spark/sql/execution/VeloxParquetWriteForHiveSuite.scala: ## @@ -130,60 +122,30 @@ class

Re: [PR] [GLUTEN-4424] Explore adding Spark 35 w/ Scala 2.12 only (WIP) [incubator-gluten]

2024-04-25 Thread via GitHub
github-actions[bot] commented on PR #4425: URL: https://github.com/apache/incubator-gluten/pull/4425#issuecomment-2078482059 This PR was auto-closed because it has been stalled for 10 days with no activity. Please feel free to reopen if it is still valid. Thanks. -- This is an automated

Re: [PR] [GLUTEN-4424] Explore adding Spark 35 w/ Scala 2.12 only (WIP) [incubator-gluten]

2024-04-25 Thread via GitHub
github-actions[bot] closed pull request #4425: [GLUTEN-4424] Explore adding Spark 35 w/ Scala 2.12 only (WIP) URL: https://github.com/apache/incubator-gluten/pull/4425 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] fix shuffle OOM if input batch is extremely large [incubator-gluten]

2024-04-25 Thread via GitHub
marin-ma commented on code in PR #5536: URL: https://github.com/apache/incubator-gluten/pull/5536#discussion_r1580321600 ## cpp/velox/shuffle/VeloxShuffleWriter.cc: ## @@ -340,25 +340,42 @@ arrow::Status VeloxShuffleWriter::split(std::shared_ptr cb, int64

Re: [PR] fix shuffle OOM if input batch is extremely large [incubator-gluten]

2024-04-25 Thread via GitHub
marin-ma commented on code in PR #5536: URL: https://github.com/apache/incubator-gluten/pull/5536#discussion_r1580320425 ## cpp/velox/shuffle/VeloxShuffleWriter.cc: ## @@ -892,6 +909,8 @@ uint32_t VeloxShuffleWriter::calculatePartitionBufferSize(const facebook::velox:

Re: [PR] fix shuffle OOM if input batch is extremely large [incubator-gluten]

2024-04-25 Thread via GitHub
marin-ma commented on code in PR #5536: URL: https://github.com/apache/incubator-gluten/pull/5536#discussion_r1580317008 ## cpp/velox/shuffle/VeloxShuffleWriter.cc: ## @@ -892,6 +909,8 @@ uint32_t VeloxShuffleWriter::calculatePartitionBufferSize(const facebook::velox:

Re: [PR] [VL] Daily Update Velox Version (2024_04_26) [incubator-gluten]

2024-04-25 Thread via GitHub
github-actions[bot] commented on PR #5543: URL: https://github.com/apache/incubator-gluten/pull/5543#issuecomment-2078426056 Thanks for opening a pull request! Could you open an issue for this pull request on Github Issues?

[PR] [VL] Daily Update Velox Version (2024_04_26) [incubator-gluten]

2024-04-25 Thread via GitHub
GlutenPerfBot opened a new pull request, #5543: URL: https://github.com/apache/incubator-gluten/pull/5543 Upstream Velox's New Commits: ```txt 1daeb9dc7 by Masha Basmanova, Optimize copying null bits in FlatVector::copyValuesAndNulls 6e2184f51 by Bikramjeet Vig, Back out "Back

Re: [PR] [GLUTEN-5532] Clean up some dead code in GlutenPlugin [incubator-gluten]

2024-04-25 Thread via GitHub
GlutenPerfBot commented on PR #5533: URL: https://github.com/apache/incubator-gluten/pull/5533#issuecomment-2078423353 = Performance report for TPCH SF2000 with Velox backend, for reference only query

Re: [PR] [VL] Daily Update Velox Version (2024_04_25) [incubator-gluten]

2024-04-25 Thread via GitHub
zhouyuan closed pull request #5542: [VL] Daily Update Velox Version (2024_04_25) URL: https://github.com/apache/incubator-gluten/pull/5542 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

(incubator-gluten) branch main updated: [VL] UDF: Support variable arity in function sigatures (#5495)

2024-04-25 Thread marong
This is an automated email from the ASF dual-hosted git repository. marong pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git The following commit(s) were added to refs/heads/main by this push: new 1e07bde47 [VL] UDF: Support variable

Re: [I] UDF and UDAF do not support Variable length parameter [incubator-gluten]

2024-04-25 Thread via GitHub
marin-ma closed issue #5197: UDF and UDAF do not support Variable length parameter URL: https://github.com/apache/incubator-gluten/issues/5197 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] [VL] UDF: Support variable arity in function sigatures [incubator-gluten]

2024-04-25 Thread via GitHub
marin-ma merged PR #5495: URL: https://github.com/apache/incubator-gluten/pull/5495 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

(incubator-gluten) branch main updated: remove duplicate content for local cache part (#5535)

2024-04-25 Thread felixybw
This is an automated email from the ASF dual-hosted git repository. felixybw pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git The following commit(s) were added to refs/heads/main by this push: new c19175331 remove duplicate content for

Re: [PR] [VL][Doc] Remove duplicate content for local cache part [incubator-gluten]

2024-04-25 Thread via GitHub
FelixYBW merged PR #5535: URL: https://github.com/apache/incubator-gluten/pull/5535 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [I] [VL] row_constructor in gluten will make top-level struct to null if its child fields have null value [incubator-gluten]

2024-04-25 Thread via GitHub
rui-mo commented on issue #1303: URL: https://github.com/apache/incubator-gluten/issues/1303#issuecomment-2078395329 @yma11 Could you help check which aggregate functions are still using `row_constructor_with_null`? If the corresponding logics on Velox side are customized well for Spark,

Re: [PR] [CORE] Upgrade Arrow to 15.0.0 [incubator-gluten]

2024-04-25 Thread via GitHub
github-actions[bot] commented on PR #5174: URL: https://github.com/apache/incubator-gluten/pull/5174#issuecomment-2078392726 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-5461] FEAT: ColumnarArrowPythonEvalExec support for Velox backend [incubator-gluten]

2024-04-25 Thread via GitHub
github-actions[bot] commented on PR #5462: URL: https://github.com/apache/incubator-gluten/pull/5462#issuecomment-2078385013 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

(incubator-gluten) branch main updated (8ade7a9cb -> 731c17ea4)

2024-04-25 Thread rui
This is an automated email from the ASF dual-hosted git repository. rui pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git from 8ade7a9cb [VL] Use slice instead of resize in ensureFlattened (#5523) add 731c17ea4 [GLUTEN-5532] Code

Re: [PR] [GLUTEN-5532] Clean up some dead code in GlutenPlugin [incubator-gluten]

2024-04-25 Thread via GitHub
rui-mo merged PR #5533: URL: https://github.com/apache/incubator-gluten/pull/5533 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] [GLUTEN-5532] Clean up some dead code in GlutenPlugin [incubator-gluten]

2024-04-25 Thread via GitHub
ivoson commented on PR #5533: URL: https://github.com/apache/incubator-gluten/pull/5533#issuecomment-2078339721 Hi @rui-mo , can you please take a look at this PR? Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] [VL] Daily Update Velox Version (2024_04_25) [incubator-gluten]

2024-04-25 Thread via GitHub
github-actions[bot] commented on PR #5542: URL: https://github.com/apache/incubator-gluten/pull/5542#issuecomment-2078332326 Thanks for opening a pull request! Could you open an issue for this pull request on Github Issues?

[PR] [VL] Daily Update Velox Version (2024_04_25) [incubator-gluten]

2024-04-25 Thread via GitHub
GlutenPerfBot opened a new pull request, #5542: URL: https://github.com/apache/incubator-gluten/pull/5542 Upstream Velox's New Commits: ```txt 6e2184f51 by Bikramjeet Vig, Back out "Back out "[velox][PR] Refactor greatest and least Presto functions using simple function API""

Re: [PR] [GLUTEN-1632][CH]Daily Update Clickhouse Version (20240426) [incubator-gluten]

2024-04-25 Thread via GitHub
github-actions[bot] commented on PR #5541: URL: https://github.com/apache/incubator-gluten/pull/5541#issuecomment-2078309055 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-1632][CH]Daily Update Clickhouse Version (20240426) [incubator-gluten]

2024-04-25 Thread via GitHub
github-actions[bot] commented on PR #5541: URL: https://github.com/apache/incubator-gluten/pull/5541#issuecomment-2078308454 https://github.com/apache/incubator-gluten/issues/1632 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[PR] [GLUTEN-1632][CH]Daily Update Clickhouse Version (20240426) [incubator-gluten]

2024-04-25 Thread via GitHub
kyligence-git opened a new pull request, #5541: URL: https://github.com/apache/incubator-gluten/pull/5541 Auto commit by gluten daily build, please check the build status and merge it if it's green. -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] [VL] Use slice instead of resize in ensureFlattened [incubator-gluten]

2024-04-25 Thread via GitHub
GlutenPerfBot commented on PR #5523: URL: https://github.com/apache/incubator-gluten/pull/5523#issuecomment-2078272604 = Performance report for TPCH SF2000 with Velox backend, for reference only query

Re: [PR] Extend adapter setup to support Debian distributions [incubator-gluten]

2024-04-25 Thread via GitHub
surnaik commented on PR #5540: URL: https://github.com/apache/incubator-gluten/pull/5540#issuecomment-2077897670 Hi @acvictor, nice to hear from you. Sure, please raise it in upstream velox. That seems like the right place. Thanks @Preetesh2110 for this change. -- This is an automated

Re: [PR] [VL] Enable array_size Spark function [incubator-gluten]

2024-04-25 Thread via GitHub
github-actions[bot] commented on PR #5539: URL: https://github.com/apache/incubator-gluten/pull/5539#issuecomment-2077892244 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] Extend adapter setup to support Debian distributions [incubator-gluten]

2024-04-25 Thread via GitHub
acvictor commented on PR #5540: URL: https://github.com/apache/incubator-gluten/pull/5540#issuecomment-2077869676 > @acvictor could you please take a look at it. Thanks @Preetesh2110 let's update this in Velox directly? It will become painful to maintain every time the upstream file

Re: [PR] [VL] Extend adapter setup to support Debian distributions [incubator-gluten]

2024-04-25 Thread via GitHub
Preetesh2110 closed pull request #5415: [VL] Extend adapter setup to support Debian distributions URL: https://github.com/apache/incubator-gluten/pull/5415 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] Extend adapter setup to support Debian distributions [incubator-gluten]

2024-04-25 Thread via GitHub
Preetesh2110 commented on PR #5540: URL: https://github.com/apache/incubator-gluten/pull/5540#issuecomment-2077853494 @acvictor could you please take a look at it. Thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] Extend adapter setup to support Debian distributions [incubator-gluten]

2024-04-25 Thread via GitHub
github-actions[bot] commented on PR #5540: URL: https://github.com/apache/incubator-gluten/pull/5540#issuecomment-2077852849 Thanks for opening a pull request! Could you open an issue for this pull request on Github Issues?

[PR] Extend adapter setup to support Debian distributions [incubator-gluten]

2024-04-25 Thread via GitHub
Preetesh2110 opened a new pull request, #5540: URL: https://github.com/apache/incubator-gluten/pull/5540 ## What changes were proposed in this pull request? Extend adapter setup to support Debian distributions This change allows the setup script to work on both Ubuntu and Debian,

Re: [PR] [VL] Extend adapter setup to support Debian distributions [incubator-gluten]

2024-04-25 Thread via GitHub
github-actions[bot] commented on PR #5415: URL: https://github.com/apache/incubator-gluten/pull/5415#issuecomment-2077842171 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

(incubator-gluten) branch main updated: [VL] Use slice instead of resize in ensureFlattened (#5523)

2024-04-25 Thread felixybw
This is an automated email from the ASF dual-hosted git repository. felixybw pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git The following commit(s) were added to refs/heads/main by this push: new 8ade7a9cb [VL] Use slice instead of

Re: [PR] [VL] Use slice instead of resize in ensureFlattened [incubator-gluten]

2024-04-25 Thread via GitHub
FelixYBW merged PR #5523: URL: https://github.com/apache/incubator-gluten/pull/5523 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] [VL] Extend adapter setup to support Debian distributions [incubator-gluten]

2024-04-25 Thread via GitHub
acvictor commented on PR #5415: URL: https://github.com/apache/incubator-gluten/pull/5415#issuecomment-2077821292 Looks like the patch isn't applying. Can you rebase? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] [VL] Enable array_size Spark function [incubator-gluten]

2024-04-25 Thread via GitHub
github-actions[bot] commented on PR #5539: URL: https://github.com/apache/incubator-gluten/pull/5539#issuecomment-2077806389 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL] Enable array_size Spark function [incubator-gluten]

2024-04-25 Thread via GitHub
github-actions[bot] commented on PR #5539: URL: https://github.com/apache/incubator-gluten/pull/5539#issuecomment-2077806285 Thanks for opening a pull request! Could you open an issue for this pull request on Github Issues?

Re: [PR] [CORE] Upgrade Arrow to 15.0.0 [incubator-gluten]

2024-04-25 Thread via GitHub
github-actions[bot] commented on PR #5174: URL: https://github.com/apache/incubator-gluten/pull/5174#issuecomment-2077698385 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[I] [VL] List UTs failed when upgrading arrow to 15.0.0 [incubator-gluten]

2024-04-25 Thread via GitHub
PHILO-HE opened a new issue, #5538: URL: https://github.com/apache/incubator-gluten/issues/5538 ### Description - SPARK-23786: warning should be printed if CSV header doesn't conform to schema *** FAILED *** testAppender1.loggingEvents.exists(((msg:

Re: [PR] [CORE] Upgrade Arrow to 15.0.0 [incubator-gluten]

2024-04-25 Thread via GitHub
github-actions[bot] commented on PR #5174: URL: https://github.com/apache/incubator-gluten/pull/5174#issuecomment-2077658061 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL] UDF: Support variable arity in function sigatures [incubator-gluten]

2024-04-25 Thread via GitHub
github-actions[bot] commented on PR #5495: URL: https://github.com/apache/incubator-gluten/pull/5495#issuecomment-2077610741 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [VL] Remove linking jemalloc_extension lib belonging to DuckDB build [incubator-gluten]

2024-04-25 Thread via GitHub
GlutenPerfBot commented on PR #5537: URL: https://github.com/apache/incubator-gluten/pull/5537#issuecomment-2077584283 = Performance report for TPCH SF2000 with Velox backend, for reference only query

Re: [PR] [VL][MINOR] Don't link libjemalloc_extension.a of DuckDB if BUILD_JEMALLOC_EXTENSION is OFF [incubator-gluten]

2024-04-25 Thread via GitHub
PHILO-HE commented on PR #5147: URL: https://github.com/apache/incubator-gluten/pull/5147#issuecomment-2077402718 @wankunde, let's close this one as it's replaced by https://github.com/apache/incubator-gluten/pull/5537. -- This is an automated message from the Apache Git Service. To

Re: [PR] [VL][MINOR] Don't link libjemalloc_extension.a of DuckDB if BUILD_JEMALLOC_EXTENSION is OFF [incubator-gluten]

2024-04-25 Thread via GitHub
PHILO-HE closed pull request #5147: [VL][MINOR] Don't link libjemalloc_extension.a of DuckDB if BUILD_JEMALLOC_EXTENSION is OFF URL: https://github.com/apache/incubator-gluten/pull/5147 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] [GLUTEN-5341][VL] Fix SPARK-42782: Hive compatibility check for get_json_object [incubator-gluten]

2024-04-25 Thread via GitHub
GlutenPerfBot commented on PR #5467: URL: https://github.com/apache/incubator-gluten/pull/5467#issuecomment-2077396680 = Performance report for TPCH SF2000 with Velox backend, for reference only query

(incubator-gluten) branch main updated: [VL] Remove linking jemalloc_extension lib belonging to DuckDB build (#5537)

2024-04-25 Thread philo
This is an automated email from the ASF dual-hosted git repository. philo pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git The following commit(s) were added to refs/heads/main by this push: new 4697297e5 [VL] Remove linking

Re: [PR] [VL] Remove linking jemalloc_extension lib belonging to DuckDB build [incubator-gluten]

2024-04-25 Thread via GitHub
PHILO-HE merged PR #5537: URL: https://github.com/apache/incubator-gluten/pull/5537 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] [VL] Remove linking jemalloc_extension lib belonging to DuckDB build [incubator-gluten]

2024-04-25 Thread via GitHub
github-actions[bot] commented on PR #5537: URL: https://github.com/apache/incubator-gluten/pull/5537#issuecomment-2077382628 Thanks for opening a pull request! Could you open an issue for this pull request on Github Issues?

[PR] [VL] Remove linking jemalloc_extension lib belonging to DuckDB build [incubator-gluten]

2024-04-25 Thread via GitHub
PHILO-HE opened a new pull request, #5537: URL: https://github.com/apache/incubator-gluten/pull/5537 ## What changes were proposed in this pull request? Only few change based on https://github.com/apache/incubator-gluten/pull/5147。 ## How was this patch tested? Existing

Re: [PR] [VL][Doc] Remove duplicate content for local cache part [incubator-gluten]

2024-04-25 Thread via GitHub
PHILO-HE commented on PR #5535: URL: https://github.com/apache/incubator-gluten/pull/5535#issuecomment-2077339010 @acvictor, could you take a look? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [I] [VL] date_format returns wrong results [incubator-gluten]

2024-04-25 Thread via GitHub
PHILO-HE commented on issue #5524: URL: https://github.com/apache/incubator-gluten/issues/5524#issuecomment-2077330373 Will investigate this issue. Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] [VL] Remove unused variable in VeloxJniWrapper [incubator-gluten]

2024-04-25 Thread via GitHub
GlutenPerfBot commented on PR #5528: URL: https://github.com/apache/incubator-gluten/pull/5528#issuecomment-2077261796 = Performance report for TPCH SF2000 with Velox backend, for reference only query

Re: [PR] fix shuffle OOM if input batch is extremely large [incubator-gluten]

2024-04-25 Thread via GitHub
github-actions[bot] commented on PR #5536: URL: https://github.com/apache/incubator-gluten/pull/5536#issuecomment-2077230696 Thanks for opening a pull request! Could you open an issue for this pull request on Github Issues?

Re: [PR] [VL][Doc] Remove duplicate content for local cache part [incubator-gluten]

2024-04-25 Thread via GitHub
github-actions[bot] commented on PR #5535: URL: https://github.com/apache/incubator-gluten/pull/5535#issuecomment-2077222336 Thanks for opening a pull request! Could you open an issue for this pull request on Github Issues?

[I] [VL] shuffle writer OOM if input RowVector is extremely large [incubator-gluten]

2024-04-25 Thread via GitHub
guhaiyan0221 opened a new issue, #5534: URL: https://github.com/apache/incubator-gluten/issues/5534 ### Backend VL (Velox) ### Bug description Shuffle will OOM if the flattened rowvector's size is larger than memlimit The case is 500 partitions, row length 315k,

Re: [PR] [VL] Daily Update Velox Version (2024_04_25) [incubator-gluten]

2024-04-25 Thread via GitHub
GlutenPerfBot commented on PR #5522: URL: https://github.com/apache/incubator-gluten/pull/5522#issuecomment-2077146276 = Performance report for TPCH SF2000 with Velox backend, for reference only query

(incubator-gluten) branch main updated: [GLUTEN-5341][VL] Fix SPARK-42782: Hive compatibility check for get_json_object (#5467)

2024-04-25 Thread philo
This is an automated email from the ASF dual-hosted git repository. philo pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git The following commit(s) were added to refs/heads/main by this push: new f1b00547d [GLUTEN-5341][VL] Fix

Re: [PR] [GLUTEN-5341][VL] Fix SPARK-42782: Hive compatibility check for get_json_object [incubator-gluten]

2024-04-25 Thread via GitHub
PHILO-HE merged PR #5467: URL: https://github.com/apache/incubator-gluten/pull/5467 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

(incubator-gluten) branch main updated: [VL] Remove unused variable in VeloxJniWrapper (#5528)

2024-04-25 Thread yuanzhou
This is an automated email from the ASF dual-hosted git repository. yuanzhou pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git The following commit(s) were added to refs/heads/main by this push: new 865945337 [VL] Remove unused variable

Re: [PR] [VL] Remove unused variable in VeloxJniWrapper [incubator-gluten]

2024-04-25 Thread via GitHub
zhouyuan merged PR #5528: URL: https://github.com/apache/incubator-gluten/pull/5528 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

(incubator-gluten) branch main updated: [VL] Daily Update Velox Version (2024_04_25) (#5522)

2024-04-25 Thread yuanzhou
This is an automated email from the ASF dual-hosted git repository. yuanzhou pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git The following commit(s) were added to refs/heads/main by this push: new ec10b497a [VL] Daily Update Velox

Re: [PR] [VL] Daily Update Velox Version (2024_04_25) [incubator-gluten]

2024-04-25 Thread via GitHub
zhouyuan merged PR #5522: URL: https://github.com/apache/incubator-gluten/pull/5522 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] [GLUTEN-5476][CH] Triger merge on insert task [incubator-gluten]

2024-04-25 Thread via GitHub
loneylee commented on code in PR #5529: URL: https://github.com/apache/incubator-gluten/pull/5529#discussion_r1579324379 ## backends-clickhouse/src/main/scala/org/apache/spark/sql/delta/ClickhouseOptimisticTransaction.scala: ## @@ -119,6 +120,16 @@ class

Re: [PR] [GLUTEN-5476][CH] Triger merge on insert task [incubator-gluten]

2024-04-25 Thread via GitHub
github-actions[bot] commented on PR #5529: URL: https://github.com/apache/incubator-gluten/pull/5529#issuecomment-2076980463 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-5532] Clean up some dead code in GlutenPlugin [incubator-gluten]

2024-04-25 Thread via GitHub
github-actions[bot] commented on PR #5533: URL: https://github.com/apache/incubator-gluten/pull/5533#issuecomment-2076954869 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-5532] Clean up some dead code in GlutenPlugin [incubator-gluten]

2024-04-25 Thread via GitHub
github-actions[bot] commented on PR #5533: URL: https://github.com/apache/incubator-gluten/pull/5533#issuecomment-2076954508 https://github.com/apache/incubator-gluten/issues/5532 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[PR] [GLUTEN-5532] Clean up some dead code in GlutenPlugin [incubator-gluten]

2024-04-25 Thread via GitHub
ivoson opened a new pull request, #5533: URL: https://github.com/apache/incubator-gluten/pull/5533 ## What changes were proposed in this pull request? Clean up `SparkConfImplicits` in `GlutenPlugin` as it's not used. ## How was this patch tested? Should Pass CI. --

[I] Clean up some dead code in GlutenPlugin [incubator-gluten]

2024-04-25 Thread via GitHub
ivoson opened a new issue, #5532: URL: https://github.com/apache/incubator-gluten/issues/5532 ### Description `SparkConfImplicits` in `GlutenPlugin` is not used in the project, and since it's `gluten` private, should not be referenced by users. I am wondering that whether we

[I] It's crash when load libgluten.so [incubator-gluten]

2024-04-25 Thread via GitHub
SonixLegend opened a new issue, #5531: URL: https://github.com/apache/incubator-gluten/issues/5531 ### Backend VL (Velox) ### Bug description [Expected behavior] and [actual behavior]. ## Expected behavior Start spark job with gluten plugin ## Actual behavior

Re: [PR] [GLUTEN-5341] Fix some Spark 3.5 UTs [incubator-gluten]

2024-04-25 Thread via GitHub
ayushi-agarwal commented on code in PR #5445: URL: https://github.com/apache/incubator-gluten/pull/5445#discussion_r1579285493 ## backends-velox/src/test/scala/org/apache/gluten/execution/VeloxHashJoinSuite.scala: ## @@ -88,6 +87,8 @@ class VeloxHashJoinSuite extends

Re: [PR] [GLUTEN-5341] Fixes SPARK-42782: Hive compatibility check for get_json_object [incubator-gluten]

2024-04-25 Thread via GitHub
ayushi-agarwal commented on PR #5467: URL: https://github.com/apache/incubator-gluten/pull/5467#issuecomment-2076903223 > Only one nit. I have addressed it, thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] [GLUTEN-5341] Fixes SPARK-42782: Hive compatibility check for get_json_object [incubator-gluten]

2024-04-25 Thread via GitHub
github-actions[bot] commented on PR #5467: URL: https://github.com/apache/incubator-gluten/pull/5467#issuecomment-2076900960 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-5341] Fixes SPARK-42782: Hive compatibility check for get_json_object [incubator-gluten]

2024-04-25 Thread via GitHub
ayushi-agarwal commented on code in PR #5467: URL: https://github.com/apache/incubator-gluten/pull/5467#discussion_r1579270738 ## docs/velox-backend-limitations.md: ## @@ -47,6 +47,8 @@ In certain cases, Gluten result may be different from Vanilla spark. JSON functions

Re: [PR] [GLUTEN-5476][CH] Triger merge on insert task [incubator-gluten]

2024-04-25 Thread via GitHub
zzcclp commented on code in PR #5529: URL: https://github.com/apache/incubator-gluten/pull/5529#discussion_r1579203130 ## backends-clickhouse/src/main/scala/org/apache/spark/sql/delta/ClickhouseOptimisticTransaction.scala: ## @@ -119,6 +120,16 @@ class

Re: [PR] [VL] UDF: Support variable arity in function sigatures [incubator-gluten]

2024-04-25 Thread via GitHub
github-actions[bot] commented on PR #5495: URL: https://github.com/apache/incubator-gluten/pull/5495#issuecomment-2076747280 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-5476][CH] Triger merge on insert task [incubator-gluten]

2024-04-25 Thread via GitHub
github-actions[bot] commented on PR #5529: URL: https://github.com/apache/incubator-gluten/pull/5529#issuecomment-2076731850 https://github.com/apache/incubator-gluten/issues/5476 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

  1   2   >