Re: [PR] [CORE] Basic runnable version of ACBO (Advanced CBO) [incubator-gluten]
zhztheplayer commented on code in PR #5058:
URL: https://github.com/apache/incubator-gluten/pull/5058#discussion_r1533286761
##
gluten-cbo/common/src/main/scala/io/glutenproject/cbo/CboGroup.scala:
##
@@ -0,0 +1,57 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package io.glutenproject.cbo
+
+import io.glutenproject.cbo.memo.MemoStore
+
+trait CboGroup[T <: AnyRef] {
Review Comment:
A set of nodes that shares the same property set in the same cluster.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org
For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [CORE] Basic runnable version of ACBO (Advanced CBO) [incubator-gluten]
zhztheplayer commented on code in PR #5058:
URL: https://github.com/apache/incubator-gluten/pull/5058#discussion_r1533285973
##
gluten-cbo/common/src/main/scala/io/glutenproject/cbo/CboConfig.scala:
##
@@ -0,0 +1,31 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package io.glutenproject.cbo
+
+import io.glutenproject.cbo.CboConfig._
+
+case class CboConfig(
+plannerType: PlannerType = PlannerType.Dp
+)
+
+object CboConfig {
+ sealed trait PlannerType
+ object PlannerType {
+case object Exhaustive extends PlannerType
+case object Dp extends PlannerType
+ }
Review Comment:
`Dp` is the default planner implementation while `Exhaustive` is currently
only used for testing. It's expected that we can implement parallelized
optimization on exhaustive planner comparatively easier than on dp planner in
future.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org
For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [CORE] Basic runnable version of ACBO (Advanced CBO) [incubator-gluten]
zhztheplayer commented on code in PR #5058:
URL: https://github.com/apache/incubator-gluten/pull/5058#discussion_r1533285973
##
gluten-cbo/common/src/main/scala/io/glutenproject/cbo/CboConfig.scala:
##
@@ -0,0 +1,31 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package io.glutenproject.cbo
+
+import io.glutenproject.cbo.CboConfig._
+
+case class CboConfig(
+plannerType: PlannerType = PlannerType.Dp
+)
+
+object CboConfig {
+ sealed trait PlannerType
+ object PlannerType {
+case object Exhaustive extends PlannerType
+case object Dp extends PlannerType
+ }
Review Comment:
`Dp` is the default planner implementation while `Exhaustive` is currently
only used for testing. It's expected that we can implement parallelized
optimization on exhaustive planner comparatively easier than on dp planner.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org
For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [VL] spark url_decode function support [incubator-gluten]
github-actions[bot] commented on PR #5070: URL: https://github.com/apache/incubator-gluten/pull/5070#issuecomment-2011275255 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [CORE] Basic runnable version of ACBO (Advanced CBO) [incubator-gluten]
zhztheplayer commented on code in PR #5058:
URL: https://github.com/apache/incubator-gluten/pull/5058#discussion_r1533284131
##
gluten-cbo/common/src/main/scala/io/glutenproject/cbo/CboCluster.scala:
##
@@ -0,0 +1,80 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package io.glutenproject.cbo
+
+import io.glutenproject.cbo.memo.MemoTable
+
+import scala.collection.mutable
+
+trait CboClusterKey
+
+object CboClusterKey {
+ implicit class CboClusterKeyImplicits[T <: AnyRef](key: CboClusterKey) {
+def propSets(memoTable: MemoTable[T]): Set[PropertySet[T]] = {
+ memoTable.getClusterPropSets(key)
+}
+ }
+}
+
+trait CboCluster[T <: AnyRef] {
Review Comment:
`CboCluster` is a set of nodes sharing the same context in the original
input plan. One cluster can derive a subset of `CboGroup`s. Nodes in one
`CboGroup` shares the same properties.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org
For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [GLUTEN-4917][VL] Refactor CI to build spark.test.home when running UTs [incubator-gluten]
github-actions[bot] commented on PR #4955: URL: https://github.com/apache/incubator-gluten/pull/4955#issuecomment-2011273742 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [CORE] Basic runnable version of ACBO (Advanced CBO) [incubator-gluten]
zhztheplayer commented on code in PR #5058:
URL: https://github.com/apache/incubator-gluten/pull/5058#discussion_r1533284131
##
gluten-cbo/common/src/main/scala/io/glutenproject/cbo/CboCluster.scala:
##
@@ -0,0 +1,80 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package io.glutenproject.cbo
+
+import io.glutenproject.cbo.memo.MemoTable
+
+import scala.collection.mutable
+
+trait CboClusterKey
+
+object CboClusterKey {
+ implicit class CboClusterKeyImplicits[T <: AnyRef](key: CboClusterKey) {
+def propSets(memoTable: MemoTable[T]): Set[PropertySet[T]] = {
+ memoTable.getClusterPropSets(key)
+}
+ }
+}
+
+trait CboCluster[T <: AnyRef] {
Review Comment:
`CboCluster` is a set of nodes sharing the same context in the original
input plan. One cluster can derives a subset of `CboGroup`s. Nodes in one
`CboGroup` shares the same properties.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org
For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [CORE] Basic runnable version of ACBO (Advanced CBO) [incubator-gluten]
zhztheplayer commented on code in PR #5058:
URL: https://github.com/apache/incubator-gluten/pull/5058#discussion_r1533282556
##
gluten-cbo/common/src/main/scala/io/glutenproject/cbo/Cbo.scala:
##
@@ -0,0 +1,212 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package io.glutenproject.cbo
+
+import io.glutenproject.cbo.rule.CboRule
+
+import scala.collection.mutable
+
+/**
+ * Entrypoint of ACBO (Advanced CBO)'s search engine. See basic introduction
of ACBO:
+ * https://github.com/apache/incubator-gluten/issues/5057.
+ */
+trait Optimization[T <: AnyRef] {
+ def newPlanner(
+ plan: T,
+ reqPropSet: PropertySet[T],
+ altPropSets: Seq[PropertySet[T]]): CboPlanner[T]
+
+ def propSetsOf(plan: T): PropertySet[T]
+
+ def withNewConfig(confFunc: CboConfig => CboConfig): Optimization[T]
+}
+
+object Optimization {
+ def apply[T <: AnyRef](
+ costModel: CostModel[T],
+ planModel: PlanModel[T],
+ propertyModel: PropertyModel[T],
+ explain: CboExplain[T],
+ ruleFactory: CboRule.Factory[T]): Optimization[T] = {
+Cbo(costModel, planModel, propertyModel, explain, ruleFactory)
+ }
+
+ implicit class OptimizationImplicits[T <: AnyRef](opt: Optimization[T]) {
+def newPlanner(plan: T): CboPlanner[T] = {
+ opt.newPlanner(plan, opt.propSetsOf(plan), List.empty)
+}
+def newPlanner(plan: T, reqPropSet: PropertySet[T]): CboPlanner[T] = {
+ opt.newPlanner(plan, reqPropSet, List.empty)
+}
+ }
+}
+
+class Cbo[T <: AnyRef] private (
+val config: CboConfig,
+val costModel: CostModel[T],
+val planModel: PlanModel[T],
+val propertyModel: PropertyModel[T],
+val explain: CboExplain[T],
+val ruleFactory: CboRule.Factory[T])
+ extends Optimization[T] {
+ import Cbo._
+
+ override def withNewConfig(confFunc: CboConfig => CboConfig): Cbo[T] = {
+new Cbo(confFunc(config), costModel, planModel, propertyModel, explain,
ruleFactory)
+ }
+
+ // Normal groups start with ID 0, so it's safe to use -1 to do validation.
+ private val dummyGroup: T =
+planModel.newGroupLeaf(-1, PropertySet(Seq.empty), GroupMutator.noop())
+ private val infCost: Cost = costModel.makeInfCost()
+
+ validateModels()
+
+ private def assertThrows(message: String)(u: => Unit): Unit = {
+var notThrew: Boolean = false
+try {
+ u
+ notThrew = true
+} catch {
+ case _: Exception =>
+}
+assert(!notThrew, message)
+ }
+
+ private def validateModels(): Unit = {
Review Comment:
Do validation on user's API implementations.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org
For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [VL] spark url_decode function support [incubator-gluten]
github-actions[bot] commented on PR #5070: URL: https://github.com/apache/incubator-gluten/pull/5070#issuecomment-2011270366 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [CORE] Basic runnable version of ACBO (Advanced CBO) [incubator-gluten]
zhztheplayer commented on code in PR #5058:
URL: https://github.com/apache/incubator-gluten/pull/5058#discussion_r1533281959
##
gluten-cbo/common/src/main/scala/io/glutenproject/cbo/Cbo.scala:
##
@@ -0,0 +1,212 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package io.glutenproject.cbo
+
+import io.glutenproject.cbo.rule.CboRule
+
+import scala.collection.mutable
+
+/**
+ * Entrypoint of ACBO (Advanced CBO)'s search engine. See basic introduction
of ACBO:
+ * https://github.com/apache/incubator-gluten/issues/5057.
+ */
+trait Optimization[T <: AnyRef] {
+ def newPlanner(
+ plan: T,
+ reqPropSet: PropertySet[T],
+ altPropSets: Seq[PropertySet[T]]): CboPlanner[T]
+
+ def propSetsOf(plan: T): PropertySet[T]
+
+ def withNewConfig(confFunc: CboConfig => CboConfig): Optimization[T]
+}
+
+object Optimization {
+ def apply[T <: AnyRef](
+ costModel: CostModel[T],
+ planModel: PlanModel[T],
+ propertyModel: PropertyModel[T],
+ explain: CboExplain[T],
+ ruleFactory: CboRule.Factory[T]): Optimization[T] = {
+Cbo(costModel, planModel, propertyModel, explain, ruleFactory)
+ }
+
+ implicit class OptimizationImplicits[T <: AnyRef](opt: Optimization[T]) {
+def newPlanner(plan: T): CboPlanner[T] = {
+ opt.newPlanner(plan, opt.propSetsOf(plan), List.empty)
+}
+def newPlanner(plan: T, reqPropSet: PropertySet[T]): CboPlanner[T] = {
+ opt.newPlanner(plan, reqPropSet, List.empty)
+}
+ }
+}
+
+class Cbo[T <: AnyRef] private (
Review Comment:
`Cbo` is a stateless optimization context consisting of configs and
utilities.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org
For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [VL] spark url decode function support [incubator-gluten]
github-actions[bot] commented on PR #5070:
URL:
https://github.com/apache/incubator-gluten/pull/5070#issuecomment-2011270148
Thanks for opening a pull request!
Could you open an issue for this pull request on Github Issues?
https://github.com/apache/incubator-gluten/issues
Then could you also rename ***commit message*** and ***pull request title***
in the following format?
[GLUTEN-${ISSUES_ID}][COMPONENT]feat/fix: ${detailed message}
See also:
* [Other pull requests](https://github.com/apache/incubator-gluten/pulls/)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org
For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [CORE] Basic runnable version of ACBO (Advanced CBO) [incubator-gluten]
zhztheplayer commented on code in PR #5058:
URL: https://github.com/apache/incubator-gluten/pull/5058#discussion_r1533281959
##
gluten-cbo/common/src/main/scala/io/glutenproject/cbo/Cbo.scala:
##
@@ -0,0 +1,212 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package io.glutenproject.cbo
+
+import io.glutenproject.cbo.rule.CboRule
+
+import scala.collection.mutable
+
+/**
+ * Entrypoint of ACBO (Advanced CBO)'s search engine. See basic introduction
of ACBO:
+ * https://github.com/apache/incubator-gluten/issues/5057.
+ */
+trait Optimization[T <: AnyRef] {
+ def newPlanner(
+ plan: T,
+ reqPropSet: PropertySet[T],
+ altPropSets: Seq[PropertySet[T]]): CboPlanner[T]
+
+ def propSetsOf(plan: T): PropertySet[T]
+
+ def withNewConfig(confFunc: CboConfig => CboConfig): Optimization[T]
+}
+
+object Optimization {
+ def apply[T <: AnyRef](
+ costModel: CostModel[T],
+ planModel: PlanModel[T],
+ propertyModel: PropertyModel[T],
+ explain: CboExplain[T],
+ ruleFactory: CboRule.Factory[T]): Optimization[T] = {
+Cbo(costModel, planModel, propertyModel, explain, ruleFactory)
+ }
+
+ implicit class OptimizationImplicits[T <: AnyRef](opt: Optimization[T]) {
+def newPlanner(plan: T): CboPlanner[T] = {
+ opt.newPlanner(plan, opt.propSetsOf(plan), List.empty)
+}
+def newPlanner(plan: T, reqPropSet: PropertySet[T]): CboPlanner[T] = {
+ opt.newPlanner(plan, reqPropSet, List.empty)
+}
+ }
+}
+
+class Cbo[T <: AnyRef] private (
Review Comment:
`Cbo` is a stateless optimization context consisting configs and utilities.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org
For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [CORE] Basic runnable version of ACBO (Advanced CBO) [incubator-gluten]
zhztheplayer commented on code in PR #5058: URL: https://github.com/apache/incubator-gluten/pull/5058#discussion_r1533281350 ## .github/workflows/velox_be.yml: ## @@ -450,6 +450,16 @@ jobs: --local --preset=velox --benchmark-type=h --error-on-memleak --off-heap-size=10g -s=1.0 --threads=16 --iterations=1 \ && GLUTEN_IT_JVM_ARGS=-Xmx20G sbin/gluten-it.sh queries-compare \ --local --preset=velox --benchmark-type=ds --error-on-memleak --off-heap-size=40g -s=10.0 --threads=32 --iterations=1' + - name: TPC-H SF1.0 && TPC-DS SF10.0 Parquet local spark3.2 with advanced CBO +run: | + $PATH_TO_GLUTEN_TE/$OS_IMAGE_NAME/gha/gha-checkout/exec.sh 'cd /opt/gluten/tools/gluten-it && \ + mvn clean install -Pspark-3.2 \ + && GLUTEN_IT_JVM_ARGS=-Xmx5G sbin/gluten-it.sh queries-compare \ +--local --preset=velox --benchmark-type=h --error-on-memleak --off-heap-size=10g -s=1.0 --threads=16 --iterations=1 \ +--extra-conf=spark.gluten.sql.advanced.cbo.enabled=true \ + && GLUTEN_IT_JVM_ARGS=-Xmx20G sbin/gluten-it.sh queries-compare \ +--local --preset=velox --benchmark-type=ds --error-on-memleak --off-heap-size=40g -s=10.0 --threads=32 --iterations=1 \ +--extra-conf=spark.gluten.sql.advanced.cbo.enabled=true' Review Comment: CI Job for ACBO + Velox + TPC-H SF1 + TPC-DS SF10 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [CORE] Basic runnable version of ACBO (Advanced CBO) [incubator-gluten]
github-actions[bot] commented on PR #5058: URL: https://github.com/apache/incubator-gluten/pull/5058#issuecomment-2011266413 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [DNM][VL] GHA with pre-built docker image and github runner [incubator-gluten]
github-actions[bot] commented on PR #4936: URL: https://github.com/apache/incubator-gluten/pull/4936#issuecomment-2011262778 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [GLUTEN-4675][CH] Support write mergetree to s3 [incubator-gluten]
github-actions[bot] commented on PR #4676: URL: https://github.com/apache/incubator-gluten/pull/4676#issuecomment-2011261106 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [CORE] Basic runnable version of ACBO (Advanced CBO) [incubator-gluten]
github-actions[bot] commented on PR #5058: URL: https://github.com/apache/incubator-gluten/pull/5058#issuecomment-2011252200 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [VL] Support YearMonthIntervalType and enable make_ym_interval [incubator-gluten]
github-actions[bot] commented on PR #4798: URL: https://github.com/apache/incubator-gluten/pull/4798#issuecomment-2011248745 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [CORE] Enable bit_length Spark function [incubator-gluten]
github-actions[bot] commented on PR #5069: URL: https://github.com/apache/incubator-gluten/pull/5069#issuecomment-2011246737 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [CORE] Enable bit_length Spark function [incubator-gluten]
github-actions[bot] commented on PR #5069: URL: https://github.com/apache/incubator-gluten/pull/5069#issuecomment-2011244293 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [VL] Fix bug where session config is lost when benchmark is enabled [incubator-gluten]
GlutenPerfBot commented on PR #5054: URL: https://github.com/apache/incubator-gluten/pull/5054#issuecomment-2011244056 = Performance report for TPCH SF2000 with Velox backend, for reference only query log/native_5054_time.csv log/native_master_03_20_2024_e1f0c01a7_time.csv difference percentage q1 35.69 38.53 2.838 107.95% q2 26.74 24.00 -2.742 89.75% q3 36.67 37.50 0.832 102.27% q4 40.95 38.61 -2.344 94.28% q5 69.82 71.63 1.807 102.59% q6 7.38 7.37 -0.013 99.82% q7 85.30 83.96 -1.341 98.43% q8 85.02 84.46 -0.557 99.34% q9 123.10 125.15 2.047 101.66% q10 46.28 45.58 -0.697 98.49% q11 20.62 20.52 -0.103 99.50% q12 27.78 28.80 1.025 103.69% q13 47.35 48.70 1.347 102.84% q14 16.86 22.25 5.384 131.92% q15 30.26 31.14 0.886 102.93% q16 14.31 15.42 1.111 107.76% q17 100.25 102.47 2.216 102.21% q18 143.33 143.60 0.266 100.19% q19 13.66 13.64 -0.017 99.88%
Re: [PR] [CORE] Enable bit_length Spark function [incubator-gluten]
github-actions[bot] commented on PR #5069:
URL:
https://github.com/apache/incubator-gluten/pull/5069#issuecomment-2011244100
Thanks for opening a pull request!
Could you open an issue for this pull request on Github Issues?
https://github.com/apache/incubator-gluten/issues
Then could you also rename ***commit message*** and ***pull request title***
in the following format?
[GLUTEN-${ISSUES_ID}][COMPONENT]feat/fix: ${detailed message}
See also:
* [Other pull requests](https://github.com/apache/incubator-gluten/pulls/)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org
For additional commands, e-mail: commits-h...@gluten.apache.org
[PR] [CORE] Enable bit_length Spark function [incubator-gluten]
PHILO-HE opened a new pull request, #5069: URL: https://github.com/apache/incubator-gluten/pull/5069 ## How was this patch tested? A test added & the imported Spark tests. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [GLUTEN-5061][CH] Fix assert error when writing mergetree data with select * from table limit n [incubator-gluten]
github-actions[bot] commented on PR #5068: URL: https://github.com/apache/incubator-gluten/pull/5068#issuecomment-2011240455 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [GLUTEN-5061][CH] Fix assert error when writing mergetree data with select * from table limit n [incubator-gluten]
github-actions[bot] commented on PR #5068: URL: https://github.com/apache/incubator-gluten/pull/5068#issuecomment-2011240227 https://github.com/apache/incubator-gluten/issues/5061 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
[PR] [GLUTEN-5061][CH] Fix assert error when writing mergetree data with select * from table limit n [incubator-gluten]
zzcclp opened a new pull request, #5068: URL: https://github.com/apache/incubator-gluten/pull/5068 ## What changes were proposed in this pull request? The below writing mergetree data sql will throw the assert error: ``` insert into table lineitem_mergetree select * from lineitem **limit 10** ``` RC: with limit n, there is a wrong wrapper logical with the FakeRowAdaptor for the query plan. Close #5061. (Fixes: #5061) ## How was this patch tested? (Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests) (If this patch involves UI changes, please attach a screenshot; otherwise, remove this) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [VL] Not fallback for function spark_partition_id and monotonically_increasing_id [incubator-gluten]
gaoyangxiaozhu commented on PR #5046: URL: https://github.com/apache/incubator-gluten/pull/5046#issuecomment-2011235388 > Can you also help to add input_file_name() function? It's very useful for debug purpose. acked -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [DNM][VL] GHA with pre-built docker image and github runner [incubator-gluten]
github-actions[bot] commented on PR #4936: URL: https://github.com/apache/incubator-gluten/pull/4936#issuecomment-2011229184 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [GLUTEN-5016][CH] Fix exchange fallback in simple aggregation sql if spark.gluten.sql.columnar.preferColumnar=false [incubator-gluten]
github-actions[bot] commented on PR #5042: URL: https://github.com/apache/incubator-gluten/pull/5042#issuecomment-2011228457 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [CORE] Basic runnable version of ACBO (Advanced CBO) [incubator-gluten]
github-actions[bot] commented on PR #5058: URL: https://github.com/apache/incubator-gluten/pull/5058#issuecomment-2011228018 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [DNM][VL] GHA with pre-built docker image and github runner [incubator-gluten]
github-actions[bot] commented on PR #4936: URL: https://github.com/apache/incubator-gluten/pull/4936#issuecomment-2011225244 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [CORE] Basic runnable version of ACBO (Advanced CBO) [incubator-gluten]
github-actions[bot] commented on PR #5058: URL: https://github.com/apache/incubator-gluten/pull/5058#issuecomment-2011222339 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [DNM][VL] GHA with pre-built docker image and github runner [incubator-gluten]
github-actions[bot] commented on PR #4936: URL: https://github.com/apache/incubator-gluten/pull/4936#issuecomment-2011220260 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [DNM][VL] GHA with pre-built docker image and github runner [incubator-gluten]
github-actions[bot] commented on PR #4936: URL: https://github.com/apache/incubator-gluten/pull/4936#issuecomment-2011215694 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [CORE] Basic runnable version of ACBO (Advanced CBO) [incubator-gluten]
github-actions[bot] commented on PR #5058: URL: https://github.com/apache/incubator-gluten/pull/5058#issuecomment-2011209005 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [CORE] Basic runnable version of ACBO (Advanced CBO) [incubator-gluten]
github-actions[bot] commented on PR #5058: URL: https://github.com/apache/incubator-gluten/pull/5058#issuecomment-2011206379 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [VL] Fix bug where session config is lost when benchmark is enabled [incubator-gluten]
FelixYBW merged PR #5054: URL: https://github.com/apache/incubator-gluten/pull/5054 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
(incubator-gluten) branch main updated: [VL] Fix bug where session config is lost when benchmark is enabled (#5054)
This is an automated email from the ASF dual-hosted git repository.
felixybw pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git
The following commit(s) were added to refs/heads/main by this push:
new e317e4d5d [VL] Fix bug where session config is lost when benchmark is
enabled (#5054)
e317e4d5d is described below
commit e317e4d5dd845ab98d8ef2b46a811a6f2635ef4f
Author: BInwei Yang
AuthorDate: Wed Mar 20 21:28:47 2024 -0700
[VL] Fix bug where session config is lost when benchmark is enabled (#5054)
It's a quick bug fix.
unordered_map::merge removes the duplicated items from source.
---
cpp/velox/compute/VeloxRuntime.cc | 5 -
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/cpp/velox/compute/VeloxRuntime.cc
b/cpp/velox/compute/VeloxRuntime.cc
index f8f2a527c..a77ddfbba 100644
--- a/cpp/velox/compute/VeloxRuntime.cc
+++ b/cpp/velox/compute/VeloxRuntime.cc
@@ -220,7 +220,10 @@ std::unique_ptr
VeloxRuntime::createColumnarBatchSerial
void VeloxRuntime::dumpConf(const std::string& path) {
auto backendConf = VeloxBackend::get()->getBackendConf();
auto allConf = backendConf;
- allConf.merge(confMap_);
+
+ for (const auto& pair : confMap_) {
+allConf.insert_or_assign(pair.first, pair.second);
+ }
// Open file "velox.conf" for writing, automatically creating it if it
doesn't exist,
// or overwriting it if it does.
-
To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org
For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [GLUTEN-5060][CH] Remove unnecessary FilterExec execution when querying from MergeTree with the prewhere [incubator-gluten]
github-actions[bot] commented on PR #5067: URL: https://github.com/apache/incubator-gluten/pull/5067#issuecomment-2011188528 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [GLUTEN-5060][CH] Remove unnecessary FilterExec execution when querying from MergeTree with the prewhere [incubator-gluten]
github-actions[bot] commented on PR #5067: URL: https://github.com/apache/incubator-gluten/pull/5067#issuecomment-2011188407 https://github.com/apache/incubator-gluten/issues/5060 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [GLUTEN-4675][CH] Support write mergetree to s3 [incubator-gluten]
github-actions[bot] commented on PR #4676: URL: https://github.com/apache/incubator-gluten/pull/4676#issuecomment-2011184700 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [GLUTEN-5016][CH] Fix exchange fallback in simple aggregation sql if spark.gluten.sql.columnar.preferColumnar=false [incubator-gluten]
github-actions[bot] commented on PR #5042: URL: https://github.com/apache/incubator-gluten/pull/5042#issuecomment-2011182191 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [GLUTEN-1632][CH]Daily Update Clickhouse Version (20240321) [incubator-gluten]
github-actions[bot] commented on PR #5055: URL: https://github.com/apache/incubator-gluten/pull/5055#issuecomment-2011181892 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
(incubator-gluten) branch main updated: [CH][Minor] Fix build due to Clickhouse Refactor (#5059)
This is an automated email from the ASF dual-hosted git repository.
changchen pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git
The following commit(s) were added to refs/heads/main by this push:
new e621e52a2 [CH][Minor] Fix build due to Clickhouse Refactor (#5059)
e621e52a2 is described below
commit e621e52a23cfed7aab943b375cd6190bceba69ec
Author: Chang chen
AuthorDate: Thu Mar 21 12:05:49 2024 +0800
[CH][Minor] Fix build due to Clickhouse Refactor (#5059)
* fix build due to https://github.com/ClickHouse/ClickHouse/pull/61604
(cherry picked from commit a36b09f3adb554fd2d64e6e66b52a53aa52f7ce6)
* fix build due to https://github.com/ClickHouse/ClickHouse/pull/61604
---
cpp-ch/local-engine/Common/DebugUtils.cpp | 1 +
cpp-ch/local-engine/Shuffle/SelectorBuilder.h | 1 +
cpp-ch/local-engine/Storages/Parquet/ColumnIndexFilter.cpp | 1 +
3 files changed, 3 insertions(+)
diff --git a/cpp-ch/local-engine/Common/DebugUtils.cpp
b/cpp-ch/local-engine/Common/DebugUtils.cpp
index c9f439645..35f1f98cd 100644
--- a/cpp-ch/local-engine/Common/DebugUtils.cpp
+++ b/cpp-ch/local-engine/Common/DebugUtils.cpp
@@ -15,6 +15,7 @@
* limitations under the License.
*/
#include "DebugUtils.h"
+#include
#include
#include
#include
diff --git a/cpp-ch/local-engine/Shuffle/SelectorBuilder.h
b/cpp-ch/local-engine/Shuffle/SelectorBuilder.h
index 7646e3cf8..4c35d488a 100644
--- a/cpp-ch/local-engine/Shuffle/SelectorBuilder.h
+++ b/cpp-ch/local-engine/Shuffle/SelectorBuilder.h
@@ -26,6 +26,7 @@
#include
#include
#include
+#include
#include
#include
diff --git a/cpp-ch/local-engine/Storages/Parquet/ColumnIndexFilter.cpp
b/cpp-ch/local-engine/Storages/Parquet/ColumnIndexFilter.cpp
index 694cf49f2..0d3b07e47 100644
--- a/cpp-ch/local-engine/Storages/Parquet/ColumnIndexFilter.cpp
+++ b/cpp-ch/local-engine/Storages/Parquet/ColumnIndexFilter.cpp
@@ -24,6 +24,7 @@
#include
#include
#include
+#include
namespace local_engine
{
-
To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org
For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [CH][Minor] Fix build due to Clickhouse Refactor [incubator-gluten]
baibaichen merged PR #5059: URL: https://github.com/apache/incubator-gluten/pull/5059 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [CORE] Pullout pre-project for ExpandExec [incubator-gluten]
github-actions[bot] commented on PR #5066:
URL:
https://github.com/apache/incubator-gluten/pull/5066#issuecomment-2011175332
Thanks for opening a pull request!
Could you open an issue for this pull request on Github Issues?
https://github.com/apache/incubator-gluten/issues
Then could you also rename ***commit message*** and ***pull request title***
in the following format?
[GLUTEN-${ISSUES_ID}][COMPONENT]feat/fix: ${detailed message}
See also:
* [Other pull requests](https://github.com/apache/incubator-gluten/pulls/)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org
For additional commands, e-mail: commits-h...@gluten.apache.org
[PR] [CORE] Pullout pre-project for ExpandExec [incubator-gluten]
liujiayi771 opened a new pull request, #5066: URL: https://github.com/apache/incubator-gluten/pull/5066 ## What changes were proposed in this pull request? Support pullout pre-project for ExpandExec. ## How was this patch tested? N/A -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [CORE] Pullout pre-project for ExpandExec [incubator-gluten]
github-actions[bot] commented on PR #5066: URL: https://github.com/apache/incubator-gluten/pull/5066#issuecomment-2011175501 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [DOC] Remove arrow version setting with 11.0.0-gluten [incubator-gluten]
PHILO-HE commented on PR #5065: URL: https://github.com/apache/incubator-gluten/pull/5065#issuecomment-2011173131 Assuming Gluten's dependency on 11.0.0-gluten was depreciated long time ago, this pr is simply removing it. @zhztheplayer, could you help review? Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [CORE] Basic runnable version of ACBO (Advanced CBO) [incubator-gluten]
github-actions[bot] commented on PR #5058: URL: https://github.com/apache/incubator-gluten/pull/5058#issuecomment-2011169296 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
[PR] [DOC] Remove 11.0.0-gluten set for arrow version [incubator-gluten]
PHILO-HE opened a new pull request, #5065: URL: https://github.com/apache/incubator-gluten/pull/5065 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [DOC] Remove 11.0.0-gluten set for arrow version [incubator-gluten]
github-actions[bot] commented on PR #5065:
URL:
https://github.com/apache/incubator-gluten/pull/5065#issuecomment-2011168904
Thanks for opening a pull request!
Could you open an issue for this pull request on Github Issues?
https://github.com/apache/incubator-gluten/issues
Then could you also rename ***commit message*** and ***pull request title***
in the following format?
[GLUTEN-${ISSUES_ID}][COMPONENT]feat/fix: ${detailed message}
See also:
* [Other pull requests](https://github.com/apache/incubator-gluten/pulls/)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org
For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [GLUTEN-4946][CH] Fix avg(bigint) overflow [incubator-gluten]
github-actions[bot] commented on PR #5048: URL: https://github.com/apache/incubator-gluten/pull/5048#issuecomment-2011163819 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [GLUTEN-4956][CH] Fix parsing string with blank prefix/suffix to number [incubator-gluten]
github-actions[bot] commented on PR #5022: URL: https://github.com/apache/incubator-gluten/pull/5022#issuecomment-2011163630 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [CORE] Basic runnable version of ACBO (Advanced CBO) [incubator-gluten]
github-actions[bot] commented on PR #5058: URL: https://github.com/apache/incubator-gluten/pull/5058#issuecomment-2011163149 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [VL] Daily Update Velox Version (2024_03_21) [incubator-gluten]
github-actions[bot] commented on PR #5064:
URL:
https://github.com/apache/incubator-gluten/pull/5064#issuecomment-2011161632
Thanks for opening a pull request!
Could you open an issue for this pull request on Github Issues?
https://github.com/apache/incubator-gluten/issues
Then could you also rename ***commit message*** and ***pull request title***
in the following format?
[GLUTEN-${ISSUES_ID}][COMPONENT]feat/fix: ${detailed message}
See also:
* [Other pull requests](https://github.com/apache/incubator-gluten/pulls/)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org
For additional commands, e-mail: commits-h...@gluten.apache.org
[PR] [VL] Daily Update Velox Version (2024_03_21) [incubator-gluten]
GlutenPerfBot opened a new pull request, #5064: URL: https://github.com/apache/incubator-gluten/pull/5064 Upstream Velox's New Commits: ```txt 64f9618c3 by Wei He, Fix exception thrown in custom result verifier in WindowFuzzer (9189) 3db2a9ad8 by Nicholas Ormrod, Deshim //folly:dynamic to //folly/json:dynamic in velox (8931) 43a552715 by Sergey Pershin, Add Task:driverCounts() (9166) 7e8105a0b by rui-mo, Fix gatherFromTimestampBuffer (9170) 5186d0bc9 by joey.ljy, Fix the CMakeLists.txt to sort the cpp files in alphabetical order (9164) 6f09b8fe5 by Richard Barnes, Remove unused exception parameter from velox/common/base/tests/StatusTest.cpp (9180) b8c752abc by Richard Barnes, Enable `-Wmismatched-tags` in velox/PACKAGE (9181) 2c044edfc by Richard Barnes, Remove unused variables in velox/tpch/gen/dbgen/text.cpp (9142) e574117a4 by Richard Barnes, Remove unused variables in velox/connectors/hive/tests/HiveDataSinkTest.cpp f99cf2858 by Masha Basmanova, Extract CppToType and CastTypeChecker from Type.h (9161) 491c46fa3 by Krishna Pai, Fix build on ubuntu and gcc 11 (9163) 469e2da6c by Jacob Wujciak-Jens, Move Linux build, adapters and presto fuzzer to gha (8734) 443c72dc2 by joey.ljy, Unregister the UDTs after use in ArrayViewTest and ArrayWriterTest (9175) 35fd4b757 by Ma, Rong, Add make_ym_interval Spark function (8887) fe0c1fd52 by Jialiang Tan, Threshold based spilling removal (9069) 07d638c1b by 8dukongjian, Fix regr_r2 failure in AggregationFuzzer (9173) 8981498a9 by Masha Basmanova, Replace fromKindToScalerType with createScalarType (9162) cd2523fac by Kevin Wilfong, Fix hang in MultiThreadedTaskCursor when Task fails (9134) 5e077909f by xiaoxmeng, Table scan cleanup (9156) 21491d74a by Kevin Wilfong, Fix NPE caused by testingRunArbitration (9137) 1d95a474a by Kevin Wilfong, Fix bug in PrestoVectorSerializers when serializing empty Vectors with complex types (9157) 29d0f317d by Wei He, Fix window fuzzer test (9153) ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [GLUTEN-5016][CH] Fix exchange fallback in simple aggregation sql if spark.gluten.sql.columnar.preferColumnar=false [incubator-gluten]
github-actions[bot] commented on PR #5042: URL: https://github.com/apache/incubator-gluten/pull/5042#issuecomment-2011158227 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [GLUTEN-5016][CH] Fix exchange fallback in simple aggregation sql if spark.gluten.sql.columnar.preferColumnar=false [incubator-gluten]
github-actions[bot] commented on PR #5042: URL: https://github.com/apache/incubator-gluten/pull/5042#issuecomment-2011158088 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
(incubator-gluten) branch main updated: [GLUTEN-5049][CH] Clean code in substring function parser and fix s3 building issue(#5050)
This is an automated email from the ASF dual-hosted git repository.
changchen pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git
The following commit(s) were added to refs/heads/main by this push:
new 171601b97 [GLUTEN-5049][CH] Clean code in substring function parser
and fix s3 building issue(#5050)
171601b97 is described below
commit 171601b97e07e3d0d69b0c2608b28dcebf268a8b
Author: 李扬 <654010...@qq.com>
AuthorDate: Thu Mar 21 11:31:08 2024 +0800
[GLUTEN-5049][CH] Clean code in substring function parser and fix s3
building issue(#5050)
---
.../Parser/scalar_function_parser/substring.cpp| 34 ++
.../Storages/SubstraitSource/ReadBufferBuilder.cpp | 5 +++-
2 files changed, 19 insertions(+), 20 deletions(-)
diff --git a/cpp-ch/local-engine/Parser/scalar_function_parser/substring.cpp
b/cpp-ch/local-engine/Parser/scalar_function_parser/substring.cpp
index 2a44c3e38..64c97da80 100644
--- a/cpp-ch/local-engine/Parser/scalar_function_parser/substring.cpp
+++ b/cpp-ch/local-engine/Parser/scalar_function_parser/substring.cpp
@@ -43,27 +43,23 @@ public:
ActionsDAGPtr & actions_dag) const override
{
auto parsed_args = parseFunctionArguments(substrait_func, "",
actions_dag);
-if (parsed_args.size() != 2 && parsed_args.size() != 3)
+if (parsed_args.size() != 3)
throw Exception(DB::ErrorCodes::NUMBER_OF_ARGUMENTS_DOESNT_MATCH,
"Function {} requires two or three arguments", getName());
-DB::DataTypePtr start_index_data_type =
removeNullable(parsed_args[1]->result_type);
-if (!isInteger(start_index_data_type))
-throw Exception(DB::ErrorCodes::BAD_ARGUMENTS, "Function {}'s
second arguments must be int type", getName());
- /**
-parse substring(str, start_index, length) as
-if (start_index == 0)
-substring(str, 1, length)
-else
-substring(str, start_index, length)
+
+/*
+parse substring(str, index, length) as
+substring(str, if(index = 0, 1, index), length)
*/
-auto * const_zero_node = addColumnToActionsDAG(actions_dag,
start_index_data_type, Field(0));
-auto * const_one_node = addColumnToActionsDAG(actions_dag,
start_index_data_type, Field(1));
-auto * equals_zero_node = toFunctionNode(actions_dag, "equals",
{parsed_args[1], const_zero_node});
-auto * if_node = toFunctionNode(actions_dag, "if", {equals_zero_node,
const_one_node, parsed_args[1]});
-const DB::ActionsDAG::Node * substring_func_node;
-if (parsed_args.size() == 2)
-substring_func_node = toFunctionNode(actions_dag, "substringUTF8",
{parsed_args[0], if_node});
-else
-substring_func_node = toFunctionNode(actions_dag, "substringUTF8",
{parsed_args[0], if_node, parsed_args[2]});
+const auto * str_arg = parsed_args[0];
+const auto * index_arg = parsed_args[1];
+const auto * length_arg = parsed_args[2];
+
+auto index_type = std::make_shared();
+const auto * const_zero_node = addColumnToActionsDAG(actions_dag,
index_type, 0);
+const auto * const_one_node = addColumnToActionsDAG(actions_dag,
index_type, 1);
+const auto * equals_zero_node = toFunctionNode(actions_dag, "equals",
{index_arg, const_zero_node});
+const auto * if_node = toFunctionNode(actions_dag, "if",
{equals_zero_node, const_one_node, index_arg});
+const auto * substring_func_node = toFunctionNode(actions_dag,
"substringUTF8", {str_arg, if_node, length_arg});
return convertNodeTypeIfNeeded(substrait_func, substring_func_node,
actions_dag);
}
protected:
diff --git a/cpp-ch/local-engine/Storages/SubstraitSource/ReadBufferBuilder.cpp
b/cpp-ch/local-engine/Storages/SubstraitSource/ReadBufferBuilder.cpp
index f6862ca79..33243c7a0 100644
--- a/cpp-ch/local-engine/Storages/SubstraitSource/ReadBufferBuilder.cpp
+++ b/cpp-ch/local-engine/Storages/SubstraitSource/ReadBufferBuilder.cpp
@@ -32,7 +32,6 @@
#include
#include
#include
-#include
#include
#include
@@ -49,9 +48,13 @@
#include
#include
+#if USE_AWS_S3
+#include
#include
#include
#include
+#endif
+
#include
#include
-
To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org
For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [I] [CH] Remove useless codes in substring function parser and fix s3 building issue [incubator-gluten]
baibaichen closed issue #5049: [CH] Remove useless codes in substring function parser and fix s3 building issue URL: https://github.com/apache/incubator-gluten/issues/5049 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [GLUTEN-5049][CH] Clean code in substring function parser and fix s3 building issue [incubator-gluten]
baibaichen merged PR #5050: URL: https://github.com/apache/incubator-gluten/pull/5050 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [VL][DNM]Test Q95 post probe spill [incubator-gluten]
github-actions[bot] commented on PR #5063: URL: https://github.com/apache/incubator-gluten/pull/5063#issuecomment-2011098667 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [VL][DNM]Test Q95 post probe spill [incubator-gluten]
github-actions[bot] commented on PR #5063:
URL:
https://github.com/apache/incubator-gluten/pull/5063#issuecomment-2011098530
Thanks for opening a pull request!
Could you open an issue for this pull request on Github Issues?
https://github.com/apache/incubator-gluten/issues
Then could you also rename ***commit message*** and ***pull request title***
in the following format?
[GLUTEN-${ISSUES_ID}][COMPONENT]feat/fix: ${detailed message}
See also:
* [Other pull requests](https://github.com/apache/incubator-gluten/pulls/)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org
For additional commands, e-mail: commits-h...@gluten.apache.org
[PR] [VL][DNM]Test Q95 post probe spill [incubator-gluten]
JkSelf opened a new pull request, #5063: URL: https://github.com/apache/incubator-gluten/pull/5063 ## What changes were proposed in this pull request? Test whether post probe join spill work with velox newly spill related commits. ## How was this patch tested? Turn on Q95 failed CI. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [DNM][VL] GHA with pre-built docker image and github runner [incubator-gluten]
github-actions[bot] commented on PR #4936: URL: https://github.com/apache/incubator-gluten/pull/4936#issuecomment-2011097856 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [CORE] Basic runnable version of ACBO (Advanced CBO) [incubator-gluten]
github-actions[bot] commented on PR #5058: URL: https://github.com/apache/incubator-gluten/pull/5058#issuecomment-2011095509 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [CH][Minor] Fix build due to Clickhouse Refactor [incubator-gluten]
github-actions[bot] commented on PR #5059: URL: https://github.com/apache/incubator-gluten/pull/5059#issuecomment-2011045236 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
[I] [CH] Remove unnecessary FilterExec execution when querying from MergeTree with the prewhere [incubator-gluten]
zzcclp opened a new issue, #5060: URL: https://github.com/apache/incubator-gluten/issues/5060 ### Backend CH (ClickHouse) ### Bug description When querying from MergeTree with the prewhere, all the filters will be pushdowned to the ScanExec, so it does not need to execute the FilterExec again. ### Spark version None ### Spark configurations _No response_ ### System information _No response_ ### Relevant logs _No response_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] Support approx_percentile [incubator-gluten]
WangGuangxin commented on PR #5007: URL: https://github.com/apache/incubator-gluten/pull/5007#issuecomment-2011045153 > this yes, we should merge to upsteam first before merge this PR -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [WIP] Upload build library [incubator-gluten]
wangyum commented on PR #5056: URL: https://github.com/apache/incubator-gluten/pull/5056#issuecomment-2011043112 ``` Error: Failed to CreateArtifact: Unable to make request: ECONNRESET If you are using self-hosted runners, please make sure your runner has access to all GitHub endpoints: https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/about-self-hosted-runners#communication-between-self-hosted-runners-and-github ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [CORE] Pullout pre/post project for generate [incubator-gluten]
ulysses-you commented on code in PR #4952:
URL: https://github.com/apache/incubator-gluten/pull/4952#discussion_r1533139052
##
backends-velox/src/main/scala/io/glutenproject/backendsapi/velox/SparkPlanExecApiImpl.scala:
##
@@ -659,4 +659,14 @@ class SparkPlanExecApiImpl extends SparkPlanExecApi {
): GenerateExecTransformerBase = {
GenerateExecTransformer(generator, requiredChildOutput, outer,
generatorOutput, child)
}
+
+ override def genPreProjectForGenerate(generate: GenerateExec): SparkPlan = {
+PullOutGenerateProjectHelper.pullOutPreProject(generate)
+ }
+
+ override def genPostProjectForGenerate(generate: GenerateExec): SparkPlan = {
+PullOutGenerateProjectHelper.pullOutPostProject(generate)
+ }
}
+
+object SparkPlanExecApiImpl {}
Review Comment:
nit: unnecessary change
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org
For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [GLUTEN-4213][CORE] Refactoring insertion process of pre/post projection [incubator-gluten]
github-actions[bot] closed pull request #4245: [GLUTEN-4213][CORE] Refactoring insertion process of pre/post projection URL: https://github.com/apache/incubator-gluten/pull/4245 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [GLUTEN-4213][CORE] Refactoring insertion process of pre/post projection [incubator-gluten]
github-actions[bot] commented on PR #4245: URL: https://github.com/apache/incubator-gluten/pull/4245#issuecomment-2011032871 This PR was auto-closed because it has been stalled for 10 days with no activity. Please feel free to reopen if it is still valid. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [GLUTEN-4483][CH]Improve divide [incubator-gluten]
github-actions[bot] closed pull request #4484: [GLUTEN-4483][CH]Improve divide URL: https://github.com/apache/incubator-gluten/pull/4484 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [GLUTEN-4483][CH]Improve divide [incubator-gluten]
github-actions[bot] commented on PR #4484: URL: https://github.com/apache/incubator-gluten/pull/4484#issuecomment-2011032836 This PR was auto-closed because it has been stalled for 10 days with no activity. Please feel free to reopen if it is still valid. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [GLUTEN-1632][CH]Daily Update Clickhouse Version (20240125) [incubator-gluten]
github-actions[bot] closed pull request #4511: [GLUTEN-1632][CH]Daily Update Clickhouse Version (20240125) URL: https://github.com/apache/incubator-gluten/pull/4511 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [GLUTEN-1632][CH]Daily Update Clickhouse Version (20240125) [incubator-gluten]
github-actions[bot] commented on PR #4511: URL: https://github.com/apache/incubator-gluten/pull/4511#issuecomment-2011032819 This PR was auto-closed because it has been stalled for 10 days with no activity. Please feel free to reopen if it is still valid. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [VL] Add uniffle integration [incubator-gluten]
github-actions[bot] commented on PR #3767: URL: https://github.com/apache/incubator-gluten/pull/3767#issuecomment-2011029672 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [DNM] Velox test [incubator-gluten]
github-actions[bot] commented on PR #4929: URL: https://github.com/apache/incubator-gluten/pull/4929#issuecomment-2011023998 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [GLUTEN-1632][CH]Daily Update Clickhouse Version (20240320) [incubator-gluten]
baibaichen closed pull request #5040: [GLUTEN-1632][CH]Daily Update Clickhouse Version (20240320) URL: https://github.com/apache/incubator-gluten/pull/5040 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [GLUTEN-4946][CH] Fix avg(bigint) overflow [incubator-gluten]
loudongfeng commented on PR #5048: URL: https://github.com/apache/incubator-gluten/pull/5048#issuecomment-2011022082 CI failures unrelated. Ready for review. @zzcclp thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [CORE] Basic runnable version of ACBO (Advanced CBO) [incubator-gluten]
github-actions[bot] commented on PR #5058: URL: https://github.com/apache/incubator-gluten/pull/5058#issuecomment-2011021726 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [CORE] Basic runnable version of ACBO (Advanced CBO) [incubator-gluten]
github-actions[bot] commented on PR #5058: URL: https://github.com/apache/incubator-gluten/pull/5058#issuecomment-2011020568 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [CH][Minor] Fix build due to Clickhouse Refactor [incubator-gluten]
github-actions[bot] commented on PR #5059: URL: https://github.com/apache/incubator-gluten/pull/5059#issuecomment-2011020352 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [CH][Minor] Fix build due to https://github.com/ClickHouse/ClickHouse/pull/61604 [incubator-gluten]
github-actions[bot] commented on PR #5059:
URL:
https://github.com/apache/incubator-gluten/pull/5059#issuecomment-2011020211
Thanks for opening a pull request!
Could you open an issue for this pull request on Github Issues?
https://github.com/apache/incubator-gluten/issues
Then could you also rename ***commit message*** and ***pull request title***
in the following format?
[GLUTEN-${ISSUES_ID}][COMPONENT]feat/fix: ${detailed message}
See also:
* [Other pull requests](https://github.com/apache/incubator-gluten/pulls/)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org
For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [CORE] Pullout pre/post project for generate [incubator-gluten]
liujiayi771 commented on PR #4952: URL: https://github.com/apache/incubator-gluten/pull/4952#issuecomment-2011019082 @marin-ma LGTM. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [GLUTEN-1632][CH]Daily Update Clickhouse Version (20240321) [incubator-gluten]
github-actions[bot] commented on PR #5055: URL: https://github.com/apache/incubator-gluten/pull/5055#issuecomment-2011012709 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [CORE] Basic runnable version of ACBO (Advanced CBO) [incubator-gluten]
github-actions[bot] commented on PR #5058: URL: https://github.com/apache/incubator-gluten/pull/5058#issuecomment-2011012096 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [CORE] Basic runnable version of ACBO (Advanced CBO) [incubator-gluten]
github-actions[bot] commented on PR #5058: URL: https://github.com/apache/incubator-gluten/pull/5058#issuecomment-2011010259 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [CORE] Basic runnable version of ACBO (Advanced CBO) [incubator-gluten]
github-actions[bot] commented on PR #5058:
URL:
https://github.com/apache/incubator-gluten/pull/5058#issuecomment-2011010109
Thanks for opening a pull request!
Could you open an issue for this pull request on Github Issues?
https://github.com/apache/incubator-gluten/issues
Then could you also rename ***commit message*** and ***pull request title***
in the following format?
[GLUTEN-${ISSUES_ID}][COMPONENT]feat/fix: ${detailed message}
See also:
* [Other pull requests](https://github.com/apache/incubator-gluten/pulls/)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org
For additional commands, e-mail: commits-h...@gluten.apache.org
[PR] [CORE] Basic runnable version of ACBO (Advanced CBO) [incubator-gluten]
zhztheplayer opened a new pull request, #5058: URL: https://github.com/apache/incubator-gluten/pull/5058 See proposal https://github.com/apache/incubator-gluten/issues/5057 This is the first runnable version of ACBO with TPCH SF 1.0 and TPCDS 10.0 passed. After this patch, one could set `spark.gluten.sql.advanced.cbo.enabled=true` to enable ACBO. It's by default disabled. Issues: 1. It's now only replacing `TransformPreOverrides()` with a rough cost model to do fallback; 2. It's not tested with CH backend yet. @baibaichen @zzcclp If you would like to evaluate it for CH; Otherwise I could disable it for CH in next patch; 3. It may generates slow plan since some operators like aggregation are not considered in ACBO. The side-effect of this will be amplified in performance test; 4. The first version would only be **runnable**. Further integration works are required to make it ready for production. The following improvements are on the way: 1. Enable group mutation (for ops like AQEShuffleReadExec which could propagate children's convention to parent); 2. Enable pattern based rule-matching; The required facilities of the above were already added but not enabled yet. Will enable and test them in PRs respectively. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [WIP] Pullout pre/post project for generate [incubator-gluten]
ulysses-you commented on PR #4952: URL: https://github.com/apache/incubator-gluten/pull/4952#issuecomment-2011006314 It seems the failed tests are irrelevant, I tried to re-run the failed tests. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] Support approx_percentile [incubator-gluten]
zhztheplayer commented on PR #5007: URL: https://github.com/apache/incubator-gluten/pull/5007#issuecomment-2010990615 Hi @WangGuangxin so the current plan is to merge https://github.com/WangGuangxin/gluten/commit/97db869a56e8d42c0782e4e8218e1937e40682ce to upstream Velox before this ? Am I understanding correctly? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [GLUTEN-5024][VL] Enhance buildbundle-veloxbe.sh to run single step [incubator-gluten]
zhztheplayer commented on PR #5032: URL: https://github.com/apache/incubator-gluten/pull/5032#issuecomment-2010986896 Thanks @xumingming for the enhancement. > can you add one flag like -Dfetch_velox=ON/OFF which can fetch velox or not before build? by default let's set it ON. Yes that would be one of the commen case. I assume there could probably be a `fetch_velox` before > build_velox build_gluten_cpp Though I am not sure if it's trivial to add. If this tricky we can apply this PR and do the further step by step. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [GLUTEN-4830][VL] Support MapType substrait signature [incubator-gluten]
PHILO-HE commented on code in PR #4833:
URL: https://github.com/apache/incubator-gluten/pull/4833#discussion_r1533098874
##
cpp/velox/substrait/VeloxSubstraitSignature.cc:
##
@@ -159,16 +151,43 @@ TypePtr
VeloxSubstraitSignature::fromSubstraitSignature(const std::string& signa
}
}
types.emplace_back(fromSubstraitSignature(typeStr));
- names.emplace_back("");
childrenTypes.erase(0, endPos + delimiter.length());
}
if (childrenTypes.size() > 0 && !startWith(childrenTypes, ">")) {
types.emplace_back(fromSubstraitSignature(childrenTypes));
- names.emplace_back("");
+}
+return types;
+ };
+
+ if (startWith(signature, "dec")) {
+// Decimal type name is in the format of dec.
+auto precisionStart = signature.find_first_of('<');
+auto tokenIndex = signature.find_first_of(',');
+auto scaleEnd = signature.find_first_of('>');
+auto precision = stoi(signature.substr(precisionStart + 1, (tokenIndex -
precisionStart - 1)));
+auto scale = stoi(signature.substr(tokenIndex + 1, (scaleEnd - tokenIndex
- 1)));
+return DECIMAL(precision, scale);
+ }
+
+ if (startWith(signature, "struct")) {
+// Struct type name is in the format of struct.
+auto types = parseNestedTypeSignature(signature);
+std::vector names(types.size());
+for (int i = 0; i < types.size(); i++) {
+ names[i] = "";
}
return std::make_shared(std::move(names), std::move(types));
}
+ if (startWith(signature, "map")) {
+// Map type name is in the format of map.
+auto types = parseNestedTypeSignature(signature);
+if (types.size() != 2) {
+ VELOX_UNSUPPORTED("Substrait type signature conversion to Velox type not
supported for {}.", signature);
+}
+return MAP(std::move(types)[0], std::move(types)[1]);
+ }
+
if (startWith(signature, "list")) {
auto listStart = signature.find_first_of('<');
Review Comment:
@WangGuangxin, I see. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org
For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [DNM][VL] GHA with pre-built docker image and github runner [incubator-gluten]
github-actions[bot] commented on PR #4936: URL: https://github.com/apache/incubator-gluten/pull/4936#issuecomment-2010974653 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [DNM][VL] GHA with pre-built docker image and github runner [incubator-gluten]
github-actions[bot] commented on PR #4936: URL: https://github.com/apache/incubator-gluten/pull/4936#issuecomment-2010964281 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
Re: [PR] [DNM][VL] GHA with pre-built docker image and github runner [incubator-gluten]
github-actions[bot] commented on PR #4936: URL: https://github.com/apache/incubator-gluten/pull/4936#issuecomment-2010956869 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
(incubator-gluten) branch main updated: [GLUTEN-4933][VL] Update iceberg version to 1.4.3 for Spark 3.4 and above (#4967)
This is an automated email from the ASF dual-hosted git repository. yuanzhou pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git The following commit(s) were added to refs/heads/main by this push: new 0f5716f91 [GLUTEN-4933][VL] Update iceberg version to 1.4.3 for Spark 3.4 and above (#4967) 0f5716f91 is described below commit 0f5716f918a18be2ce69c1f84cc9b3e87e843caa Author: Yan Ma AuthorDate: Thu Mar 21 08:19:35 2024 +0800 [GLUTEN-4933][VL] Update iceberg version to 1.4.3 for Spark 3.4 and above (#4967) This patch made a clean up on the iceberg used for different Spark shims Spark 3.2 : v1.3.1 Spark 3.3 : v1.3.1 Spark 3.4 : v1.4.3 Spark 3.5 : v1.4.3 Existing unit tests on spark w/ iceberg passed --- pom.xml | 18 +++--- 1 file changed, 11 insertions(+), 7 deletions(-) diff --git a/pom.xml b/pom.xml index 54c9fd093..6bdbf678f 100644 --- a/pom.xml +++ b/pom.xml @@ -39,7 +39,6 @@ 2.9.3 2.0.1 -1.3.1 20 2.12 2.12.15 @@ -113,7 +112,8 @@ 3.2 spark-sql-columnar-shims-spark32 -3.2.2 + 3.2.2 + 1.3.1 2.0.1 20 @@ -123,8 +123,10 @@ 3.3 spark-sql-columnar-shims-spark33 -3.3.1 -2.2.0 + 3.3.1 + + 1.3.1 + 2.2.0 22 @@ -133,8 +135,9 @@ 3.4 spark-sql-columnar-shims-spark34 -3.4.2 -2.4.0 + 3.4.2 + 1.4.3 + 2.4.0 24 @@ -143,7 +146,8 @@ 3.5 spark-sql-columnar-shims-spark35 -3.5.1 + 3.5.1 + 1.4.3 2.4.0 24 - To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org For additional commands, e-mail: commits-h...@gluten.apache.org
