jayzhan211 commented on code in PR #10560:
URL: https://github.com/apache/datafusion/pull/10560#discussion_r1605955848
##
docs/source/user-guide/expressions.md:
##
@@ -304,6 +304,16 @@ select log(-1), log(0), sqrt(-1);
| rollup(exprs)
jayzhan211 merged PR #10569:
URL: https://github.com/apache/datafusion/pull/10569
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@dat
jayzhan211 closed issue #10566: Improve signature of `get_field` is function
URL: https://github.com/apache/datafusion/issues/10566
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific commen
jayzhan211 opened a new pull request, #10574:
URL: https://github.com/apache/datafusion/pull/10574
## Which issue does this PR close?
Closes #.
## Rationale for this change
1. add ahash for common, used for distinct count accumulator #10484
2. move other g
tisonkun commented on code in PR #10392:
URL: https://github.com/apache/datafusion/pull/10392#discussion_r1605950326
##
datafusion/sqllogictest/test_files/array.slt:
##
Review Comment:
Can be a bug after the JSON path parse changes.
--
This is an automated message from
viirya merged PR #447:
URL: https://github.com/apache/datafusion-comet/pull/447
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@dataf
backkem commented on issue #10557:
URL: https://github.com/apache/datafusion/issues/10557#issuecomment-2119108342
Yes, these are basically the same object. The one in DataFusion was put
there temporarily until the trait extension in the sqlparser repo is landed and
pushed to crates.io.
--
viirya commented on PR #447:
URL: https://github.com/apache/datafusion-comet/pull/447#issuecomment-2119108275
Merged. Thanks @sunchao
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
viirya closed issue #448: CometNativeExec.doCanonicalize should canonicalize
SparkPlan in Product parameters
URL: https://github.com/apache/datafusion-comet/issues/448
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
U
tisonkun commented on code in PR #10392:
URL: https://github.com/apache/datafusion/pull/10392#discussion_r1605947314
##
datafusion/sqllogictest/test_files/array.slt:
##
Review Comment:
New failure:
```
Running "array.slt"
External error: query failed: DataFusio
tisonkun commented on code in PR #10392:
URL: https://github.com/apache/datafusion/pull/10392#discussion_r1605946538
##
datafusion/sqllogictest/test_files/array.slt:
##
@@ -689,7 +689,7 @@ select column1, column2, column3, column4, column5 from
nested_arrays;
# values table
goldmedal commented on issue #10557:
URL: https://github.com/apache/datafusion/issues/10557#issuecomment-2119066019
As the mentioned in `dialect.rs`
https://github.com/apache/datafusion/blob/e7858ff0ab1c282ab46bd93cabc3dc83db583165/datafusion/sql/src/unparser/dialect.rs#L19
I think
goldmedal opened a new pull request, #10573:
URL: https://github.com/apache/datafusion/pull/10573
## Which issue does this PR close?
Closes #10557
## Rationale for this change
## What changes are included in this PR?
Only implement the default dialect in this PR.
timsaucer opened a new pull request, #709:
URL: https://github.com/apache/datafusion-python/pull/709
# Which issue does this PR close?
This PR does not close an issue, but it aims to address part of the
discussion in https://github.com/apache/datafusion-python/issues/440 . This
takes
jayzhan211 commented on issue #10102:
URL: https://github.com/apache/datafusion/issues/10102#issuecomment-2119060296
I didn't find equivalent behavior in postgres. I'm not sure should we
support this kind of `returns subset of columns based on column name matching`
--
This is an automated
github-actions[bot] closed pull request #6047: Improve round-robin
repartitioning
URL: https://github.com/apache/datafusion/pull/6047
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific com
jayzhan211 commented on issue #10200:
URL: https://github.com/apache/datafusion/issues/10200#issuecomment-2119054943
Actually, I'm thinking about whether we should change the behavior of
array_concat similar to postgres and duckdb.
It is one of the earliest array functions that we don't f
huaxingao commented on code in PR #395:
URL: https://github.com/apache/datafusion-comet/pull/395#discussion_r1605917689
##
common/src/main/java/org/apache/comet/parquet/CometParquetToSparkSchemaConverter.scala:
##
@@ -0,0 +1,403 @@
+/*
+ * Licensed to the Apache Software Foundat
shanretoo commented on issue #6747:
URL: https://github.com/apache/datafusion/issues/6747#issuecomment-2119039796
Thanks for your update! I'll work on the tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL
viirya commented on code in PR #447:
URL: https://github.com/apache/datafusion-comet/pull/447#discussion_r1605914565
##
spark/src/test/resources/tpcds-plan-stability/approved-plans-v2_7/q5a/explain.txt:
##
@@ -72,70 +72,16 @@ TakeOrderedAndProject (137)
:
tshauck opened a new pull request, #449:
URL: https://github.com/apache/datafusion-comet/pull/449
## Which issue does this PR close?
Related to https://github.com/apache/datafusion-comet/issues/341.
## Rationale for this change
I recently added `unhex` so this PR adds `he
viirya opened a new issue, #448:
URL: https://github.com/apache/datafusion-comet/issues/448
### Describe the bug
`SparkPlan.doCanonicalize` default implementation canonicalizes expressions
in Product parameters, but not for `SparkPlan` because derived classes in Spark
doesn't have su
viirya opened a new pull request, #447:
URL: https://github.com/apache/datafusion-comet/pull/447
## Which issue does this PR close?
Closes #.
## Rationale for this change
## What changes are included in this PR?
## How are these changes test
tshauck commented on code in PR #422:
URL: https://github.com/apache/datafusion-comet/pull/422#discussion_r1605893848
##
docs/source/contributor-guide/adding_a_new_expression.md:
##
@@ -0,0 +1,212 @@
+
+
+# Adding a Expression
+
+There are a number of Spark expression that are n
tshauck commented on code in PR #422:
URL: https://github.com/apache/datafusion-comet/pull/422#discussion_r1605893537
##
docs/source/contributor-guide/adding_a_new_expression.md:
##
@@ -0,0 +1,212 @@
+
+
+# Adding a Expression
+
+There are a number of Spark expression that are n
dependabot[bot] opened a new pull request, #708:
URL: https://github.com/apache/datafusion-python/pull/708
Bumps [prost-types](https://github.com/tokio-rs/prost) from 0.12.3 to 0.12.6.
Commits
https://github.com/tokio-rs/prost/commit/d42c85e790263f78f6c626ceb0dac5fda0edcb41";>d4
dependabot[bot] opened a new pull request, #707:
URL: https://github.com/apache/datafusion-python/pull/707
Bumps [object_store](https://github.com/apache/arrow-rs) from 0.9.1 to
0.10.1.
Changelog
Sourced from https://github.com/apache/arrow-rs/blob/master/CHANGELOG-old.md";>object_
dependabot[bot] opened a new pull request, #706:
URL: https://github.com/apache/datafusion-python/pull/706
Bumps [syn](https://github.com/dtolnay/syn) from 2.0.63 to 2.0.64.
Release notes
Sourced from https://github.com/dtolnay/syn/releases";>syn's
releases.
2.0.64
Su
dependabot[bot] opened a new pull request, #705:
URL: https://github.com/apache/datafusion-python/pull/705
Bumps [prost](https://github.com/tokio-rs/prost) from 0.12.4 to 0.12.6.
Commits
https://github.com/tokio-rs/prost/commit/d42c85e790263f78f6c626ceb0dac5fda0edcb41";>d42c85e
codecov-commenter commented on PR #445:
URL: https://github.com/apache/datafusion-comet/pull/445#issuecomment-2118934008
##
[Codecov](https://app.codecov.io/gh/apache/datafusion-comet/pull/445?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campai
codecov-commenter commented on PR #437:
URL: https://github.com/apache/datafusion-comet/pull/437#issuecomment-2118931077
##
[Codecov](https://app.codecov.io/gh/apache/datafusion-comet/pull/437?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campai
twitu opened a new issue, #10572:
URL: https://github.com/apache/datafusion/issues/10572
### Describe the bug
Datafusion is reading row groups out of order and sometimes with completely
different values for the row groups. The data is verified by reading the same
files using the Pyth
comphead commented on code in PR #10561:
URL: https://github.com/apache/datafusion/pull/10561#discussion_r1605851167
##
datafusion/functions/Cargo.toml:
##
@@ -74,7 +74,7 @@ datafusion-common = { workspace = true }
datafusion-execution = { workspace = true }
datafusion-expr =
andygrove commented on code in PR #445:
URL: https://github.com/apache/datafusion-comet/pull/445#discussion_r1605843645
##
spark/src/test/scala/org/apache/spark/sql/CometTestBase.scala:
##
@@ -261,7 +261,10 @@ abstract class CometTestBase
}
val extendedInfo =
ne
viirya commented on PR #412:
URL: https://github.com/apache/datafusion-comet/pull/412#issuecomment-2118898041
Merged. Thanks @ceppelli @kazuyukitanimura @andygrove
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
UR
viirya closed issue #411: compatibility issue with AWS EMR 6.15.0 SPARK 3.4.1
URL: https://github.com/apache/datafusion-comet/issues/411
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific c
viirya merged PR #412:
URL: https://github.com/apache/datafusion-comet/pull/412
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@dataf
codecov-commenter commented on PR #412:
URL: https://github.com/apache/datafusion-comet/pull/412#issuecomment-2118876243
##
[Codecov](https://app.codecov.io/gh/apache/datafusion-comet/pull/412?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campai
goldmedal commented on issue #10557:
URL: https://github.com/apache/datafusion/issues/10557#issuecomment-2118875053
>
https://github.com/sqlparser-rs/sqlparser-rs/blob/54184460b5d873a67c2801e8b7c6e4f145bc65df/src/dialect/mod.rs#L113-L116
>
> The dialect specific implementations just n
viirya commented on PR #441:
URL: https://github.com/apache/datafusion-comet/pull/441#issuecomment-2118872698
Merged. Thanks @kazuyukitanimura @sunchao
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to
viirya merged PR #441:
URL: https://github.com/apache/datafusion-comet/pull/441
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@dataf
viirya closed issue #439: `CometBroadcastExchangeExec` cannot be reused by
Spark `ReuseExchangeAndSubquery` rule
URL: https://github.com/apache/datafusion-comet/issues/439
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use t
viirya commented on code in PR #1:
URL:
https://github.com/apache/datafusion-benchmarks/pull/1#discussion_r1605813002
##
tpch/queries/q15.sql:
##
@@ -0,0 +1,33 @@
+-- SQLBench-H query 15 derived from TPC-H query 15 under the terms of the TPC
Fair Use Policy.
+-- TPC-H queries
andygrove opened a new pull request, #2:
URL: https://github.com/apache/datafusion-benchmarks/pull/2
(no comment)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubsc
viirya commented on code in PR #1:
URL:
https://github.com/apache/datafusion-benchmarks/pull/1#discussion_r1605813002
##
tpch/queries/q15.sql:
##
@@ -0,0 +1,33 @@
+-- SQLBench-H query 15 derived from TPC-H query 15 under the terms of the TPC
Fair Use Policy.
+-- TPC-H queries
viirya commented on PR #412:
URL: https://github.com/apache/datafusion-comet/pull/412#issuecomment-2118865381
I take the liberty to commit some suggestions on code comment and style as
it is not responded for days. I will merge this once CI passes.
--
This is an automated message from the
timsaucer commented on issue #6747:
URL: https://github.com/apache/datafusion/issues/6747#issuecomment-2118864412
Great! I've rebased @alamb 's branch and added the changes I suggested. I
was about to start testing the code and then I was going to write up the unit
tests. My work in progres
viirya commented on code in PR #441:
URL: https://github.com/apache/datafusion-comet/pull/441#discussion_r1605809454
##
spark/src/main/scala/org/apache/comet/CometSparkSessionExtensions.scala:
##
@@ -576,11 +576,13 @@ class CometSparkSessionExtensions
// exchange. It is
MohamedAbdeen21 commented on issue #9294:
URL: https://github.com/apache/datafusion/issues/9294#issuecomment-2118859061
Hey @l1t1, as per Andy's comments on #9452, datafusion-cli releases should
be handled in the python repo.
--
This is an automated message from the Apache Git Service.
To
shanretoo commented on issue #6747:
URL: https://github.com/apache/datafusion/issues/6747#issuecomment-2118840627
I am willing to help with this task.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go
xinlifoobar commented on issue #10314:
URL: https://github.com/apache/datafusion/issues/10314#issuecomment-2118825197
Hi @alamb, I am trying to work on this.
I am not very familiar on the `InterleaveExec` in the optimizer. As initial
thought, the interleaveExec is acting as a **Repart
LorrensP-2158466 opened a new issue, #10571:
URL: https://github.com/apache/datafusion/issues/10571
### Is your feature request related to a problem or challenge?
This is really a feature request but more of a question.
Currently `UserDefinedLogicalNode::from_template` only retu
LorrensP-2158466 closed issue #10570: UserDefinedLogicalNode::from_template
does not return a Result<...> >
URL: https://github.com/apache/datafusion/issues/10570
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL ab
LorrensP-2158466 opened a new issue, #10570:
URL: https://github.com/apache/datafusion/issues/10570
### Is your feature request related to a problem or challenge?
_No response_
### Describe the solution you'd like
_No response_
### Describe alternatives you've cons
andygrove commented on code in PR #331:
URL: https://github.com/apache/datafusion-comet/pull/331#discussion_r1605784115
##
spark/src/test/scala/org/apache/comet/CometExpressionCoverageSuite.scala:
##
@@ -123,7 +134,7 @@ class CometExpressionCoverageSuite extends CometTestBase
w
advancedxy commented on code in PR #442:
URL: https://github.com/apache/datafusion-comet/pull/442#discussion_r1605766340
##
spark/src/main/scala/org/apache/spark/sql/comet/operators.scala:
##
@@ -899,6 +899,40 @@ case class CometSortMergeJoinExec(
"join_time" -> SQLMetric
backkem commented on issue #10557:
URL: https://github.com/apache/datafusion/issues/10557#issuecomment-2118742184
Indeed, there is already a function on the sqlparser::dialect trait that
takes this into account:
https://github.com/sqlparser-rs/sqlparser-rs/blob/54184460b5d873a67c2801
goldmedal commented on issue #10557:
URL: https://github.com/apache/datafusion/issues/10557#issuecomment-2118721347
Provide something I surveyed.
I think we can follow how Calcite handles the quoted issue. The `SqlDialect`
of Calcite has a check rule `identifierNeedsQuote`.
ht
caicancai commented on issue #375:
URL:
https://github.com/apache/datafusion-comet/issues/375#issuecomment-2118707359
I am working on it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the spec
xinlifoobar commented on code in PR #10555:
URL: https://github.com/apache/datafusion/pull/10555#discussion_r1605709554
##
datafusion/sql/src/unparser/expr.rs:
##
@@ -411,9 +411,34 @@ impl Unparser<'_> {
Expr::Wildcard { qualifier: _ } => {
not_impl
60 matches
Mail list logo
