sunchao commented on a change in pull request #9612:
URL: https://github.com/apache/arrow/pull/9612#discussion_r594048206
##
File path: rust/parquet/src/arrow/schema.rs
##
@@ -364,32 +385,51 @@ fn arrow_to_parquet_type(field: &Field) -> Result {
DataType::Float64 => Ty
liyafan82 commented on issue #9636:
URL: https://github.com/apache/arrow/issues/9636#issuecomment-799069748
The float8 data in the list vector and double array are both in a continuous
memory region.
So in theory, it is possible to populate the data in the `double[]` in a
single copy.
cyb70289 commented on pull request #9706:
URL: https://github.com/apache/arrow/pull/9706#issuecomment-799050981
Please note **this patch is not tested**. I only checked the added lines
manually that there is no syntax error.
github-actions[bot] commented on pull request #9706:
URL: https://github.com/apache/arrow/pull/9706#issuecomment-799049802
https://issues.apache.org/jira/browse/ARROW-11941
This is an automated message from the Apache Git Ser
github-actions[bot] commented on pull request #9706:
URL: https://github.com/apache/arrow/pull/9706#issuecomment-799049571
Thanks for opening a pull request!
Could you open an issue for this pull request on JIRA?
https://issues.apache.org/jira/browse/ARROW
Then could
cyb70289 opened a new pull request #9706:
URL: https://github.com/apache/arrow/pull/9706
When environment variable DEBUG=1, merge_arrow_pr.py should only try
the merge steps without updating anything. Though PR status is not
changed, Jira issue is updated even if DEBUG=1. This patch fi
houqp commented on pull request #9692:
URL: https://github.com/apache/arrow/pull/9692#issuecomment-799038552
@jorgecarleitao yeah, you are right, sorry I got it mixed up with `dyn` :P
I think `IntoIterator` won't require copy or clone because the
iterator owns those values. `AsRef` w
nevi-me commented on pull request #9705:
URL: https://github.com/apache/arrow/pull/9705#issuecomment-799030871
@sunchao I've created this on top of #9612, PTAL when you can.
This is an automated message from the Apache Git Se
github-actions[bot] commented on pull request #9705:
URL: https://github.com/apache/arrow/pull/9705#issuecomment-799030723
https://issues.apache.org/jira/browse/ARROW-11365
This is an automated message from the Apache Git Ser
nevi-me opened a new pull request #9705:
URL: https://github.com/apache/arrow/pull/9705
This implements the parser and printer for logical types, allowing us to
read and generate the schema in the form `REQUIRED INT32 field_name
(INTEGER(16,false))`.
-
nevi-me commented on a change in pull request #9612:
URL: https://github.com/apache/arrow/pull/9612#discussion_r593995897
##
File path: rust/parquet/src/arrow/schema.rs
##
@@ -583,48 +626,109 @@ impl ParquetTypeConverter<'_> {
}
fn from_int32(&self) -> Result {
-
nevi-me commented on a change in pull request #9612:
URL: https://github.com/apache/arrow/pull/9612#discussion_r593995672
##
File path: rust/parquet/src/arrow/schema.rs
##
@@ -364,32 +385,51 @@ fn arrow_to_parquet_type(field: &Field) -> Result {
DataType::Float64 => Ty
Dandandan commented on pull request #9704:
URL: https://github.com/apache/arrow/pull/9704#issuecomment-798985687
> @Dandandan I have mixed feelings about this change - I get the performance
improvement side, but in my opinion it makes the explain output less readable
by replacing the strin
yordan-pavlov commented on a change in pull request #9704:
URL: https://github.com/apache/arrow/pull/9704#discussion_r593965350
##
File path: rust/datafusion/src/physical_plan/parquet.rs
##
@@ -645,8 +654,29 @@ fn build_predicate_expression(
use crate::logical_plan;
/
yordan-pavlov commented on a change in pull request #9704:
URL: https://github.com/apache/arrow/pull/9704#discussion_r593965090
##
File path: rust/datafusion/src/physical_plan/parquet.rs
##
@@ -645,8 +654,29 @@ fn build_predicate_expression(
use crate::logical_plan;
/
yordan-pavlov commented on pull request #9704:
URL: https://github.com/apache/arrow/pull/9704#issuecomment-798984352
@Dandandan I have mixed feelings about this change - I get the performance
improvement side, but in my opinion it makes the explain output less readable
by replacing the st
Dandandan commented on a change in pull request #9704:
URL: https://github.com/apache/arrow/pull/9704#discussion_r593964624
##
File path: rust/datafusion/src/physical_plan/parquet.rs
##
@@ -645,8 +654,29 @@ fn build_predicate_expression(
use crate::logical_plan;
// pr
Dandandan commented on a change in pull request #9704:
URL: https://github.com/apache/arrow/pull/9704#discussion_r593962933
##
File path: rust/datafusion/src/physical_plan/parquet.rs
##
@@ -645,8 +654,29 @@ fn build_predicate_expression(
use crate::logical_plan;
// pr
yordan-pavlov commented on a change in pull request #9704:
URL: https://github.com/apache/arrow/pull/9704#discussion_r593961778
##
File path: rust/datafusion/src/physical_plan/parquet.rs
##
@@ -645,8 +654,29 @@ fn build_predicate_expression(
use crate::logical_plan;
/
jorgecarleitao edited a comment on pull request #9692:
URL: https://github.com/apache/arrow/pull/9692#issuecomment-798977003
Another option is to use `T: AsRef<[Expr]>`, which works for any type that
can be represented as a reference to an array (e.g. `vec` and &[]`)
@houqp , AFAIK `
jorgecarleitao commented on pull request #9692:
URL: https://github.com/apache/arrow/pull/9692#issuecomment-798977003
Another option is to use `T: AsRef<[Expr]>`, which works for any type that
can be represented as a reference to an array (e.g. `vec` and &[]`)
@houqp , AFAIK `impl` i
houqp commented on pull request #9692:
URL: https://github.com/apache/arrow/pull/9692#issuecomment-798975788
Yes, I think if the function only requires sequential access, iterator would
have been even better. Although I would prefer to write it as a generic
function rather than using Trait
kou commented on pull request #8717:
URL: https://github.com/apache/arrow/pull/8717#issuecomment-798973718
I close this because this isn't active in 4 months.
This is an automated message from the Apache Git Service.
To r
kou closed pull request #8717:
URL: https://github.com/apache/arrow/pull/8717
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the sp
kou closed pull request #9687:
URL: https://github.com/apache/arrow/pull/9687
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the sp
kou commented on pull request #9687:
URL: https://github.com/apache/arrow/pull/9687#issuecomment-798972719
+1
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and us
mathyingzhou commented on pull request #8648:
URL: https://github.com/apache/arrow/pull/8648#issuecomment-798971411
@pitrou Could you please check again? My next PR is going to be ready soon
and it is dependent on this one merging to be clean.
-
houqp commented on a change in pull request #9701:
URL: https://github.com/apache/arrow/pull/9701#discussion_r593941886
##
File path: rust/datafusion/README.md
##
@@ -19,11 +19,48 @@
# DataFusion
-DataFusion is an in-memory query engine that uses Apache Arrow as the memory
andygrove commented on a change in pull request #9695:
URL: https://github.com/apache/arrow/pull/9695#discussion_r593934133
##
File path: rust/datafusion/src/physical_plan/union.rs
##
@@ -0,0 +1,76 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more co
Dandandan removed a comment on pull request #9704:
URL: https://github.com/apache/arrow/pull/9704#issuecomment-798943827
@yordan-pavlov I am trying to use the filtering in parquet - however on any
query the row groups being filtered seems to be 0. Do you know what could be
happening here?
Dandandan commented on pull request #9704:
URL: https://github.com/apache/arrow/pull/9704#issuecomment-798943827
@yordan-pavlov I am trying to use the filtering in parquet - however on any
query the row groups being filtered seems to be 0. Do you know what could be
happening here?
--
Dandandan opened a new pull request #9704:
URL: https://github.com/apache/arrow/pull/9704
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to
github-actions[bot] commented on pull request #9704:
URL: https://github.com/apache/arrow/pull/9704#issuecomment-798937300
https://issues.apache.org/jira/browse/ARROW-11964
This is an automated message from the Apache Git Ser
returnString commented on pull request #9703:
URL: https://github.com/apache/arrow/pull/9703#issuecomment-798918286
> I couldn't figure out how to allow the Dataframe API (which is a Trait) to
take an impl Iterator
Yeah, it's a pain but `impl trait` (in both return and argument posit
Dandandan commented on a change in pull request #9602:
URL: https://github.com/apache/arrow/pull/9602#discussion_r593904547
##
File path: rust/arrow/src/compute/kernels/sort.rs
##
@@ -686,90 +815,124 @@ pub fn lexsort_to_indices(columns: &[SortColumn]) ->
Result {
};
Dandandan commented on a change in pull request #9602:
URL: https://github.com/apache/arrow/pull/9602#discussion_r593904547
##
File path: rust/arrow/src/compute/kernels/sort.rs
##
@@ -686,90 +815,124 @@ pub fn lexsort_to_indices(columns: &[SortColumn]) ->
Result {
};
Dandandan commented on a change in pull request #9701:
URL: https://github.com/apache/arrow/pull/9701#discussion_r593902991
##
File path: rust/datafusion/README.md
##
@@ -19,11 +19,48 @@
# DataFusion
-DataFusion is an in-memory query engine that uses Apache Arrow as the me
alamb commented on a change in pull request #9703:
URL: https://github.com/apache/arrow/pull/9703#discussion_r593902253
##
File path: rust/datafusion/src/logical_plan/builder.rs
##
@@ -220,20 +260,28 @@ impl LogicalPlanBuilder {
}))
}
-/// Apply an aggregate
alamb commented on a change in pull request #9703:
URL: https://github.com/apache/arrow/pull/9703#discussion_r593901579
##
File path: rust/datafusion/src/logical_plan/builder.rs
##
@@ -39,6 +39,43 @@ use crate::logical_plan::{DFField, DFSchema, DFSchemaRef,
Partitioning};
use
alamb commented on a change in pull request #9703:
URL: https://github.com/apache/arrow/pull/9703#discussion_r593901365
##
File path: rust/benchmarks/src/bin/tpch.rs
##
@@ -1636,7 +1636,7 @@ mod tests {
.file_extension(".out");
let df = ctx.read_cs
github-actions[bot] commented on pull request #9703:
URL: https://github.com/apache/arrow/pull/9703#issuecomment-798907390
https://issues.apache.org/jira/browse/ARROW-11790
This is an automated message from the Apache Git Ser
alamb opened a new pull request #9703:
URL: https://github.com/apache/arrow/pull/9703
# NOTE:
Since is a fairly major backwards incompatible change (many callsites need
to be updated, though mostly mechanically); I gathered some feedback on this
approach in https://github.com/apache/arr
sundy-li commented on a change in pull request #9602:
URL: https://github.com/apache/arrow/pull/9602#discussion_r593899532
##
File path: rust/arrow/src/compute/kernels/sort.rs
##
@@ -686,90 +815,124 @@ pub fn lexsort_to_indices(columns: &[SortColumn]) ->
Result {
};
sundy-li commented on a change in pull request #9602:
URL: https://github.com/apache/arrow/pull/9602#discussion_r593899532
##
File path: rust/arrow/src/compute/kernels/sort.rs
##
@@ -686,90 +815,124 @@ pub fn lexsort_to_indices(columns: &[SortColumn]) ->
Result {
};
sundy-li commented on a change in pull request #9602:
URL: https://github.com/apache/arrow/pull/9602#discussion_r593899532
##
File path: rust/arrow/src/compute/kernels/sort.rs
##
@@ -686,90 +815,124 @@ pub fn lexsort_to_indices(columns: &[SortColumn]) ->
Result {
};
github-actions[bot] commented on pull request #9702:
URL: https://github.com/apache/arrow/pull/9702#issuecomment-798904875
https://issues.apache.org/jira/browse/ARROW-11297
This is an automated message from the Apache Git Ser
mathyingzhou opened a new pull request #9702:
URL: https://github.com/apache/arrow/pull/9702
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above
alamb commented on pull request #9692:
URL: https://github.com/apache/arrow/pull/9692#issuecomment-798900651
@Dandandan and @houqp
What do you think about something like this (which is even more Idiomatic
(TM) I think), though harder for beginner Rust programmers to grok?
`
alamb commented on a change in pull request #9701:
URL: https://github.com/apache/arrow/pull/9701#discussion_r593891961
##
File path: rust/datafusion/README.md
##
@@ -19,11 +19,48 @@
# DataFusion
-DataFusion is an in-memory query engine that uses Apache Arrow as the memory
alamb commented on a change in pull request #9701:
URL: https://github.com/apache/arrow/pull/9701#discussion_r593891533
##
File path: rust/datafusion/README.md
##
@@ -19,11 +19,48 @@
# DataFusion
-DataFusion is an in-memory query engine that uses Apache Arrow as the memory
alamb commented on a change in pull request #9701:
URL: https://github.com/apache/arrow/pull/9701#discussion_r593891472
##
File path: rust/datafusion/DEVELOPERS.md
##
@@ -0,0 +1,79 @@
+# Developer's guide
+
+This section describes how you can get started at developing DataFusio
alamb commented on a change in pull request #9701:
URL: https://github.com/apache/arrow/pull/9701#discussion_r593891392
##
File path: rust/datafusion/DEVELOPERS.md
##
@@ -0,0 +1,79 @@
+# Developer's guide
Review comment:
I pulled this file into its own separate file so
github-actions[bot] commented on pull request #9701:
URL: https://github.com/apache/arrow/pull/9701#issuecomment-798897258
https://issues.apache.org/jira/browse/ARROW-11962
This is an automated message from the Apache Git Ser
alamb opened a new pull request #9701:
URL: https://github.com/apache/arrow/pull/9701
# Rationale
1. It would be nice to market / explain DataFusion a bit more and explain
what it is good for
# Changes
1. Describe usecases for DataFusion (Add some marketing "spin"??)
2. Add
github-actions[bot] commented on pull request #9700:
URL: https://github.com/apache/arrow/pull/9700#issuecomment-798895560
https://issues.apache.org/jira/browse/ARROW-11960
This is an automated message from the Apache Git Ser
Crystrix opened a new pull request #9700:
URL: https://github.com/apache/arrow/pull/9700
Add gdv_fn_like_utf8_utf8_int8 function in Gandiva to support escape char in
LIKE. An escape char is stored in an int8 type which is compatible with char
type in C++.
alamb commented on a change in pull request #9645:
URL: https://github.com/apache/arrow/pull/9645#discussion_r593885249
##
File path: rust/datafusion/examples/flight_server.rs
##
@@ -94,28 +94,22 @@ impl FlightService for FlightServiceImpl {
"alltypes_plain
Dandandan commented on a change in pull request #9602:
URL: https://github.com/apache/arrow/pull/9602#discussion_r593885246
##
File path: rust/arrow/src/compute/kernels/sort.rs
##
@@ -686,90 +815,124 @@ pub fn lexsort_to_indices(columns: &[SortColumn]) ->
Result {
};
codecov-io edited a comment on pull request #9695:
URL: https://github.com/apache/arrow/pull/9695#issuecomment-798734537
# [Codecov](https://codecov.io/gh/apache/arrow/pull/9695?src=pr&el=h1) Report
> Merging
[#9695](https://codecov.io/gh/apache/arrow/pull/9695?src=pr&el=desc) (ac89542)
Dandandan commented on a change in pull request #9645:
URL: https://github.com/apache/arrow/pull/9645#discussion_r593884833
##
File path: rust/datafusion/examples/flight_server.rs
##
@@ -94,28 +94,26 @@ impl FlightService for FlightServiceImpl {
"alltypes_p
alamb closed pull request #9639:
URL: https://github.com/apache/arrow/pull/9639
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the
alamb commented on a change in pull request #9639:
URL: https://github.com/apache/arrow/pull/9639#discussion_r593883317
##
File path: rust/datafusion/src/execution/context.rs
##
@@ -1702,6 +1702,23 @@ mod tests {
}
Ok(())
}
+#[test]
+fn ctx_sql_sh
alamb commented on pull request #9691:
URL: https://github.com/apache/arrow/pull/9691#issuecomment-79368
I am not sure how much of an improvement this is and I would be happy to
simply close the PR, but I didn't want to leave it unresolved outstanding
-
Dandandan commented on a change in pull request #9695:
URL: https://github.com/apache/arrow/pull/9695#discussion_r593883069
##
File path: rust/datafusion/src/physical_plan/union.rs
##
@@ -0,0 +1,76 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more co
alamb closed pull request #9699:
URL: https://github.com/apache/arrow/pull/9699
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the
alamb commented on pull request #9600:
URL: https://github.com/apache/arrow/pull/9600#issuecomment-798885671
> The above is the primary reason why I did not introduce this idea before;
it will lead to either no one changing that parameter once the system is running
Yes -- this is wha
alamb closed pull request #9690:
URL: https://github.com/apache/arrow/pull/9690
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the
nevi-me commented on a change in pull request #9232:
URL: https://github.com/apache/arrow/pull/9232#discussion_r593875606
##
File path: rust/arrow/src/compute/kernels/cast.rs
##
@@ -1511,6 +1641,65 @@ mod tests {
assert!(9.0 - c.value(4) < f64::EPSILON);
}
+
68 matches
Mail list logo