Dandandan commented on pull request #9038:
URL: https://github.com/apache/arrow/pull/9038#issuecomment-752382480
1. Yes I think there should be a different/more efficient implementation
that handles the "scalar" case, where the scalar in this case is the list with
values.
2. I believe t
jorgecarleitao opened a new pull request #9044:
URL: https://github.com/apache/arrow/pull/9044
This PR addresses a performance issue in how we allocate and reallocate the
`MutableBuffer`.
# Problem
See #9032
# This PR
This PR changes `MutableBuffer::reserve` to c
mqy commented on a change in pull request #9025:
URL: https://github.com/apache/arrow/pull/9025#discussion_r55043
##
File path: rust/arrow/src/datatypes.rs
##
@@ -1594,7 +1620,7 @@ impl Field {
impl fmt::Display for Field {
fn fmt(&self, f: &mut fmt::Formatter) -> f
mqy commented on a change in pull request #9025:
URL: https://github.com/apache/arrow/pull/9025#discussion_r55043
##
File path: rust/arrow/src/datatypes.rs
##
@@ -1594,7 +1620,7 @@ impl Field {
impl fmt::Display for Field {
fn fmt(&self, f: &mut fmt::Formatter) -> f
github-actions[bot] commented on pull request #9044:
URL: https://github.com/apache/arrow/pull/9044#issuecomment-752403701
Thanks for opening a pull request!
Could you open an issue for this pull request on JIRA?
https://issues.apache.org/jira/browse/ARROW
Then could
mqy commented on a change in pull request #9025:
URL: https://github.com/apache/arrow/pull/9025#discussion_r550113231
##
File path: rust/arrow/src/util/integration_util.rs
##
@@ -60,13 +60,22 @@ pub struct ArrowJsonField {
impl From<&Field> for ArrowJsonField {
fn from(
mqy commented on a change in pull request #9025:
URL: https://github.com/apache/arrow/pull/9025#discussion_r550113422
##
File path: rust/arrow/src/datatypes.rs
##
@@ -1903,9 +1929,20 @@ mod tests {
#[test]
fn serde_struct_type() {
+let kv_array = [("k".to_st
mqy commented on pull request #9041:
URL: https://github.com/apache/arrow/pull/9041#issuecomment-752413596
@nevi-me format in this way should avoid the problem
I have fixed it in https://github.com/apache/arrow/pull/9025
Also I'm thinking about add `cargo +nightly-2020-11-24-x8
mqy commented on a change in pull request #9011:
URL: https://github.com/apache/arrow/pull/9011#discussion_r550138436
##
File path: rust/arrow/src/ipc/writer.rs
##
@@ -355,41 +381,83 @@ pub struct FileWriter {
impl FileWriter {
/// Try create a new writer, with the sche
mqy commented on pull request #9011:
URL: https://github.com/apache/arrow/pull/9011#issuecomment-752415900
> Hey @mqy, I'm on vacation, so I haven't been checking the project much.
>
> [ARROW-10299](https://issues.apache.org/jira/browse/ARROW-10299) is for
implementing ipc::MetadataV
mqy commented on a change in pull request #9025:
URL: https://github.com/apache/arrow/pull/9025#discussion_r549939736
##
File path: rust/arrow/src/datatypes.rs
##
@@ -22,6 +22,7 @@
//! * [`Field`](crate::datatypes::Field) to describe one field within a
schema.
//! * [`Data
Dandandan commented on pull request #9042:
URL: https://github.com/apache/arrow/pull/9042#issuecomment-752418069
@jorgecarleitao Yes, small things can have a big effect...
The difference is a bit "magnified" though because of the `n*n` behavior in
the join.
If you have ideas of ho
jhorstmann commented on pull request #9040:
URL: https://github.com/apache/arrow/pull/9040#issuecomment-752424442
Cool! A few small comments:
- In postgres the argument order is the other way around
[`date_trunc('week', timestamp)`][1]. I haven't compared with other databases,
but
Dandandan commented on a change in pull request #9044:
URL: https://github.com/apache/arrow/pull/9044#discussion_r550162644
##
File path: rust/arrow/src/memory.rs
##
@@ -180,65 +187,62 @@ pub unsafe fn free_aligned(ptr: *mut u8, size: usize) {
///
/// * new_size, when rounded
alamb commented on a change in pull request #9043:
URL: https://github.com/apache/arrow/pull/9043#discussion_r550163355
##
File path: rust/datafusion/src/physical_plan/planner.rs
##
@@ -110,6 +111,16 @@ impl DefaultPhysicalPlanner {
// leaf node, children cannot be
MarcoGorelli opened a new pull request #9045:
URL: https://github.com/apache/arrow/pull/9045
Previously, cmake format wasn't running (likely because there were two
`entry` keys, so the command getting overridden) - furthermore, it was
unnecessarily slow as it didn't take advantage of pre-c
jorgecarleitao commented on a change in pull request #9044:
URL: https://github.com/apache/arrow/pull/9044#discussion_r550168432
##
File path: rust/arrow/src/memory.rs
##
@@ -180,65 +187,62 @@ pub unsafe fn free_aligned(ptr: *mut u8, size: usize) {
///
/// * new_size, when ro
github-actions[bot] commented on pull request #9045:
URL: https://github.com/apache/arrow/pull/9045#issuecomment-752475426
Thanks for opening a pull request!
Could you open an issue for this pull request on JIRA?
https://issues.apache.org/jira/browse/ARROW
Then could
alamb opened a new pull request #9046:
URL: https://github.com/apache/arrow/pull/9046
Rustfmt error was introduced in this PR:
https://github.com/apache/arrow/commit/30ce2eb5d4dc6136594f005f6b7ec7315afc9a88
This PR fixes the issue
alamb commented on pull request #9041:
URL: https://github.com/apache/arrow/pull/9041#issuecomment-752490751
🤦 Sorry @nevi-me -- I also have a PR here to fix it:
https://github.com/apache/arrow/pull/9046
I think it is blocking other PRs so I will merge
https://github.com/apache/ar
alamb commented on pull request #9046:
URL: https://github.com/apache/arrow/pull/9046#issuecomment-752495469
Lint is passed, so I am merging this in to unblock master / other PRs.
This is an automated message from the Apache
alamb closed pull request #9046:
URL: https://github.com/apache/arrow/pull/9046
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the
mqy commented on a change in pull request #9025:
URL: https://github.com/apache/arrow/pull/9025#discussion_r550173817
##
File path: rust/arrow/src/datatypes.rs
##
@@ -1903,9 +1929,20 @@ mod tests {
#[test]
fn serde_struct_type() {
+let kv_array = [("k".to_st
mqy commented on a change in pull request #9025:
URL: https://github.com/apache/arrow/pull/9025#discussion_r550173817
##
File path: rust/arrow/src/datatypes.rs
##
@@ -1903,9 +1929,20 @@ mod tests {
#[test]
fn serde_struct_type() {
+let kv_array = [("k".to_st
sweb opened a new pull request #9047:
URL: https://github.com/apache/arrow/pull/9047
This PR adds capabilities to read decimal columns in parquet files that
store them as i32 or i64.
I tried to follow the approach in #8926 by using casts. However, there is an
issue with my solution
github-actions[bot] commented on pull request #9047:
URL: https://github.com/apache/arrow/pull/9047#issuecomment-752563024
Thanks for opening a pull request!
Could you open an issue for this pull request on JIRA?
https://issues.apache.org/jira/browse/ARROW
Then could
nevi-me commented on pull request #9046:
URL: https://github.com/apache/arrow/pull/9046#issuecomment-752593629
Thanks @alamb
This is an automated message from the Apache Git Service.
To respond to the message, please log on
mqy commented on pull request #9047:
URL: https://github.com/apache/arrow/pull/9047#issuecomment-752615117
ARROW-${JIRA_ID} :)
This is an automated message from the Apache Git Service.
To respond to the message, please log on
nevi-me commented on a change in pull request #9025:
URL: https://github.com/apache/arrow/pull/9025#discussion_r550200770
##
File path: rust/arrow/src/datatypes.rs
##
@@ -1594,7 +1620,7 @@ impl Field {
impl fmt::Display for Field {
fn fmt(&self, f: &mut fmt::Formatter)
nevi-me commented on a change in pull request #9025:
URL: https://github.com/apache/arrow/pull/9025#discussion_r550201222
##
File path: rust/arrow/src/datatypes.rs
##
@@ -1903,9 +1929,20 @@ mod tests {
#[test]
fn serde_struct_type() {
+let kv_array = [("k".t
andygrove commented on a change in pull request #9043:
URL: https://github.com/apache/arrow/pull/9043#discussion_r550216625
##
File path: rust/datafusion/src/physical_plan/coalesce_batches.rs
##
@@ -0,0 +1,295 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+/
github-actions[bot] commented on pull request #9047:
URL: https://github.com/apache/arrow/pull/9047#issuecomment-752645838
https://issues.apache.org/jira/browse/ARROW-11072
This is an automated message from the Apache Git Ser
andygrove commented on a change in pull request #9043:
URL: https://github.com/apache/arrow/pull/9043#discussion_r550217442
##
File path: rust/datafusion/src/physical_plan/planner.rs
##
@@ -110,6 +111,16 @@ impl DefaultPhysicalPlanner {
// leaf node, children canno
github-actions[bot] removed a comment on pull request #9047:
URL: https://github.com/apache/arrow/pull/9047#issuecomment-752563024
Thanks for opening a pull request!
Could you open an issue for this pull request on JIRA?
https://issues.apache.org/jira/browse/ARROW
Th
Dandandan opened a new pull request #9048:
URL: https://github.com/apache/arrow/pull/9048
This applies some refactoring to `build_batch_from_indices` which is
supposed to make further changes easier, e.g. solving
https://issues.apache.org/jira/browse/ARROW-11030
* This starts handli
Dandandan commented on pull request #9036:
URL: https://github.com/apache/arrow/pull/9036#issuecomment-752675210
@andygrove thought more about this, I think we are able to use
`indices.len()` for the *exact* required capacity rather than using previous
sizes. I included the change among ot
github-actions[bot] commented on pull request #9048:
URL: https://github.com/apache/arrow/pull/9048#issuecomment-752678852
https://issues.apache.org/jira/browse/ARROW-11076
This is an automated message from the Apache Git Ser
mqy commented on a change in pull request #9025:
URL: https://github.com/apache/arrow/pull/9025#discussion_r550250838
##
File path: rust/arrow/src/datatypes.rs
##
@@ -1903,9 +1929,20 @@ mod tests {
#[test]
fn serde_struct_type() {
+let kv_array = [("k".to_st
nealrichardson commented on a change in pull request #9034:
URL: https://github.com/apache/arrow/pull/9034#discussion_r550251046
##
File path: r/vignettes/install.Rmd
##
@@ -175,25 +181,24 @@ tune one of several parameters. Here are some known
complications and ways to ad
If
nealrichardson closed pull request #9033:
URL: https://github.com/apache/arrow/pull/9033
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to g
nealrichardson commented on a change in pull request #9039:
URL: https://github.com/apache/arrow/pull/9039#discussion_r550251893
##
File path: r/tests/testthat/test-python-flight.R
##
@@ -0,0 +1,63 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more cont
nealrichardson closed pull request #9039:
URL: https://github.com/apache/arrow/pull/9039
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to g
mqy commented on a change in pull request #9025:
URL: https://github.com/apache/arrow/pull/9025#discussion_r550250838
##
File path: rust/arrow/src/datatypes.rs
##
@@ -1903,9 +1929,20 @@ mod tests {
#[test]
fn serde_struct_type() {
+let kv_array = [("k".to_st
andygrove commented on pull request #9036:
URL: https://github.com/apache/arrow/pull/9036#issuecomment-752685504
Closed in favor of https://github.com/apache/arrow/pull/9048
This is an automated message from the Apache Git Se
andygrove closed pull request #9036:
URL: https://github.com/apache/arrow/pull/9036
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
andygrove commented on a change in pull request #9048:
URL: https://github.com/apache/arrow/pull/9048#discussion_r550257236
##
File path: rust/datafusion/src/physical_plan/hash_join.rs
##
@@ -276,55 +276,58 @@ fn build_batch_from_indices(
todo!("Create empty record bat
andygrove commented on pull request #9048:
URL: https://github.com/apache/arrow/pull/9048#issuecomment-752688847
For TPC-H q12 at SF=100 and 8 partitions:
| Batch Size | Master | #9043 | #9043 + This PR |
| --- | --- | --- |
| 4096 | ??? | ??? | 25.2 s |
| 8192 | 617.5 s | 70
andygrove edited a comment on pull request #9048:
URL: https://github.com/apache/arrow/pull/9048#issuecomment-752688847
For TPC-H q12 at SF=100 and 8 partitions:
| Batch Size | Master | #9043 | #9043 + This PR |
| --- | --- | --- | --- |
| 4096 | ??? | ??? | 25.2 s |
| 8192 |
Dandandan commented on a change in pull request #9048:
URL: https://github.com/apache/arrow/pull/9048#discussion_r550260158
##
File path: rust/datafusion/src/physical_plan/hash_join.rs
##
@@ -276,55 +276,58 @@ fn build_batch_from_indices(
todo!("Create empty record bat
andygrove closed pull request #9043:
URL: https://github.com/apache/arrow/pull/9043
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
andygrove commented on pull request #9035:
URL: https://github.com/apache/arrow/pull/9035#issuecomment-752691811
I will rebase this and address feedback once
https://github.com/apache/arrow/pull/9048 is merged
This is an aut
Dandandan commented on pull request #9048:
URL: https://github.com/apache/arrow/pull/9048#issuecomment-752692256
Awesome, better than I expected!
This is an automated message from the Apache Git Service.
To respond to the mes
nealrichardson closed pull request #9034:
URL: https://github.com/apache/arrow/pull/9034
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to g
jonkeane commented on a change in pull request #8947:
URL: https://github.com/apache/arrow/pull/8947#discussion_r550266972
##
File path: r/tests/testthat/test-dplyr.R
##
@@ -133,6 +133,42 @@ test_that("filtering with expression", {
)
})
+test_that("filtering with arithmet
Dandandan commented on a change in pull request #9048:
URL: https://github.com/apache/arrow/pull/9048#discussion_r550269050
##
File path: rust/datafusion/src/physical_plan/hash_join.rs
##
@@ -276,55 +276,58 @@ fn build_batch_from_indices(
todo!("Create empty record bat
Dandandan commented on a change in pull request #9048:
URL: https://github.com/apache/arrow/pull/9048#discussion_r550269050
##
File path: rust/datafusion/src/physical_plan/hash_join.rs
##
@@ -276,55 +276,58 @@ fn build_batch_from_indices(
todo!("Create empty record bat
alamb commented on a change in pull request #9043:
URL: https://github.com/apache/arrow/pull/9043#discussion_r550271524
##
File path: rust/datafusion/src/physical_plan/coalesce_batches.rs
##
@@ -0,0 +1,295 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or
jhorstmann commented on a change in pull request #8975:
URL: https://github.com/apache/arrow/pull/8975#discussion_r550272926
##
File path: rust/arrow/src/array/array_primitive.rs
##
@@ -67,19 +67,6 @@ impl PrimitiveArray {
self.data.is_empty()
}
-/// Returns
AWSjswinney commented on pull request #8491:
URL: https://github.com/apache/arrow/pull/8491#issuecomment-752707138
I believe this PR now has everything needed to build and publish aarch64
wheels for manylinux2014. Thanks for reviewing it!
--
carols10cents opened a new pull request #9049:
URL: https://github.com/apache/arrow/pull/9049
This PR has a few refactorings and then the main commit contains a new
Flight integration test client and server 🎉
The middleware scenario tests are currently skipped because they will fail
nealrichardson opened a new pull request #9050:
URL: https://github.com/apache/arrow/pull/9050
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL abov
nealrichardson commented on a change in pull request #8947:
URL: https://github.com/apache/arrow/pull/8947#discussion_r550293049
##
File path: r/R/expression.R
##
@@ -59,6 +59,44 @@ build_array_expression <- function(.Generic, e1, e2, ...) {
} else {
e1 <- .wrap_arrow(e
github-actions[bot] commented on pull request #9050:
URL: https://github.com/apache/arrow/pull/9050#issuecomment-752727272
https://issues.apache.org/jira/browse/ARROW-11079
This is an automated message from the Apache Git Ser
github-actions[bot] commented on pull request #9049:
URL: https://github.com/apache/arrow/pull/9049#issuecomment-752727271
https://issues.apache.org/jira/browse/ARROW-8853
This is an automated message from the Apache Git Serv
sunchao commented on a change in pull request #9047:
URL: https://github.com/apache/arrow/pull/9047#discussion_r550321322
##
File path: rust/parquet/src/arrow/schema.rs
##
@@ -657,6 +645,22 @@ impl ParquetTypeConverter<'_> {
}
}
+fn to_decimal(&self) -> Resu
kou commented on pull request #8386:
URL: https://github.com/apache/arrow/pull/8386#issuecomment-752764866
We're still working on it at #8915.
This is an automated message from the Apache Git Service.
To respond to the messag
kou commented on a change in pull request #9045:
URL: https://github.com/apache/arrow/pull/9045#discussion_r550335310
##
File path: .pre-commit-config.yaml
##
@@ -40,9 +40,10 @@ repos:
- id: cmake-format
name: CMake Format
language: python
-entr
kou commented on a change in pull request #9045:
URL: https://github.com/apache/arrow/pull/9045#discussion_r550335701
##
File path: .pre-commit-config.yaml
##
@@ -40,9 +40,10 @@ repos:
- id: cmake-format
name: CMake Format
language: python
-entr
seddonm1 commented on pull request #9038:
URL: https://github.com/apache/arrow/pull/9038#issuecomment-752776874
> 1. Yes I think there should be a different/more efficient implementation
that handles the "scalar" case, where the scalar in this case is the list with
values.
Agree. I can
seddonm1 edited a comment on pull request #9038:
URL: https://github.com/apache/arrow/pull/9038#issuecomment-752776874
> 1. Yes I think there should be a different/more efficient implementation
that handles the "scalar" case, where the scalar in this case is the list with
values.
Ag
nealrichardson closed pull request #9050:
URL: https://github.com/apache/arrow/pull/9050
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to g
jonkeane opened a new pull request #9051:
URL: https://github.com/apache/arrow/pull/9051
and tests for the .env pronoun
This is an automated message from the Apache Git Service.
To respond to the message, please log on to Git
nealrichardson commented on pull request #9051:
URL: https://github.com/apache/arrow/pull/9051#issuecomment-752791737
Can you add a news bullet for this (making sure you're rebased on latest
master)?
This is an automated mes
nealrichardson commented on a change in pull request #9051:
URL: https://github.com/apache/arrow/pull/9051#discussion_r550358517
##
File path: r/R/dplyr.R
##
@@ -265,6 +267,8 @@ filter_mask <- function(.data) {
# Then add the column references
# Renaming is handled automa
github-actions[bot] commented on pull request #9051:
URL: https://github.com/apache/arrow/pull/9051#issuecomment-752794206
https://issues.apache.org/jira/browse/ARROW-10668
This is an automated message from the Apache Git Ser
seddonm1 commented on pull request #9038:
URL: https://github.com/apache/arrow/pull/9038#issuecomment-752807355
I have updated this PR with a reimplementation of the logic so that the
kernel which has two undesired behaviour (see points 1 and 2) is no longer
invoked. It should also support
codecov-io edited a comment on pull request #9007:
URL: https://github.com/apache/arrow/pull/9007#issuecomment-750924264
# [Codecov](https://codecov.io/gh/apache/arrow/pull/9007?src=pr&el=h1) Report
> Merging
[#9007](https://codecov.io/gh/apache/arrow/pull/9007?src=pr&el=desc) (516bf56)
ElenaHenderson opened a new pull request #9052:
URL: https://github.com/apache/arrow/pull/9052
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL abov
github-actions[bot] commented on pull request #9052:
URL: https://github.com/apache/arrow/pull/9052#issuecomment-752833739
Thanks for opening a pull request!
Could you open an issue for this pull request on JIRA?
https://issues.apache.org/jira/browse/ARROW
Then could
liyafan82 opened a new pull request #9053:
URL: https://github.com/apache/arrow/pull/9053
By making it immutable, the following benefits can be obtained:
1. It makes the code easier to reason about.
2. It allows JIT to make more optimizations.
3. Immutable objects can be shared,
liyafan82 commented on pull request #9053:
URL: https://github.com/apache/arrow/pull/9053#issuecomment-752834119
This PR also fixes a bug in ArrowMessage#ArrowMessage(ArrowDictionaryBatch,
IpcOption)
This is an automated mes
github-actions[bot] commented on pull request #9053:
URL: https://github.com/apache/arrow/pull/9053#issuecomment-752838345
https://issues.apache.org/jira/browse/ARROW-11081
This is an automated message from the Apache Git Ser
ElenaHenderson closed pull request #9052:
URL: https://github.com/apache/arrow/pull/9052
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to g
jorgecarleitao commented on pull request #9016:
URL: https://github.com/apache/arrow/pull/9016#issuecomment-752862936
I have re-opened this PR as #9032 shows that creating a buffer from a
mutable buffer is 2x faster.
This is
jorgecarleitao commented on a change in pull request #9016:
URL: https://github.com/apache/arrow/pull/9016#discussion_r550412011
##
File path: rust/arrow/src/array/array_string.rs
##
@@ -126,25 +126,28 @@ impl
GenericStringArray {
}
pub(crate) fn from_vec(v: Vec<&s
jorgecarleitao commented on a change in pull request #9016:
URL: https://github.com/apache/arrow/pull/9016#discussion_r550416288
##
File path: rust/arrow/src/array/array_string.rs
##
@@ -126,25 +126,28 @@ impl
GenericStringArray {
}
pub(crate) fn from_vec(v: Vec<&s
jorgecarleitao opened a new pull request #9054:
URL: https://github.com/apache/arrow/pull/9054
This also simplifies some code and adds a test for the boolean case, which
is special due to bit-packing.
This is an automated me
jorgecarleitao opened a new pull request #9055:
URL: https://github.com/apache/arrow/pull/9055
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL abov
github-actions[bot] commented on pull request #9054:
URL: https://github.com/apache/arrow/pull/9054#issuecomment-752880441
https://issues.apache.org/jira/browse/ARROW-11082
This is an automated message from the Apache Git Ser
github-actions[bot] commented on pull request #9055:
URL: https://github.com/apache/arrow/pull/9055#issuecomment-752880440
https://issues.apache.org/jira/browse/ARROW-11084
This is an automated message from the Apache Git Ser
90 matches
Mail list logo
