jorgecarleitao commented on pull request #9297:
URL: https://github.com/apache/arrow/pull/9297#issuecomment-765885411
NOTE: this PR is 40 LOC change. The rest comes from the other PR. Please
wait for the merge, this is only a draft to indicate what can be done with this.
codecov-io commented on pull request #9297:
URL: https://github.com/apache/arrow/pull/9297#issuecomment-765883730
# [Codecov](https://codecov.io/gh/apache/arrow/pull/9297?src=pr=h1) Report
> Merging
[#9297](https://codecov.io/gh/apache/arrow/pull/9297?src=pr=desc) (02ba749)
into
github-actions[bot] commented on pull request #9297:
URL: https://github.com/apache/arrow/pull/9297#issuecomment-765873864
https://issues.apache.org/jira/browse/ARROW-11354
This is an automated message from the Apache Git
jorgecarleitao opened a new pull request #9297:
URL: https://github.com/apache/arrow/pull/9297
This PR improves the performance of certain time / date casts by using the
brand new API proposed in #9235 .
That API allows for a very fast execution of unary and infalible operations
on
tyrelr commented on a change in pull request #9215:
URL: https://github.com/apache/arrow/pull/9215#discussion_r563020240
##
File path: rust/arrow/src/array/array_primitive.rs
##
@@ -86,13 +86,9 @@ impl PrimitiveArray {
}
/// Returns the primitive value at index
jorgecarleitao edited a comment on pull request #9271:
URL: https://github.com/apache/arrow/pull/9271#issuecomment-765865323
Thanks a lot for your points. I am learning a lot! :)
Note that for small arrays, we are basically in the metadata problem on
which the "payload size" of
jorgecarleitao commented on pull request #9271:
URL: https://github.com/apache/arrow/pull/9271#issuecomment-765865323
Thanks a lot for your points. I am learning a lot! :)
Note that for small arrays, we are basically in the metadata problem on
which the "payload size" of
tyrelr commented on a change in pull request #9215:
URL: https://github.com/apache/arrow/pull/9215#discussion_r562999845
##
File path: rust/arrow/src/array/array_primitive.rs
##
@@ -86,13 +86,9 @@ impl PrimitiveArray {
}
/// Returns the primitive value at index
nealrichardson commented on pull request #9296:
URL: https://github.com/apache/arrow/pull/9296#issuecomment-765742440
@kszucs that didn't trigger a Travis build for some reason
This is an automated message from the
ianmcook edited a comment on pull request #9287:
URL: https://github.com/apache/arrow/pull/9287#issuecomment-765740804
FYI, the version of vcpkg that is currently preinstalled on the Github
Actions Windows images is 2020.11.12 (as noted
ianmcook commented on pull request #9287:
URL: https://github.com/apache/arrow/pull/9287#issuecomment-765740804
FYI, the version of vcpkg that is currently preinstalled on the Github
Actions Windows images is 2020.11.12 (as noted
alamb commented on pull request #9240:
URL: https://github.com/apache/arrow/pull/9240#issuecomment-765718704
Thank you for understanding @sunchao
This is an automated message from the Apache Git Service.
To respond to
Dandandan edited a comment on pull request #9271:
URL: https://github.com/apache/arrow/pull/9271#issuecomment-763553182
Thanks @jorgecarleitao makes sense.
I'm planning a blog post soon again, might put some details there.
In short it is `cargo profiler` installed from the repo
github-actions[bot] commented on pull request #9296:
URL: https://github.com/apache/arrow/pull/9296#issuecomment-765679694
https://issues.apache.org/jira/browse/ARROW-11350
This is an automated message from the Apache Git
github-actions[bot] commented on pull request #9296:
URL: https://github.com/apache/arrow/pull/9296#issuecomment-765678557
Revision: 92c53d0e9081e0d92aa2469c644343f209abd0ff
Submitted crossbow builds: [ursacomputing/crossbow @
nealrichardson commented on pull request #9296:
URL: https://github.com/apache/arrow/pull/9296#issuecomment-765678197
@github-actions crossbow submit homebrew-r-autobrew
This is an automated message from the Apache Git
houqp commented on pull request #9230:
URL: https://github.com/apache/arrow/pull/9230#issuecomment-765663967
Is there an easy way to reproduce this integration test failure locally?
This is an automated message from the
sunchao commented on pull request #9240:
URL: https://github.com/apache/arrow/pull/9240#issuecomment-765652549
No worries @alamb . I'll do review on this closed PR and we can address any
feedback in followups.
This is an
Dandandan commented on a change in pull request #9293:
URL: https://github.com/apache/arrow/pull/9293#discussion_r562868988
##
File path: rust/arrow/src/array/array_primitive.rs
##
@@ -94,6 +94,32 @@ impl PrimitiveArray {
let offset = i + self.offset();
jorgecarleitao commented on pull request #9111:
URL: https://github.com/apache/arrow/pull/9111#issuecomment-765638937
@wesm , is there any update around this? If we do not plan to pursue this,
please let me know and we close this PR.
jorgecarleitao commented on a change in pull request #9215:
URL: https://github.com/apache/arrow/pull/9215#discussion_r562862070
##
File path: rust/arrow/src/array/array_primitive.rs
##
@@ -86,13 +86,9 @@ impl PrimitiveArray {
}
/// Returns the primitive value at
westonpace commented on issue #9295:
URL: https://github.com/apache/arrow/issues/9295#issuecomment-765634730
Thanks for asking. There are a number of things to consider when looking at
memory allocations by Arrow. Also, which language are you working with?
Out of the box Arrow
jorgecarleitao commented on a change in pull request #9262:
URL: https://github.com/apache/arrow/pull/9262#discussion_r562857366
##
File path: .github/workflows/rust.yml
##
@@ -109,14 +109,14 @@ jobs:
export CARGO_HOME="/github/home/.cargo"
export
nealrichardson commented on a change in pull request #9294:
URL: https://github.com/apache/arrow/pull/9294#discussion_r562854049
##
File path: r/tests/testthat/test-compute-arith.R
##
@@ -18,32 +18,31 @@
test_that("Addition", {
a <- Array$create(c(1:4, NA_integer_))
jorgecarleitao commented on a change in pull request #9235:
URL: https://github.com/apache/arrow/pull/9235#discussion_r562855082
##
File path: rust/arrow/src/buffer.rs
##
@@ -963,11 +970,157 @@ impl MutableBuffer {
/// Extends the buffer by `additional` bytes equal to
jorgecarleitao commented on a change in pull request #9293:
URL: https://github.com/apache/arrow/pull/9293#discussion_r562853687
##
File path: rust/arrow/src/array/array_primitive.rs
##
@@ -94,6 +94,32 @@ impl PrimitiveArray {
let offset = i + self.offset();
jorgecarleitao commented on pull request #9293:
URL: https://github.com/apache/arrow/pull/9293#issuecomment-765628986
Sorry for the noise, you are of course right. Let me just review it :)
This is an automated message from
westonpace commented on pull request #8240:
URL: https://github.com/apache/arrow/pull/8240#issuecomment-765628471
Ah, no problem. Looks good. We will have to keep this need in mind if we
ever end up adopting the multi-queue approach.
alamb edited a comment on pull request #9240:
URL: https://github.com/apache/arrow/pull/9240#issuecomment-765625395
I am really sorry @sunchao -- I missed your earlier comment that you would
be reviewing this more carefully. I have been trying to clear out the backlog
of Rust PRs and I
alamb commented on pull request #9240:
URL: https://github.com/apache/arrow/pull/9240#issuecomment-765625395
I am sorry @sunchao -- I missed your earlier comment that you would be
reviewing this more carefully. Would you like me to revert this PR and prepare
a new one to re-merge?
nealrichardson commented on pull request #9296:
URL: https://github.com/apache/arrow/pull/9296#issuecomment-765620806
It does not:
https://github.com/nealrichardson/arrow/runs/1750764290?check_suite_focus=true#step:9:523
Dandandan commented on pull request #9293:
URL: https://github.com/apache/arrow/pull/9293#issuecomment-765619033
@jorgecarleitao yes, that would make more sense for the particular use case
I mentioned, and probably would be more performant as well (it could even use
`memset` /
nealrichardson opened a new pull request #9296:
URL: https://github.com/apache/arrow/pull/9296
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL
kummishra opened a new issue #9295:
URL: https://github.com/apache/arrow/issues/9295
hello,
we are noticing this issue when having our data placed in arrow ipc format ,
which does read very fast , almost no time. but we see a strange behaviour
while reading the data values from the
jorisvandenbossche commented on a change in pull request #9294:
URL: https://github.com/apache/arrow/pull/9294#discussion_r562833164
##
File path: docs/source/cpp/compute.rst
##
@@ -744,3 +749,34 @@ Structural transforms
* \(2) For each value in the list child array, the
jorgecarleitao commented on pull request #9293:
URL: https://github.com/apache/arrow/pull/9293#issuecomment-765605635
@Dandandan , thanks a lot for this.
Looking at the use-case, couldn't it make more sense to offer a method that
creates a constant non-null array and a constant null
bkietz closed pull request #9274:
URL: https://github.com/apache/arrow/pull/9274
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
github-actions[bot] commented on pull request #9294:
URL: https://github.com/apache/arrow/pull/9294#issuecomment-765587080
https://issues.apache.org/jira/browse/ARROW-8919
This is an automated message from the Apache Git
bkietz opened a new pull request #9294:
URL: https://github.com/apache/arrow/pull/9294
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go
Dandandan commented on pull request #9271:
URL: https://github.com/apache/arrow/pull/9271#issuecomment-765564334
This PR itself is ready for review.
I think the performance for slicing for small slices would be something to
look at later.
github-actions[bot] commented on pull request #9293:
URL: https://github.com/apache/arrow/pull/9293#issuecomment-765558530
https://issues.apache.org/jira/browse/ARROW-11349
This is an automated message from the Apache Git
Dandandan opened a new pull request #9293:
URL: https://github.com/apache/arrow/pull/9293
The idea of this PR is to have a function `from_iter_values` that (just like
`from_iter`) creates an array based on an iterator, but from `T` instead of
`Option`.
I have seen some places in
pitrou commented on pull request #8240:
URL: https://github.com/apache/arrow/pull/8240#issuecomment-765553385
@westonpace Sorry, I wasn't clear. I was just cc'ing you for the PR itself,
not for the CI failure.
This is an
ianmcook commented on pull request #9287:
URL: https://github.com/apache/arrow/pull/9287#issuecomment-765515335
Replies inline
> Thanks Ian for working on this!
>
> I have a couple of questions:
>
> * could we define multiple manifest files for different use cases (e.g.
sunchao edited a comment on pull request #9240:
URL: https://github.com/apache/arrow/pull/9240#issuecomment-765491705
@alamb given that this is a 1k+ line PR, could you give us a chance to
review it properly before eagerly merging it?
sunchao commented on pull request #9240:
URL: https://github.com/apache/arrow/pull/9240#issuecomment-765491705
Given that this is a 1k+ line PR, could you give us a chance to review it
properly before eagerly merging it?
alamb closed pull request #9279:
URL: https://github.com/apache/arrow/pull/9279
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the
alamb commented on a change in pull request #9262:
URL: https://github.com/apache/arrow/pull/9262#discussion_r562702562
##
File path: .github/workflows/rust.yml
##
@@ -109,14 +109,14 @@ jobs:
export CARGO_HOME="/github/home/.cargo"
export
cyb70289 commented on pull request #9292:
URL: https://github.com/apache/arrow/pull/9292#issuecomment-765433306
travis ci error not related
This is an automated message from the Apache Git Service.
To respond to the message,
cyb70289 commented on a change in pull request #9274:
URL: https://github.com/apache/arrow/pull/9274#discussion_r562660367
##
File path: python/pyarrow/_compute.pyx
##
@@ -481,88 +481,89 @@ cdef class FunctionOptions(_Weakrefable):
cdef class _CastOptions(FunctionOptions):
bkietz commented on a change in pull request #9274:
URL: https://github.com/apache/arrow/pull/9274#discussion_r562634717
##
File path: python/pyarrow/_compute.pyx
##
@@ -481,88 +481,89 @@ cdef class FunctionOptions(_Weakrefable):
cdef class _CastOptions(FunctionOptions):
bkietz commented on a change in pull request #9274:
URL: https://github.com/apache/arrow/pull/9274#discussion_r562633900
##
File path: python/pyarrow/_compute.pyx
##
@@ -481,88 +481,89 @@ cdef class FunctionOptions(_Weakrefable):
cdef class _CastOptions(FunctionOptions):
Dandandan commented on pull request #9271:
URL: https://github.com/apache/arrow/pull/9271#issuecomment-765392170
@nevi-me
I don't think indeed it is very expensive on large Arrays compared to the
size / operations on the array, but it turns out to be expensive on very small
nevi-me commented on a change in pull request #9262:
URL: https://github.com/apache/arrow/pull/9262#discussion_r562623291
##
File path: .github/workflows/rust.yml
##
@@ -109,14 +109,14 @@ jobs:
export CARGO_HOME="/github/home/.cargo"
export
liyafan82 commented on pull request #9187:
URL: https://github.com/apache/arrow/pull/9187#issuecomment-765383673
> @liyafan82
>
> > The reason is that, for variable width vectors, it is not possible to
estimate the buffer size without actually filling up the vector.
>
> Why
nevi-me commented on pull request #9271:
URL: https://github.com/apache/arrow/pull/9271#issuecomment-765375980
> I found the "offending" code is this function in `array/data.rs` which
does a `self.clone()`.
This relates to the other discussion that we had on how slicing an array
alamb commented on pull request #9264:
URL: https://github.com/apache/arrow/pull/9264#issuecomment-765373605
@jorgecarleitao @andygrove and @seddonm1 and @Dandandan -- what do you
think of this approach to testing DataFusion output?
alamb closed pull request #9240:
URL: https://github.com/apache/arrow/pull/9240
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the
jhorstmann commented on pull request #9271:
URL: https://github.com/apache/arrow/pull/9271#issuecomment-765371442
Cloning the vector of Buffers and child ArrayData has some overhead.
Incrementing the reference counts should be relatively cheap unless there are
concurrent threads accessing
alamb closed pull request #9290:
URL: https://github.com/apache/arrow/pull/9290
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the
alamb commented on a change in pull request #9232:
URL: https://github.com/apache/arrow/pull/9232#discussion_r562604038
##
File path: rust/arrow/src/datatypes/mod.rs
##
@@ -199,6 +207,81 @@ pub struct Field {
metadata: Option>,
}
+// Decimal (precision, scale) =
kszucs edited a comment on pull request #9285:
URL: https://github.com/apache/arrow/pull/9285#issuecomment-765362746
@xhochy a set of declaration issues with the static build:
```
/opt/vcpkg/installed/arm64-linux-static-release/include/aws/common/math.gcc_arm64_asm.inl:166:26:
alamb commented on a change in pull request #9235:
URL: https://github.com/apache/arrow/pull/9235#discussion_r562596060
##
File path: rust/arrow/src/buffer.rs
##
@@ -1003,6 +1156,28 @@ impl PartialEq for MutableBuffer {
unsafe impl Sync for MutableBuffer {}
unsafe impl Send
kszucs commented on pull request #9285:
URL: https://github.com/apache/arrow/pull/9285#issuecomment-765362746
@xhochy a set of declaration issues with the static build:
```
In file included from
/opt/vcpkg/installed/arm64-linux-static-release/include/aws/common/math.inl:29,
xhochy commented on pull request #9285:
URL: https://github.com/apache/arrow/pull/9285#issuecomment-765333956
@kszucs What is the problem with AWS, can you post the linker error here?
Otherwise this is quite neat
This is
cyb70289 commented on pull request #9292:
URL: https://github.com/apache/arrow/pull/9292#issuecomment-765287468
On Xeon(R) Gold 5218, compiled with clang-9.
```
-
Benchmark
github-actions[bot] commented on pull request #9292:
URL: https://github.com/apache/arrow/pull/9292#issuecomment-765286541
https://issues.apache.org/jira/browse/ARROW-11346
This is an automated message from the Apache Git
cyb70289 opened a new pull request #9292:
URL: https://github.com/apache/arrow/pull/9292
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to
kszucs commented on pull request #9287:
URL: https://github.com/apache/arrow/pull/9287#issuecomment-765260248
Thanks Ian for working on this!
I have a couple of questions:
- could we define multiple manifest files for different use cases (e.g. for
building wheels or having a
cyb70289 commented on a change in pull request #9274:
URL: https://github.com/apache/arrow/pull/9274#discussion_r562484017
##
File path: python/pyarrow/_compute.pyx
##
@@ -481,88 +481,89 @@ cdef class FunctionOptions(_Weakrefable):
cdef class _CastOptions(FunctionOptions):
cyb70289 commented on a change in pull request #9274:
URL: https://github.com/apache/arrow/pull/9274#discussion_r562483502
##
File path: python/pyarrow/_compute.pyx
##
@@ -481,88 +481,89 @@ cdef class FunctionOptions(_Weakrefable):
cdef class _CastOptions(FunctionOptions):
Dandandan commented on pull request #9271:
URL: https://github.com/apache/arrow/pull/9271#issuecomment-765223781
@jorgecarleitao
Maybe, reasonably, yes (as long the underlying vecs have a few items).
I think the clone on the `ArrayData` structure itself is expensive when
72 matches
Mail list logo
