[ https://issues.apache.org/jira/browse/ARROW-17054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17565868#comment-17565868 ]
Antoine Pitrou edited comment on ARROW-17054 at 7/12/22 3:07 PM: ----------------------------------------------------------------- Interesting. I suspect that there's some incorrect casting along the way. This doesn't happen with PyArrow. [~paleolimbot] {code:python} >>> import numpy as np >>> arr = np.zeros(2**31, dtype='bool') >>> arr.dtype dtype('bool') >>> arr.nbytes 2147483648 >>> >>> parr = pa.array(arr) >>> parr <pyarrow.lib.BooleanArray object at 0x7f059554e8e0> [ false, false, false, false, false, false, false, false, false, false, ... false, false, false, false, false, false, false, false, false, false ] >>> len(parr) 2147483648 {code} was (Author: pitrou): Interesting. I suspect that there's some incorrect casting along the way. This doesn't happen with PyArrow. [~paleolimbot] > [R] Creating an Array from an object bigger than 2^31 results in an Array of > length 0 > ------------------------------------------------------------------------------------- > > Key: ARROW-17054 > URL: https://issues.apache.org/jira/browse/ARROW-17054 > Project: Apache Arrow > Issue Type: Improvement > Components: R > Reporter: Nicola Crane > Priority: Major > > Apologies for the lack of proper reprex but it crashes my session when I try > to make one. > I'm working on ARROW-16977 which is all about the reporting of object size > having integer overflow issues, but this affects object creation. > {code:r} > library(arrow, warn.conflicts = TRUE) > # works - creates a huge array, hurrah > big_logical <- vector(mode = "logical", length = .Machine$integer.max) > big_logical_array <- Array$create(big_logical) > length(big_logical) > ## [1] 2147483647 > length(big_logical_array) > ## [1] 2147483647 > # creates an array of length 0, boo! > too_big <- vector(mode = "logical", length = .Machine$integer.max + 1) > too_big_array <- Array$create(too_big) > length(too_big) > ## [1] 2147483648 > length(too_big_array) > ## [1] 0 > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010)