Alignment not stored in arrow metadata

As far as I can tell, the alignment padding used in an IPC stream/file isn't stored explicitly, and not really "inferrable", though maybe technically possible if you calculated what bytes are *necessary* given a buffer's data vs. what's actually stored. Just wondering if this has been brought up

Re: Status of Arrow Julia implementation?

Responses inline below: On Tue, Apr 6, 2021 at 9:46 PM Jorge Cardoso Leitão < jorgecarlei...@gmail.com> wrote: > Hi, > > > you all did not attempt to work in the community for any meaningful > amount of time and > are choosing not to try based on the perception that it will create > unacceptable

Re: Status of Arrow Julia implementation?

Hi, > you all did not attempt to work in the community for any meaningful amount of time and are choosing not to try based on the perception that it will create unacceptable overhead for you It is not self-evident to me that Julia's community was sufficiently informed about what they had to give

Re: [NIGHTLY] Arrow Build Report for Job nightly-2021-04-06-0

I just checked crossbow's logs and the jpype build has been failing since October 26 (i.e., we're coming up on 6 months of solid failure). I don't know the use case for why we have it, but if we are going to ignore a failing build for 6 months (seemingly without repercussion), we should probably

Re: [Rust] Is Rust Arrow's deserialization code zero-copy?

Hi Al; Yeah, that's true. The ownership rules of Rust prevents resource ownership outside the container that the resource lifetime taken. That's why there are buffer copies to take ownership and operate on it. Imo, it should be also removed from the docs if it's written like that (for Rust).

Re: [Rust] Is Rust Arrow's deserialization code zero-copy?

I ran into this issue as well. See https://issues.apache.org/jira/projects/ARROW/issues/ARROW-11696 . As I understand it, https://github.com/jorgecarleitao/arrow2 will make reading more efficient. On Apr 6,

[Rust] Is Rust Arrow's deserialization code zero-copy?

Hi, I was reading around the rust-arrow codebase, evaluating it for potential future use. I'm particularly interested in zero-copy processing. I could very well be wrong here, as I don't have a lot of rust experience, but it looks like the code for reading buffers out of IPC messages is

Re: 4.0 release preparation

NVM, I'm dumb. I was looking at the blocking issues on my phone and didn't scroll enough. Carry on. On Tue, Apr 6, 2021 at 8:50 AM Eric Burden wrote: > Where's the wiki page? I looked over the 6 blocking issues, and they seem > a bit beyond my scope at the moment. I'd like to review the 157 to

Re: 4.0 release preparation

Where's the wiki page? I looked over the 6 blocking issues, and they seem a bit beyond my scope at the moment. I'd like to review the 157 to see if there's anything I can tackle. On Mon, Apr 5, 2021 at 9:39 PM Neal Richardson wrote: > Hi all, > It's release time again. By our quarterly release

[NIGHTLY] Arrow Build Report for Job nightly-2021-04-06-0

Arrow Build Report for Job nightly-2021-04-06-0 All tasks: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-04-06-0 Failed Tasks: - conda-linux-gcc-py36-aarch64: URL:

Re: Can't deserialize Arrow Schema with empty metadata?

Schema::empty() serializes to "{\"fields\":[]}". Metadata is not present because of skip_serializing_if. This is the error that would be given if the above string is