Re: [Discuss] Arrow Release Schedule

2020-11-10 Thread Wes McKinney
+1 to everything that Kou said. On Tue, Nov 10, 2020 at 8:53 PM Sutou Kouhei wrote: > Hi, > > I agree with Wes. We can automate more release related > tasks. I hope that we produce and verify release artifacts > nightly. > > See also: > [DISCUSS] Release cadence and release vote conventions > >

Re: [Discuss] Arrow Release Schedule

2020-11-10 Thread Sutou Kouhei
Hi, I agree with Wes. We can automate more release related tasks. I hope that we produce and verify release artifacts nightly. See also: [DISCUSS] Release cadence and release vote conventions

Re: [ANNOUNCE] New Arrow committer: Andrew Lamb

2020-11-10 Thread Fan Liya
Congratulations, Andrew. Best, Liya Fan On Wed, Nov 11, 2020 at 7:50 AM Keerat Singh wrote: > Congratulations Andrew.  > > On Tue, Nov 10, 2020 at 11:12 AM Andrew Lamb wrote: > > > Thank you all for your welcome and the effort you put into fostering this > > great community. I look forward

Re: [Discuss] Arrow Release Schedule

2020-11-10 Thread Wes McKinney
We do need a PMC member to sign the release artifacts. Aside from that, IMHO there is a lot that can be done to improve the automation around producing the release artifacts and preparing the release branch. As Krisztian can attest, producing a release currently requires a _lot_ of human time

Re: Pandas Block Manager

2020-11-10 Thread Micah Kornfield
Sorry, I should clarify, I'm not familiar with zero copy from Pandas to Arrow, so there might be something else going on here. But once an arrow file is written out, buffers will be padded/aligned to 8 bytes. In general, I think relying on exact memory translation from systems that aren't used

Re: Pandas Block Manager

2020-11-10 Thread Micah Kornfield
> > My question is: why are these addresses not 40 bytes apart from each other? > What's in the gaps between the buffers? It's not null bitsets - there's > only one buffer for each column. Thanks - All buffers are padded to at least 8 bytes (and per the spec 64 is recommended). On Tue, Nov 10,

Re: [ANNOUNCE] New Arrow committer: Andrew Lamb

2020-11-10 Thread Keerat Singh
Congratulations Andrew.  On Tue, Nov 10, 2020 at 11:12 AM Andrew Lamb wrote: > Thank you all for your welcome and the effort you put into fostering this > great community. I look forward to continuing to work with you all. > > Andrew > > On Tue, Nov 10, 2020 at 1:01 PM Chao Sun wrote: > > >

Re: Pandas Block Manager

2020-11-10 Thread Nicholas White
I've done a bit more digging. This code: df = pd.DataFrame(np.random.randint(10, size=(5, 5))) table = pa.Table.from_pandas(df) mem = [] for c in table.columns: buf = c.chunks[0].buffers()[1] mem.append((buf.address, buf.size)) sorted(mem) ...prints... [(140262915478912,

Re: [Discuss] Arrow Release Schedule

2020-11-10 Thread Keerat Singh
Hi Wes, Is it only the PMC members that can volunteer to drive this or can someone from the community volunteer and drive as well if they desire to have a release sooner? I see that the release process has a fairly comprehensive checklist of tasks here(

Re: [ANNOUNCE] New Arrow committer: Andrew Lamb

2020-11-10 Thread Andrew Lamb
Thank you all for your welcome and the effort you put into fostering this great community. I look forward to continuing to work with you all. Andrew On Tue, Nov 10, 2020 at 1:01 PM Chao Sun wrote: > Congrats Andrew! > > On Tue, Nov 10, 2020 at 9:21 AM Micah Kornfield > wrote: > > > Congrats!

Re: [ANNOUNCE] New Arrow committer: Andrew Lamb

2020-11-10 Thread Chao Sun
Congrats Andrew! On Tue, Nov 10, 2020 at 9:21 AM Micah Kornfield wrote: > Congrats! > > On Tue, Nov 10, 2020 at 8:51 AM Ying Zhou wrote: > > > Congratulations Andrew! > > > > > On Nov 10, 2020, at 10:42 AM, Andy Grove > wrote: > > > > > > On behalf of the Arrow PMC, I'm happy to announce that

Re: [ANNOUNCE] New Arrow committer: Andrew Lamb

2020-11-10 Thread Micah Kornfield
Congrats! On Tue, Nov 10, 2020 at 8:51 AM Ying Zhou wrote: > Congratulations Andrew! > > > On Nov 10, 2020, at 10:42 AM, Andy Grove wrote: > > > > On behalf of the Arrow PMC, I'm happy to announce that Andrew Lamb has > > accepted an invitation to become a committer on Apache Arrow. > > > >

Re: [ANNOUNCE] New Arrow committer: Andrew Lamb

2020-11-10 Thread Ying Zhou
Congratulations Andrew! > On Nov 10, 2020, at 10:42 AM, Andy Grove wrote: > > On behalf of the Arrow PMC, I'm happy to announce that Andrew Lamb has > accepted an invitation to become a committer on Apache Arrow. > > Welcome, and thank you for your contributions!

Re: [ANNOUNCE] New Arrow committer: Andrew Lamb

2020-11-10 Thread Neville Dipale
Congrats Andrew! Look forward to working more with you On Tue, 10 Nov 2020, 18:07 Jorge Cardoso Leitão, wrote: > Congrats, Andrew! > > Andrew has been doing an amazing job, both on the implementation but also > at reviewing and helping others. He taught me a lot, I am having a great > time

Re: [ANNOUNCE] New Arrow committer: Andrew Lamb

2020-11-10 Thread Jorge Cardoso Leitão
Congrats, Andrew! Andrew has been doing an amazing job, both on the implementation but also at reviewing and helping others. He taught me a lot, I am having a great time working with him and I am thus really happy about this. Best, Jorge On Tue, Nov 10, 2020 at 4:42 PM Andy Grove wrote: > On

Re: [ANNOUNCE] New Arrow committer: Andrew Lamb

2020-11-10 Thread Wes McKinney
Congrats Andrew! On Tue, Nov 10, 2020 at 9:42 AM Andy Grove wrote: > On behalf of the Arrow PMC, I'm happy to announce that Andrew Lamb has > accepted an invitation to become a committer on Apache Arrow. > > Welcome, and thank you for your contributions! >

[ANNOUNCE] New Arrow committer: Andrew Lamb

2020-11-10 Thread Andy Grove
On behalf of the Arrow PMC, I'm happy to announce that Andrew Lamb has accepted an invitation to become a committer on Apache Arrow. Welcome, and thank you for your contributions!

Re: Using arrow/compute/kernels/*internal.h headers

2020-11-10 Thread Wes McKinney
Yes, open a Jira and propose a PR implementing the changes you need On Mon, Nov 9, 2020 at 8:31 PM Niranda Perera wrote: > > @wes How should I proceed with this nevertheless? should I open a JIRA? > > On Mon, Nov 9, 2020 at 11:09 AM Wes McKinney wrote: > > > On Mon, Nov 9, 2020 at 9:32 AM

[NIGHTLY] Arrow Build Report for Job nightly-2020-11-10-0

2020-11-10 Thread Crossbow
Arrow Build Report for Job nightly-2020-11-10-0 All tasks: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-11-10-0 Failed Tasks: - conda-linux-gcc-py38-cuda: URL: