Re: [Discuss] Compatibility Guarantees and Versioning Post "1.0.0"

2019-07-13 Thread Micah Kornfield
SGTM could you or another PMC member start one? Thanks, Micah On Saturday, July 13, 2019, Wes McKinney wrote: > Micah -- I would suggest that -- absent more opinions -- we vote about > adopting the versioning scheme you described here (Format Version and > Library Version) > > On Wed, Jul 10, 2

[jira] [Created] (ARROW-5946) [Rust] [DataFusion] Projection push down with aggregate producing incorrect results

2019-07-13 Thread Andy Grove (JIRA)
Andy Grove created ARROW-5946: - Summary: [Rust] [DataFusion] Projection push down with aggregate producing incorrect results Key: ARROW-5946 URL: https://issues.apache.org/jira/browse/ARROW-5946 Project:

Re: [Discuss] Compatibility Guarantees and Versioning Post "1.0.0"

2019-07-13 Thread Wes McKinney
Micah -- I would suggest that -- absent more opinions -- we vote about adopting the versioning scheme you described here (Format Version and Library Version) On Wed, Jul 10, 2019 at 8:46 AM Wes McKinney wrote: > > On Wed, Jul 10, 2019 at 12:43 AM Micah Kornfield > wrote: > > > > Hi Eric, > > Sh

Re: [DISCUSS] Need for 0.14.1 release due to Python package problems, Parquet forward compatibility problems

2019-07-13 Thread Wes McKinney
On Sat, Jul 13, 2019 at 12:57 PM Wes McKinney wrote: > > OK, that's been merged and updated. Here's a Crossbow build > https://github.com/ursa-labs/crossbow/branches/all?utf8=%E2%9C%93&query=build-665 I'll keep an eye on CI. Anything else I can do to help get an RC out please let me know

Re: [DISCUSS] Need for 0.14.1 release due to Python package problems, Parquet forward compatibility problems

2019-07-13 Thread Wes McKinney
Sorry, spoke too soon, https://github.com/apache/arrow/pull/4856 is the last patch to go in, I'm reviewing that now On Sat, Jul 13, 2019 at 12:06 PM Wes McKinney wrote: > > Thanks Kou. > > I've updated the patch release script [1], pushed the maint-0.14.x > branch [2], and just submitted a Crossb

Re: [DISCUSS] Need for 0.14.1 release due to Python package problems, Parquet forward compatibility problems

2019-07-13 Thread Wes McKinney
Thanks Kou. I've updated the patch release script [1], pushed the maint-0.14.x branch [2], and just submitted a Crossbow packaging run [3] If all looks good, I think this branch can be used to create an RC [1]: https://gist.github.com/wesm/1e4ac14baaa8b27bf13b071d2d715014 [2]: https://github.com

Re: [DISCUSS] Format additions for encoding/compression (Was: [Discuss] Format additions to Arrow for sparse data and data integrity)

2019-07-13 Thread Wes McKinney
On Sat, Jul 13, 2019 at 11:23 AM Antoine Pitrou wrote: > > On Fri, 12 Jul 2019 20:37:15 -0700 > Micah Kornfield wrote: > > > > If the latter, I wonder why Parquet cannot simply be used instead of > > > reinventing something similar but different. > > > > This is a reasonable point. However there

Re: [DISCUSS] Format additions for encoding/compression (Was: [Discuss] Format additions to Arrow for sparse data and data integrity)

2019-07-13 Thread Antoine Pitrou
On Fri, 12 Jul 2019 20:37:15 -0700 Micah Kornfield wrote: > > If the latter, I wonder why Parquet cannot simply be used instead of > > reinventing something similar but different. > > This is a reasonable point. However there is continuum here between file > size and read and write times. P

Re: [DISCUSS][C++][Proposal] Threading engine for Arrow

2019-07-13 Thread Wes McKinney
hi Anton, Ideally PRs like https://github.com/aregm/arrow/pull/6 would be made into apache/arrow where the community can see them. I had no idea this PR existed. I have looked at the demo repository a little bit, but I'm not sure what conclusions it will help reach. What we are missing at the mom

Re: [DISCUSS] Release cadence and release vote conventions

2019-07-13 Thread Wes McKinney
To be effective at improving the life of release managers, the nightly release process really should use as close as possible to the same scripts that the RM uses to produce the release. Otherwise we could have a situation where the nightlies succeed but there is some problem that either fails an R

Re: [DISCUSS] Release cadence and release vote conventions

2019-07-13 Thread Andy Grove
I would like to volunteer to help with Java and Rust release process work, especially nightly releases. Although I'm not that familiar with the Java implementation of Arrow, I have been using Java and Maven for a very long time. Do we envisage a single nightly release process that releases all la

[jira] [Created] (ARROW-5945) [Rust] [DataFusion] Table trait should support building complete queries

2019-07-13 Thread Andy Grove (JIRA)
Andy Grove created ARROW-5945: - Summary: [Rust] [DataFusion] Table trait should support building complete queries Key: ARROW-5945 URL: https://issues.apache.org/jira/browse/ARROW-5945 Project: Apache Arro