Re: [DISCUSS] Moving the format directory to arrow-format repository

2021-05-02 Thread Neville Dipale
Hi, Thanks for the feedback, in light of what's been said, I'm also now fine with leaving the format as is. Changes to the format are visible enough that we shouldn't miss them, as there's normally be a discussion in the ML. Regards Neville On Wed, 28 Apr 2021 at 17:31, Jorge Cardoso Leitão wr

Re: [Format][RFC] Introduce COMPLEX type for IntervalUnit

2021-05-02 Thread Wes McKinney
I also don't understand the comment about closed / open / semi-open intervals. Perhaps there is a confusion, since "interval" as we mean it here is called a "time delta" in some other projects. An interval here does not refer to a time span with a distinct start and end point (I understand this mig

Re: [Format][RFC] Introduce COMPLEX type for IntervalUnit

2021-05-02 Thread Micah Kornfield
Hi Jacek, I'm not sure I fully understand the proposal, could you elaborate with more examples/details? For instance DAY_TIME isn't just a UINT64, it actually contains 2 seperate fields (days and milliseconds). In terms of closed vs half-open, in my limited understanding, that is more a concern o

[DISCUSS] Host DataFusion website on GitHub pages

2021-05-02 Thread Andy Grove
The Arrow Rust community has been discussing the idea of hosting a DataFusion website using GitHub pages. For further details please see the GibHub issue [1] and Google doc [2]. All feedback is welcome. [1] https://github.com/apache/arrow-datafusion/issues/18 [2] https://docs.google.com/document/

Re: [RUST] Proposal for more frequent Rust Arrow release process

2021-05-02 Thread Adam Lippai
Hi, my two cents: at my previous workplace (Tresorit) we created releases every week and that process was even heavier including cross platform verification and approval, a 8-12 hours long dedicated manual QA process, discussions with the marketing and support teams. Open source is certainly diffe

Re: [RUST] Proposal for more frequent Rust Arrow release process

2021-05-02 Thread Andrew Lamb
Micah and Julian, thank you both for your thoughts. I largely agree with Micah; While the Apache process may be heavy weight in certain aspects, I think we can achieve Rust releases every 2 week within that framework. As I doubt very much we'll get it perfect on the first try, I was envisioning

[NIGHTLY] Arrow Build Report for Job nightly-2021-05-02-0

2021-05-02 Thread Crossbow
Arrow Build Report for Job nightly-2021-05-02-0 All tasks: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-05-02-0 Failed Tasks: - conda-linux-gcc-py36-arm64: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-05-02-0-drone-conda-linux-g

Re: [Format][RFC] Introduce COMPLEX type for IntervalUnit

2021-05-02 Thread Jacek Pliszka
Hi! I wonder if it were possible to have generic interval with integers of specified size just to have common base for interval arithmetic. Then user can convert their period to ordinals and use the arithmetic (joining, deoverlapping, common parts, explosion etc.). So YEAR_MONTH and DAY_TIME wou