Re: Introducing Cylon

2020-07-26 Thread Micah Kornfield
Hi Niranda, Interesting results. Did you do any analysis to understand what was the main contributor to the performance differences? Along these lines, did you try joins on any real world datasets? Are you using Spark SQL for comparisons? Also why not use parquet as a starting point? Thanks,

Re: Dummy Scalar in Filter and pass the scalar through Evaluate.

2020-07-26 Thread Micah Kornfield
I haven't used Gandiva, but it seems like you could potentially make a dummy field node [1] for this purpose where you provide the value. As a side note, I'm not sure varying parameters each time is the best way to implement a group-by-operation. -Micah [1]

Gandiva and Threads

2020-07-26 Thread Matt Youill
Hi, Are there any guidelines for using the Gandiva C++ lib from multiple threads? As I increase the number of threads in an application Gandiva begins to raise memory faults (e.g. double free). Sometimes it is on projector creation, sometimes evaluation, seems to happen in multiple

Re: [VOTE] Release Apache Arrow 1.0.0 - RC2

2020-07-26 Thread Sutou Kouhei
MSYS2 package is updated: 1. [done] rebase master 2. [done] upload source 3. [done] upload binaries 4. [done/PR-ready] update website 5. [done] upload ruby gems 6. [ ] upload js packages 8. [done] upload C# packages 9. [andygrove] upload rust crates 10. [xhochy] update conda recipes 11.

Re: [Gandiva] Additional types supported by abs, ceil, floor, etc.

2020-07-26 Thread Troy Zimmerman
Apologies for the noise, but would it make sense if the functions returned the same type as the parameter to improve composition with other functions & operators? My examples might be a hassle to use as-is. > On Jul 26, 2020, at 1:10 PM, Troy Zimmerman > wrote: > > Hi, > > Would it be

[Gandiva] Additional types supported by abs, ceil, floor, etc.

2020-07-26 Thread Troy Zimmerman
Hi, Would it be possible to support additional types in the following functions, or are they purposefully limited to decimal128? Naively, I assume the following make sense: abs(float64) -> float64 abs(int64) -> int64 ceil(float64) -> int64 floor(float64) -> int64 trunc(float64) -> int64

[NIGHTLY] Arrow Build Report for Job nightly-2020-07-26-0

2020-07-26 Thread Crossbow
Arrow Build Report for Job nightly-2020-07-26-0 All tasks: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-07-26-0 Failed Tasks: - conda-linux-gcc-py36-cpu: URL: