[jira] [Created] (ARROW-6020) [Java] Refactor ByteFunctionHelper#hash with new added ArrowBufHasher

2019-07-23 Thread Ji Liu (JIRA)
Ji Liu created ARROW-6020: - Summary: [Java] Refactor ByteFunctionHelper#hash with new added ArrowBufHasher Key: ARROW-6020 URL: https://issues.apache.org/jira/browse/ARROW-6020 Project: Apache Arrow

[jira] [Created] (ARROW-6019) [Java] Port Jdbc and Avro adapter to new directory

2019-07-23 Thread Ji Liu (JIRA)
Ji Liu created ARROW-6019: - Summary: [Java] Port Jdbc and Avro adapter to new directory Key: ARROW-6019 URL: https://issues.apache.org/jira/browse/ARROW-6019 Project: Apache Arrow Issue Type: Task

Re: [Discuss] Do a 0.15.0 release before 1.0.0?

2019-07-23 Thread Micah Kornfield
> > Could we detect the 4-byte length, incur a penalty copying the memory to > an aligned buffer, then continue consuming the stream? I think that is the plan (or at least would be my plan) if we go ahead with the change > (It's probably > fine if we only write the 8-byte length, since consumer

Arrow sync call July 24 at 12:00 US/Eastern, 16:00 UTC

2019-07-23 Thread Neal Richardson
Hi everyone, Reminder that the biweekly Arrow call is tomorrow (well, already today for some of you) at https://meet.google.com/vtm-teks-phx. All are welcome to join. Notes will be sent out to the mailing list afterwards. Neal

Re: Error building cuDF on new Arrow with std::variant backport

2019-07-23 Thread Keith Kraus
Just following up in case anyone was following that this turned out to be an NVCC bug that we've reported to the relevant team internally. We moved the `ipc.cu` file to `ipc.cpp` and it works as expected with gcc. Thanks everyone! -Keith On 7/22/19, 12:52 PM, "Keith Kraus" wrote: We're w

Re: [DISCUSS][C++][Proposal] Threading engine for Arrow

2019-07-23 Thread Wes McKinney
Useful read on this topic today from the Julia language https://julialang.org/blog/2019/07/multithreading On Tue, Jul 23, 2019, 12:22 AM Jacques Nadeau wrote: > There are two main things that have been important to us in Dremio around > threading: > > Separate threading model from algorithms. W

Re: [Discuss] Do a 0.15.0 release before 1.0.0?

2019-07-23 Thread Paul Taylor
+1 for a 0.15.0 before 1.0 if we go ahead with this. I'm curious to hear other's thoughts about compatibility. I think we should avoid breaking backwards compatibility if possible. It's common for apps/libs to be pinned on specific Arrow versions, and I worry it'd cause a lot of work for downs

[ANNOUNCE] Apache Arrow 0.14.1 released

2019-07-23 Thread Krisztián Szűcs
The Apache Arrow community is pleased to announce the 0.14.1 release. This is a patch release including 46 resolved issues ([1]) since the 0.14.0 release. The release is available now from our website, [2] and [3]: http://arrow.apache.org/install/ Release notes are available at: https://a

[jira] [Created] (ARROW-6018) [Release] Gen apidocs step fails with multiple issues

2019-07-23 Thread Krisztian Szucs (JIRA)
Krisztian Szucs created ARROW-6018: -- Summary: [Release] Gen apidocs step fails with multiple issues Key: ARROW-6018 URL: https://issues.apache.org/jira/browse/ARROW-6018 Project: Apache Arrow

Re: [VOTE] Release Apache Arrow 0.14.1 - RC0

2019-07-23 Thread Neal Richardson
I'll handle R. On Mon, Jul 22, 2019 at 5:42 PM Krisztián Szűcs wrote: > On Tue, Jul 23, 2019 at 12:31 AM Krisztián Szűcs < > szucs.kriszt...@gmail.com> > wrote: > > > The remaining tasks are: > > - Updating website (after https://github.com/apache/arrow/pull/4922 is > > merged) > > > I'm generat

[jira] [Created] (ARROW-6017) [FlightRPC] Allow creating Locations with unknown schemes

2019-07-23 Thread lidavidm (JIRA)
lidavidm created ARROW-6017: --- Summary: [FlightRPC] Allow creating Locations with unknown schemes Key: ARROW-6017 URL: https://issues.apache.org/jira/browse/ARROW-6017 Project: Apache Arrow Issue Ty

Re: [DISCUSS][JAVA] Designs & goals for readers/writers

2019-07-23 Thread Wes McKinney
Yes I think text files are OK but I want to make sure that committers are reviewing patches for binary files because there have been a number of incidents in the past where I had to roll back patches to remove such files. On Tue, Jul 23, 2019, 10:37 AM Micah Kornfield wrote: > Hi Wes, > I haven'

Re: [DISCUSS][JAVA] Designs & goals for readers/writers

2019-07-23 Thread Micah Kornfield
Hi Wes, I haven't checked locally but that file at least for me renders as text file in GitHub (with an Apache header). If we want all test data in the testing package I can make sure to move it but I thought text files might be ok in the main repo? Thanks, Micah On Tuesday, July 23, 2019, Wes M

Re: [DISCUSS] Passing the torch on Python wheel (binary) maintenance

2019-07-23 Thread Krisztián Szűcs
The pyarrow windows wheels for version 0.14.1 are no longer available. On Tue, Jul 23, 2019 at 4:19 PM Krisztián Szűcs wrote: > Ok, I'm deleting the 0.14.1 windows wheels then. > > > On Tue, Jul 23, 2019 at 3:40 PM Wes McKinney wrote: > >> I agree that we should not let people install broken wh

Re: [DISCUSS] Passing the torch on Python wheel (binary) maintenance

2019-07-23 Thread Krisztián Szűcs
Ok, I'm deleting the 0.14.1 windows wheels then. On Tue, Jul 23, 2019 at 3:40 PM Wes McKinney wrote: > I agree that we should not let people install broken wheels. > > On Tue, Jul 23, 2019 at 8:38 AM Krisztián Szűcs > wrote: > > > > Although we have a quick fix for that [1] and the fixed wheel

Re: [DISCUSS][JAVA] Designs & goals for readers/writers

2019-07-23 Thread Wes McKinney
I noticed that test data-related files are beginning to be checked in https://github.com/apache/arrow/blob/master/java/adapter/avro/src/test/resources/schema/test.avsc I wanted to make sure this doesn't turn into a slippery slope where we end up with several megabytes or more of test data files

Re: [DISCUSS] Passing the torch on Python wheel (binary) maintenance

2019-07-23 Thread Wes McKinney
I agree that we should not let people install broken wheels. On Tue, Jul 23, 2019 at 8:38 AM Krisztián Szűcs wrote: > > Although we have a quick fix for that [1] and the fixed wheels will be > available soon [2] but sadly pypi doesn't support the update of already > uploaded packages. > > We have

Re: [DISCUSS] Passing the torch on Python wheel (binary) maintenance

2019-07-23 Thread Krisztián Szűcs
Although we have a quick fix for that [1] and the fixed wheels will be available soon [2] but sadly pypi doesn't support the update of already uploaded packages. We have three options: 1. delete the 0.14.1 windows wheels 2. draft a post release [3] only for the windows wheels, last time we did it

Re: [DISCUSS] Passing the torch on Python wheel (binary) maintenance

2019-07-23 Thread Wes McKinney
As we just found in https://issues.apache.org/jira/browse/ARROW-6015, our 0.14.1 wheels have more problems (this time on Windows), so more evidence that we don't have the bandwidth to properly support these packages. On Tue, Jul 16, 2019 at 3:08 PM Jacques Nadeau wrote: > > I think what you sugge

[jira] [Created] (ARROW-6016) [Python] pyarrow get_library_dirs assertion error

2019-07-23 Thread Matthijs Brobbel (JIRA)
Matthijs Brobbel created ARROW-6016: --- Summary: [Python] pyarrow get_library_dirs assertion error Key: ARROW-6016 URL: https://issues.apache.org/jira/browse/ARROW-6016 Project: Apache Arrow

[jira] [Created] (ARROW-6015) [Python] pyarrow: `DLL load failed` when importing on windows

2019-07-23 Thread Ruslan Kuprieiev (JIRA)
Ruslan Kuprieiev created ARROW-6015: --- Summary: [Python] pyarrow: `DLL load failed` when importing on windows Key: ARROW-6015 URL: https://issues.apache.org/jira/browse/ARROW-6015 Project: Apache Ar

[jira] [Created] (ARROW-6014) [Release] Dockerize post-03-website.sh script

2019-07-23 Thread Krisztian Szucs (JIRA)
Krisztian Szucs created ARROW-6014: -- Summary: [Release] Dockerize post-03-website.sh script Key: ARROW-6014 URL: https://issues.apache.org/jira/browse/ARROW-6014 Project: Apache Arrow Issue

[jira] [Created] (ARROW-6013) Support range searcher

2019-07-23 Thread Liya Fan (JIRA)
Liya Fan created ARROW-6013: --- Summary: Support range searcher Key: ARROW-6013 URL: https://issues.apache.org/jira/browse/ARROW-6013 Project: Apache Arrow Issue Type: New Feature Component

[jira] [Created] (ARROW-6012) [C++] Fall back on known Apache mirror for Thrift downloads

2019-07-23 Thread Antoine Pitrou (JIRA)
Antoine Pitrou created ARROW-6012: - Summary: [C++] Fall back on known Apache mirror for Thrift downloads Key: ARROW-6012 URL: https://issues.apache.org/jira/browse/ARROW-6012 Project: Apache Arrow

Re: [Discuss] Do a 0.15.0 release before 1.0.0?

2019-07-23 Thread Uwe L. Korn
It is also a good way to test the change in public. We don't want to adjust something like this anymore in a 1.0.0 release. Already doing this in 0.15.0 and then maybe doing adjustments due to issues that appear "in the wild" is psychologically the easier way. There is a lot of thinking of users

[jira] [Created] (ARROW-6011) Data incomplete when using pyarrow in pyspark in python 3.x

2019-07-23 Thread jiangyu (JIRA)
jiangyu created ARROW-6011: -- Summary: Data incomplete when using pyarrow in pyspark in python 3.x Key: ARROW-6011 URL: https://issues.apache.org/jira/browse/ARROW-6011 Project: Apache Arrow Issue T