Re: Making a bugfix 0.11.1 release

2018-10-18 Thread Krisztián Szűcs
I've added the two zlib issues to 0.11.1 version: https://issues.apache.org/jira/projects/ARROW/versions/12344316 On Wed, Oct 17, 2018 at 10:51 PM Wes McKinney wrote: > Got it, thank you for clarifying. It wasn't clear whether the bug > would occur in the build environment (CentOS 5 + devtoolset

[jira] [Created] (ARROW-3550) [C++] Use kUnknownNullCount in NumericArray constructor

2018-10-18 Thread Wolf Vollprecht (JIRA)
Wolf Vollprecht created ARROW-3550: -- Summary: [C++] Use kUnknownNullCount in NumericArray constructor Key: ARROW-3550 URL: https://issues.apache.org/jira/browse/ARROW-3550 Project: Apache Arrow

[jira] [Created] (ARROW-3551) Change MapD to OmniSci on Powered By page

2018-10-18 Thread Todd Mostak (JIRA)
Todd Mostak created ARROW-3551: -- Summary: Change MapD to OmniSci on Powered By page Key: ARROW-3551 URL: https://issues.apache.org/jira/browse/ARROW-3551 Project: Apache Arrow Issue Type: Improv

Re: Making a bugfix 0.11.1 release

2018-10-18 Thread Kevin Gurney
Hi All, We are working with the arrow version 0.9.0 C++ libraries in conjunction with separate parquet-cpp version 1.4.0. Questions: 1. Does this zlib issue affect all clients of the arrow C++ libraries or just the Python PyArrow code? 2. Does this zlib compression issue also affect the

[jira] [Created] (ARROW-3552) [Python] Implement pa.RecordBatch.serialize_to to write single message to an OutputStream

2018-10-18 Thread Wes McKinney (JIRA)
Wes McKinney created ARROW-3552: --- Summary: [Python] Implement pa.RecordBatch.serialize_to to write single message to an OutputStream Key: ARROW-3552 URL: https://issues.apache.org/jira/browse/ARROW-3552

Re: Making a bugfix 0.11.1 release

2018-10-18 Thread Antoine Pitrou
Le 18/10/2018 à 15:44, Kevin Gurney a écrit : > Hi All, > > We are working with the arrow version 0.9.0 C++ libraries in conjunction with > separate parquet-cpp version 1.4.0. > > Questions: > > 1. Does this zlib issue affect all clients of the arrow C++ libraries or > just the Python PyA

[jira] [Created] (ARROW-3553) [R] Error when losing data on int64, uint64 conversions to double

2018-10-18 Thread Wes McKinney (JIRA)
Wes McKinney created ARROW-3553: --- Summary: [R] Error when losing data on int64, uint64 conversions to double Key: ARROW-3553 URL: https://issues.apache.org/jira/browse/ARROW-3553 Project: Apache Arrow

Re: Making a bugfix 0.11.1 release

2018-10-18 Thread Kevin Gurney
Hi Antoine, Thanks for the quick response! This helps to clear up my confusion. Best Regards, Kevin Gurney From: Antoine Pitrou Sent: Thursday, October 18, 2018 9:54:47 AM To: dev@arrow.apache.org Subject: Re: Making a bugfix 0.11.1 release Le 18/10/2018

[jira] [Created] (ARROW-3554) [C++] Reverse traits for C++

2018-10-18 Thread Wolf Vollprecht (JIRA)
Wolf Vollprecht created ARROW-3554: -- Summary: [C++] Reverse traits for C++ Key: ARROW-3554 URL: https://issues.apache.org/jira/browse/ARROW-3554 Project: Apache Arrow Issue Type: Improvement

Support for TIMESTAMP_NANOS in parquet-cpp

2018-10-18 Thread Roman Karlstetter
Hi everyone, in parquet-format, there is now support for TIMESTAMP_NANOS: https://github.com/apache/parquet-format/pull/102 For parquet-cpp, this is not yet supported. I have a few questions now: • is there an overview of what release of parquet-format is currently fully support in parquet-cpp (

[jira] [Created] (ARROW-3555) [Plasma] Unify plasma client get function using metadata.

2018-10-18 Thread Yuhong Guo (JIRA)
Yuhong Guo created ARROW-3555: - Summary: [Plasma] Unify plasma client get function using metadata. Key: ARROW-3555 URL: https://issues.apache.org/jira/browse/ARROW-3555 Project: Apache Arrow Issu

[jira] [Created] (ARROW-3556) [CI] Disable optimizations on Windows

2018-10-18 Thread Antoine Pitrou (JIRA)
Antoine Pitrou created ARROW-3556: - Summary: [CI] Disable optimizations on Windows Key: ARROW-3556 URL: https://issues.apache.org/jira/browse/ARROW-3556 Project: Apache Arrow Issue Type: Impr

[jira] [Created] (ARROW-3557) [Python] Set language_level in Cython sources

2018-10-18 Thread Antoine Pitrou (JIRA)
Antoine Pitrou created ARROW-3557: - Summary: [Python] Set language_level in Cython sources Key: ARROW-3557 URL: https://issues.apache.org/jira/browse/ARROW-3557 Project: Apache Arrow Issue Ty

[RESULT] [VOTE] Accept donation of Arrow C# .NET implementation

2018-10-18 Thread Wes McKinney
hi folks With 6 binding +1 votes and 2 non-binding +1, the vote carries. I will work with the contributors to complete the IP clearance process On Tue, Oct 16, 2018 at 10:07 PM Andrew Brust wrote: > > +1 > > -Original Message- > From: Wes McKinney > Sent: Monday, October 15, 2018 11:28

[VOTE] Accept donation of Ruby bindings to Parquet GLib

2018-10-18 Thread Wes McKinney
hello, Kouhei Sutou is proposing to donate Ruby bindings to the Parquet GLib library, which was received as a donation in September. This Ruby library was originally developed at https://github.com/red-data-tools/red-parquet/ Kou has submitted the work as a pull request https://github.com/apache

Re: [VOTE] Accept donation of Ruby bindings to Parquet GLib

2018-10-18 Thread Uwe L. Korn
+1 > Am 18.10.2018 um 22:59 schrieb Wes McKinney : > > hello, > > Kouhei Sutou is proposing to donate Ruby bindings to the Parquet GLib > library, which was received as a donation in September. This Ruby > library was originally developed at > > https://github.com/red-data-tools/red-parquet/ >

[jira] [Created] (ARROW-3558) Remove fatal error when plasma client calls get on an unsealed object that it created.

2018-10-18 Thread Robert Nishihara (JIRA)
Robert Nishihara created ARROW-3558: --- Summary: Remove fatal error when plasma client calls get on an unsealed object that it created. Key: ARROW-3558 URL: https://issues.apache.org/jira/browse/ARROW-3558

Re: [VOTE] Accept donation of Ruby bindings to Parquet GLib

2018-10-18 Thread Kouhei Sutou
+1 In "[VOTE] Accept donation of Ruby bindings to Parquet GLib" on Thu, 18 Oct 2018 16:59:41 -0400, Wes McKinney wrote: > hello, > > Kouhei Sutou is proposing to donate Ruby bindings to the Parquet GLib > library, which was received as a donation in September. This Ruby > library was orig

[jira] [Created] (ARROW-3559) Statically link libraries for plasma_store_server executable.

2018-10-18 Thread Robert Nishihara (JIRA)
Robert Nishihara created ARROW-3559: --- Summary: Statically link libraries for plasma_store_server executable. Key: ARROW-3559 URL: https://issues.apache.org/jira/browse/ARROW-3559 Project: Apache Arr

Efficient Pandas serialization for mixed object and numeric DataFrames

2018-10-18 Thread Mitar
Hi! It seems that if a DataFrame contains both numeric and object columns, the whole DataFrame is pickled and not that only object columns are pickled? Is this right? Are there any plans to improve this? Mitar -- http://mitar.tnode.com/ https://twitter.com/mitar_m

Re: Efficient Pandas serialization for mixed object and numeric DataFrames

2018-10-18 Thread Robert Nishihara
How are you serializing the dataframe? If you use *pyarrow.serialize(df)*, then each column should be serialized separately and numeric columns will be handled efficiently. On Thu, Oct 18, 2018 at 9:10 PM Mitar wrote: > Hi! > > It seems that if a DataFrame contains both numeric and object column