[jira] [Created] (ARROW-1439) [Packaging] Automate updating RPM in RPM build

2017-08-30 Thread Kouhei Sutou (JIRA)
Kouhei Sutou created ARROW-1439: --- Summary: [Packaging] Automate updating RPM in RPM build Key: ARROW-1439 URL: https://issues.apache.org/jira/browse/ARROW-1439 Project: Apache Arrow Issue

Re: Apache Arrow at JupyterCon

2017-08-30 Thread Ivan Sadikov
Great presentation! Thank you for sharing. On Thu, 31 Aug 2017 at 8:02 AM, Wes McKinney wrote: > Absolutely. I will do that now > > On Wed, Aug 30, 2017 at 3:33 PM, Julian Hyde wrote: > > Thanks for sharing. Can we tweet those videos as well? I see that

Re: Apache Arrow at JupyterCon

2017-08-30 Thread Wes McKinney
Absolutely. I will do that now On Wed, Aug 30, 2017 at 3:33 PM, Julian Hyde wrote: > Thanks for sharing. Can we tweet those videos as well? I see that > https://twitter.com/apachearrow only > tweeted your slides. > >> On Aug 26, 2017, at 1:11

[jira] [Created] (ARROW-1436) PyArrow Timestamps written to Parquet as INT96 appear in Spark as 'bigint'

2017-08-30 Thread Lucas Pickup (JIRA)
Lucas Pickup created ARROW-1436: --- Summary: PyArrow Timestamps written to Parquet as INT96 appear in Spark as 'bigint' Key: ARROW-1436 URL: https://issues.apache.org/jira/browse/ARROW-1436 Project:

[jira] [Created] (ARROW-1435) PyArrow not propagating timezone information from Parquet to Pyhon

2017-08-30 Thread Lucas Pickup (JIRA)
Lucas Pickup created ARROW-1435: --- Summary: PyArrow not propagating timezone information from Parquet to Pyhon Key: ARROW-1435 URL: https://issues.apache.org/jira/browse/ARROW-1435 Project: Apache Arrow

Re: Apache Arrow at JupyterCon

2017-08-30 Thread Julian Hyde
Thanks for sharing. Can we tweet those videos as well? I see that https://twitter.com/apachearrow only tweeted your slides. > On Aug 26, 2017, at 1:11 PM, Wes McKinney wrote: > > hi all, > > In case folks here are interested, I gave a

Re: PyArrow not retaining Parquet metadata

2017-08-30 Thread Wes McKinney
hi Lucas, Can you open a JIRA with this information on https://issues.apache.org/jira so we can investigate and resolve there in case a patch is required? No need to post a reply-to e-mail address -- development discussions should stay on public channels like the mailing list or JIRA. Thanks

RE: PyArrow not retaining Parquet metadata

2017-08-30 Thread Lucas Pickup
Please reply to: lucas.pic...@microsoft.com Outlook isn't playing nice. Apologies, Lucas Pickup -Original Message- From: Lucas Pickup [mailto:lucas.pic...@microsoft.com.INVALID] Sent: Wednesday, August 30, 2017 10:47 AM To: dev@arrow.apache.org Subject: PyArrow not retaining Parquet

PyArrow not retaining Parquet metadata

2017-08-30 Thread Lucas Pickup
Hi All, I've encounter an issue where PyArrow does not appear to be propagating datetime metadata from parquet files into the resuling python objects. λ python Python 3.5.2 |Continuum Analytics, Inc.| (default, Jul 5 2016, 11:41:13) [MSC v.1900 64 bit (AMD64)] on win32 Type "help",

[jira] [Created] (ARROW-1434) [C++/Python] pyarrow.Array.from_pandas does not support datetime64[D] arrays

2017-08-30 Thread Uwe L. Korn (JIRA)
Uwe L. Korn created ARROW-1434: -- Summary: [C++/Python] pyarrow.Array.from_pandas does not support datetime64[D] arrays Key: ARROW-1434 URL: https://issues.apache.org/jira/browse/ARROW-1434 Project:

how to use arrow in project?

2017-08-30 Thread big data
I want to use Arrow as a middle layer between spark and parquet data in HDFS, but I don't find any docs about how to load parquet data to arrow in memory, and how spark read arrow data format. Does anyone provide some examples or manuals to describe it? thanks.