Re: Use Arrow instead of Pickle without pandas_udf

2018-07-31 Thread Hichame El Khalfi
Thanks Bryan for the pointer +1 Hichame From: cutl...@gmail.com Sent: July 30, 2018 6:40 PM To: hich...@elkhalfi.com Cc: hol...@pigscanfly.ca; user@spark.apache.org Subject: Re: Use Arrow instead of Pickle without pandas_udf Here is a link to the JIRA for adding StructType support for scalar

Re: Use Arrow instead of Pickle without pandas_udf

2018-07-30 Thread Bryan Cutler
...@elkhalfi.com > *Cc:* user@spark.apache.org > *Subject:* Re: Use Arrow instead of Pickle without pandas_udf > > Not currently. What's the problem with pandas_udf for your use case? > > On Wed, Jul 25, 2018 at 1:27 PM, Hichame El Khalfi > wrote: > >> Hi There, >&g

Re: Use Arrow instead of Pickle without pandas_udf

2018-07-25 Thread Hichame El Khalfi
PM To: hich...@elkhalfi.com Cc: user@spark.apache.org Subject: Re: Use Arrow instead of Pickle without pandas_udf Not currently. What's the problem with pandas_udf for your use case? On Wed, Jul 25, 2018 at 1:27 PM, Hichame El Khalfi mailto:hich...@elkhalfi.com>> wrot

Re: Use Arrow instead of Pickle without pandas_udf

2018-07-25 Thread Holden Karau
Not currently. What's the problem with pandas_udf for your use case? On Wed, Jul 25, 2018 at 1:27 PM, Hichame El Khalfi wrote: > Hi There, > > > Is there a way to use Arrow format instead of Pickle but without using > pandas_udf ? > > > Thank for your help, > > > Hichame > -- Twitter:

Use Arrow instead of Pickle without pandas_udf

2018-07-25 Thread Hichame El Khalfi
Hi There, Is there a way to use Arrow format instead of Pickle but without using pandas_udf ? Thank for your help, Hichame