Re: [DISCUSS] Function plugins

2018-12-18 Thread Ryan Blue
row formats is worthwhile if >> only because it’s easier to use than Hive UDFs. >> >> >> >> -Matt Cheah >> >> >> >> *From: *Reynold Xin >> *Date: *Friday, December 14, 2018 at 1:49 PM >> *To: *"rb...@netflix.com" >>

Re: [DISCUSS] Function plugins

2018-12-14 Thread Reynold Xin
From: *Reynold Xin > *Date: *Friday, December 14, 2018 at 1:49 PM > *To: *"rb...@netflix.com" > *Cc: *Spark Dev List > *Subject: *Re: [DISCUSS] Function plugins > > > > [image: Image removed by sender.] > > Having a way to register UDFs that are not u

Re: [DISCUSS] Function plugins

2018-12-14 Thread Matt Cheah
than Hive UDFs. -Matt Cheah From: Reynold Xin Date: Friday, December 14, 2018 at 1:49 PM To: "rb...@netflix.com" Cc: Spark Dev List Subject: Re: [DISCUSS] Function plugins Having a way to register UDFs that are not using Hive APIs would be great! On Fri, Dec 14,

Re: [DISCUSS] Function plugins

2018-12-14 Thread Reynold Xin
Having a way to register UDFs that are not using Hive APIs would be great! On Fri, Dec 14, 2018 at 1:30 PM, Ryan Blue < rb...@netflix.com.invalid > wrote: > > > > Hi everyone, > I’ve been looking into improving how users of our Spark platform register > and use UDFs and I’d like to discuss a

[DISCUSS] Function plugins

2018-12-14 Thread Ryan Blue
Hi everyone, I’ve been looking into improving how users of our Spark platform register and use UDFs and I’d like to discuss a few ideas for making this easier. The motivation for this is the use case of defining a UDF from SparkSQL or PySpark. We want to make it easy to write JVM UDFs and use