Dataframes: PrunedFilteredScan without Spark Side Filtering

2015-09-25 Thread Russell Spitzer
Hi! First time poster, long time reader. I'm wondering if there is a way to let cataylst know that it doesn't need to repeat a filter on the spark side after a filter has been applied by the Source Implementing PrunedFilterScan. This is for a usecase in which we except a filter on a

Re: [VOTE] Release Apache Spark 1.5.1 (RC1)

2015-09-25 Thread vaquar khan
+1 (non-binding) Regards, Vaquar khan On 25 Sep 2015 18:28, "Eugene Zhulenev" wrote: > +1 > > Running latest build from 1.5 branch, SO much more stable than 1.5.0 > release. > > On Fri, Sep 25, 2015 at 8:55 AM, Doug Balog > wrote: > >> +1

Re: [VOTE] Release Apache Spark 1.5.1 (RC1)

2015-09-25 Thread Tom Graves
+1. Tested Spark on Yarn on Hadoop 2.6 and 2.7. Tom On Thursday, September 24, 2015 2:34 AM, Reynold Xin wrote: Please vote on releasing the following candidate as Apache Spark version 1.5.1. The vote is open until Sun, Sep 27, 2015 at 10:00 UTC and passes if

Re: [VOTE] Release Apache Spark 1.5.1 (RC1)

2015-09-25 Thread Marcelo Vanzin
Ignoring my previous question, +1. Tested several different jobs on YARN and standalone with dynamic allocation on. On Fri, Sep 25, 2015 at 11:32 AM, Marcelo Vanzin wrote: > Mostly for my education (I hope), but I was testing > "spark-1.5.1-bin-without-hadoop.tgz" assuming

Re: [VOTE] Release Apache Spark 1.5.1 (RC1)

2015-09-25 Thread Marcelo Vanzin
Mostly for my education (I hope), but I was testing "spark-1.5.1-bin-without-hadoop.tgz" assuming it would contain everything (including HiveContext support), just without the Hadoop common jars in the assembly. But HiveContext is not there. Is this expected? On Thu, Sep 24, 2015 at 12:27 AM,

Re: RFC: packaging Spark without assemblies

2015-09-25 Thread Marcelo Vanzin
On Wed, Sep 23, 2015 at 4:43 PM, Patrick Wendell wrote: > For me a key step in moving away would be to fully audit/understand > all compatibility implications of removing it. If other people are > supportive of this plan I can offer to help spend some time thinking > about any

Re: [Discuss] NOTICE file for transitive "NOTICE"s

2015-09-25 Thread Steve Loughran
> On 24 Sep 2015, at 21:11, Sean Owen wrote: > > Yes, but the ASF's reading seems to be clear: > http://www.apache.org/dev/licensing-howto.html#permissive-deps > "In LICENSE, add a pointer to the dependency's license within the > source tree and a short note summarizing its

Re: [Discuss] NOTICE file for transitive "NOTICE"s

2015-09-25 Thread Sean Owen
Update: I *think* the conclusion was indeed that nothing needs to happen with NOTICE. However, along the way in https://issues.apache.org/jira/browse/LEGAL-226 it emerged that the BSD/MIT licenses should be inlined into LICENSE (or copied in the distro somewhere). I can get on that -- just some

Re: unsubscribe

2015-09-25 Thread Akhil Das
Send an email to dev-unsubscr...@spark.apache.org instead of dev@spark.apache.org Thanks Best Regards On Fri, Sep 25, 2015 at 4:00 PM, Nirmal R Kumar wrote: >

unsubscribe

2015-09-25 Thread Nirmal R Kumar

Re: [Discuss] NOTICE file for transitive "NOTICE"s

2015-09-25 Thread Sean Owen
On Fri, Sep 25, 2015 at 10:06 AM, Steve Loughran wrote: > regarding the spark one, I don't see that you need to refer to transitive > dependencies for the non-binary distros, and, for any binaries, to bother > listing the licensing of all the ASF dependencies. Things