Limit Query Performance Suggestion

2017-01-12 Thread sujith chacko
When limit is being added in the terminal of the physical plan there will be possibility of memory bottleneck if the limit value is too large and system will try to aggregate all the partition limit values as part of single partition. Description: Eg: create table src_temp as select * from src

Unsubscribe

2017-01-12 Thread anuj ojha
Unsubscribe

Re: [PYSPARK] Python tests organization

2017-01-12 Thread Maciej Szymkiewicz
Thanks Holden. If you have some spare time would you take a look at https://github.com/apache/spark/pull/16534? It is somewhat related to https://issues.apache.org/jira/browse/SPARK-18777 (Return UDF objects when registering from Python). On 01/12/2017 07:34 PM, Holden Karau wrote: > I'd be

Re: [PYSPARK] Python tests organization

2017-01-12 Thread Maciej Szymkiewicz
Sounds good, but it looks like JIRA is still down. Personally I can look at sql.tests and see what can be done there. Depending on the resolution of https://issues.apache.org/jira/browse/SPARK-19160 I may have to adjust some tests anyway. On 01/12/2017 07:36 PM, Saikat Kanjilal wrote: > >

Re: [PYSPARK] Python tests organization

2017-01-12 Thread Saikat Kanjilal
Maciej? LGTM, what do you think? I can create a JIRA and drive this. From: Holden Karau Sent: Thursday, January 12, 2017 10:34 AM To: Saikat Kanjilal; dev@spark.apache.org Subject: Re: [PYSPARK] Python tests organization I'd be happy to

Re: [PYSPARK] Python tests organization

2017-01-12 Thread Holden Karau
I'd be happy to help with reviewing Python test improvements. Maybe make an umbrella JIRA and do one sub components at a time? On Thu, Jan 12, 2017 at 12:20 PM Saikat Kanjilal wrote: > > > > > > > > > > > > > > > Following up, any thoughts on next steps for this? > > > > >

Re: [PYSPARK] Python tests organization

2017-01-12 Thread Saikat Kanjilal
Following up, any thoughts on next steps for this? From: Maciej Szymkiewicz Sent: Wednesday, January 11, 2017 10:14 AM To: Saikat Kanjilal Subject: Re: [PYSPARK] Python tests organization Not yet, I want to see if there is any consensus

Unsubscribe

2017-01-12 Thread williamtellme123
Please unsubscribe me From: Varanasi, Venkata [mailto:venkata.varan...@bankofamerica.com] Sent: Thursday, April 28, 2016 1:35 PM To: dev@spark.apache.org Subject: Unsubscribe _ This message, and any attachments, is for the intended recipient(s) only, may contain information

FOSDEM 2017 Open Source Conference - Brussels

2017-01-12 Thread Sharan F
Hello Everyone This email is to tell you about ASF participation at FOSDEM. The event will be held in Brussels on 4^th & 5^th February 2017 and we are hoping that many people from our ASF projects will be there. https://fosdem.org/2017/ Attending FOSDEM is completely free and the ASF will