Re: Use Spark extension points to implement row-level security

Hi, I've added table level security using spark extensions based on the ongoing work proposed for ranger in RANGER-2128. Following the same logic, you could mask columns and work on the logical plan, but not filtering or skipping rows, as those are not present in these hooks. The only difficult

Dynamic Allocation not removing executors

Hi, I found an issue trying to use dynamic allocation in 2.3.1 where the driver does not remove idle executors under some circunstances. For the first instance of this happening, it seems that a change introduced in 2.2.1/2.3.0 (SPARK-21656 )

Security in pyspark using extensions

Hi, I'm trying to build an authorization/security extension for spark using the hooks provided in SPARK-18127 ( https://issues.apache.org/jira/browse/SPARK-18127). The problem I've encountered is that those hooks aren't available for pyspark, as the exensions are loaded in the getOrCreate method

Re: Problems initializing SparkUI

relying on this email's technical content is explicitly disclaimed. > The author will in no case be liable for any monetary damages arising from > such loss, damage or destruction. > > > > On 1 August 2016 at 22:03, Maximiliano Patricio Méndez < > mmen...@despegar.c

Re: Problems initializing SparkUI

rise > from relying on this email's technical content is explicitly disclaimed. > The author will in no case be liable for any monetary damages arising from > such loss, damage or destruction. > > > > On 1 August 2016 at 21:23, Maximiliano Patricio Méndez < > mmen...@despe

Re: Problems initializing SparkUI

hor will in no case be liable for any monetary damages arising from > such loss, damage or destruction. > > > > On 1 August 2016 at 19:08, Maximiliano Patricio Méndez < > mmen...@despegar.com> wrote: > >> I just recently tried again, the port 4040 is not used

Re: Problems initializing SparkUI

f data or any other property which may arise > from relying on this email's technical content is explicitly disclaimed. > The author will in no case be liable for any monetary damages arising from > such loss, damage or destruction. > > > > On 1 August 2016 at 17:44, Maximiliano Patrici

Re: Problems initializing SparkUI

-- >> https://medium.com/@jaceklaskowski/ >> Mastering Apache Spark http://bit.ly/mastering-apache-spark >> Follow me at https://twitter.com/jaceklaskowski >> >> >> On Thu, Jul 28, 2016 at 11:37 PM, Maximiliano Patricio Méndez >> <mmen...@despegar.com> wro