Disabling distributing local conf file during spark-submit

Hello, It looks like local conf archives always get copied to the target (HDFS) every time a job is submitted 1. Other

Re: Encrypting jobs submitted by the client

t; > On 1 Feb 2016, at 20:48, eugene miretsky <eugene.miret...@gmail.com> > wrote: > > > > Spark supports client authentication via shared secret or kerberos (on > YARN). However, the job itself is sent unencrypted over the network. Is > there a way to encrypt the

Encrypting jobs submitted by the client

Spark supports client authentication via shared secret or kerberos (on YARN). However, the job itself is sent unencrypted over the network. Is there a way to encrypt the jobs the client submits to cluster? The rational for this is very similar to encrypting the HTTP file server traffic - Jars

Secure multi tenancy on in stand alone mode

When having multiple users sharing the same Spark cluster, it's a good idea to isolate the users - make sure that each users runs under a different Linux account and prevent them from accessing data in jobs submitted by other users. Is it currently possible to do with Spark? The only thing I

Kafka consumer: Upgrading to use the the new Java Consumer

Hi, The Kafka connector currently uses the older Kafka Scala consumer. Kafka 0.9 came out with a new Java Kafka consumer. One of the main differences is that the Scala consumer uses a Decoder( kafka.serializer.decoder) trait to decode keys/values while the Java consumer uses the Deserializer