Hi all,
Does anyone know of any effort from the community on security testing spark
clusters.
I.e.
Static source code analysis to find security flaws
Penetration testing to identify ways to compromise spark cluster
Fuzzing to crash spark
Thanks,
Judy
Hi everyone,
Found a thrift server reliability issue on spark 1.3.1 that causes thrift to
fail.
When thrift server has too little memory allocated to the driver to process the
request, its Spark SQL session exits with OutOfMemory exception, causing thrift
server to stop working.
Is this a
account in this case). With this fix,
slave will be able to run without admin permission.
FYI: master thrift server works fine with only user permission, so no issue
there.
From: Judy Nash [mailto:judyn...@exchange.microsoft.com]
Sent: Thursday, February 19, 2015 12:26 AM
To: Akhil Das; dev
@01D04BDA.A74C65E0]
From: Akhil Das [mailto:ak...@sigmoidanalytics.com]
Sent: Wednesday, February 18, 2015 10:40 PM
To: Judy Nash
Cc: u...@spark.apache.org
Subject: Re: spark slave cannot execute without admin permission on windows
You need not require admin permission, but just make sure all those jars has
Hello,
Working on SPARK-5708https://issues.apache.org/jira/browse/SPARK-5708 - Add
Slf4jSink to Spark Metrics Sink.
Wrote a new Slf4jSink class (see patch attached), but the new class is not
packaged as part of spark-assembly jar.
Do I need to update build config somewhere to have this
Thanks Patrick! That was the issue.
Built the jars on windows env with mvn and forgot to run make-distributions.ps1
afterward, so was looking at old jars.
From: Patrick Wendell [mailto:pwend...@gmail.com]
Sent: Monday, February 9, 2015 10:43 PM
To: Judy Nash
Cc: dev@spark.apache.org
Subject: Re
Thanks Josh. That was the issue.
From: Josh Rosen [mailto:rosenvi...@gmail.com]
Sent: Friday, December 5, 2014 3:21 PM
To: Judy Nash; dev@spark.apache.org
Subject: Re: build in IntelliJ IDEA
If you go to “File - Project Structure” and click on “Project” under the
“Project settings” heading, do
Hi everyone,
Have a newbie question on using IntelliJ to build and debug.
I followed this wiki to setup IntelliJ:
https://cwiki.apache.org/confluence/display/SPARK/Useful+Developer+Tools#UsefulDeveloperTools-BuildingSparkinIntelliJIDEA
Afterward I tried to build via Toolbar (Build Rebuild
