and Regards
Aniruddh
-- Forwarded message --
From: Aniruddh Sharma asharma...@gmail.com
Date: Fri, Jul 17, 2015 at 6:05 PM
Subject: Re: Problem in Understanding concept of Physical Cores
To: user user@spark.apache.org
Dear Community
Request to help on below queries
Dear Community
Request to help on below queries they are unanswered.
Thanks and Regards
Aniruddh
On Wed, Jul 15, 2015 at 12:37 PM, Aniruddh Sharma asharma...@gmail.com
wrote:
Hi TD,
Request your guidance on below 5 queries. Following is the context of them
that I would use to evaluate
Hi TD,
Request your guidance on below 5 queries. Following is the context of them
that I would use to evaluate based on your response.
a) I need to decide whether to deploy Spark in Standalone mode or in Yarn.
But it seems to me that Spark in Yarn is more parallel than Standalone mode
(given
Hi TD,
Thanks for elaboration. I have further doubts based on further test that I
did after your guidance
Case 1: Standalone Spark--
In standalone mode, as you explained,master in spark-submit local[*]
implicitly, so it uses as creates threads as the number of cores that VM
has, but User can
Query 1) What spark runs is tasks in task slots, whatever is the mapping ot
tasks to physical cores it does not matter. If there are two task slots (2
threads in local mode, or an executor with 2 task slots in distributed
mode), it can only run two tasks concurrently. That is true even if the
task
Hi
I am new to Spark. Following is the problem that I am facing
Test 1) I ran a VM on CDH distribution with only 1 core allocated to it and
I ran simple Streaming example in spark-shell with sending data on
port and trying to read it. With 1 core allocated to this nothing happens
in my
There are several levels of indirection going on here, let me clarify.
In the local mode, Spark runs tasks (which includes receivers) using the
number of threads defined in the master (either local, or local[2], or
local[*]).
local or local[1] = single thread, so only one task at a time
local[2]
