RE: Going it alone.

[seemanto.barua]

Have we been tricked by a bot ?

From: Matt Smith 
Sent: Wednesday, April 15, 2020 2:23 PM
To: jane thorpe
Cc: dh.lo...@gmail.com; user@spark.apache.org; janethor...@aol.com; 
em...@yeikel.com
Subject: Re: Going it alone.

CAUTION EXTERNAL EMAIL: DO NOT CLICK ON LINKS OR OPEN ATTACHMENTS THAT ARE 
UNEXPECTED OR SENT FROM UNKNOWN SENDERS. IF IN DOUBT REPORT TO SPAM SUBMISSIONS.

This is so entertaining.

1. Ask for help
2. Compare those you need help from to a lower order primate.
3. Claim you provided information you did not
4. Explain that providing any information would be "too revealing"
5. ???

Can't wait to hear what comes next, but please keep it up.  This is a bright 
spot in my day.


On Tue, Apr 14, 2020 at 4:47 PM jane thorpe 
mailto:janethor...@aol.com.invalid>> wrote:

I did write a long email in response to you.
But then I deleted it because I felt it would be too revealing.






On Tuesday, 14 April 2020 David Hesson 
mailto:dh.lo...@gmail.com>> wrote:
I want to know  if Spark is headed in my direction.
You are implying  Spark could be.

What direction are you headed in, exactly? I don't feel as if anything were 
implied when you were asked for use cases or what problem you are solving. You 
were asked to identify some use cases, of which you don't appear to have any.

On Tue, Apr 14, 2020 at 4:49 PM jane thorpe 
mailto:janethor...@aol.com.invalid>> wrote:

That's what  I want to know,  Use Cases.
I am looking for  direction as I described and I want to know  if Spark is 
headed in my direction.

You are implying  Spark could be.

So tell me about the USE CASES and I'll do the rest.


On Tuesday, 14 April 2020 yeikel valdes 
mailto:em...@yeikel.com>> wrote:
It depends on your use case. What are you trying to solve?


 On Tue, 14 Apr 2020 15:36:50 -0400 
janethor...@aol.com.INVALID wrote 

Hi,

I consider myself to be quite good in Software Development especially using 
frameworks.

I like to get my hands  dirty. I have spent the last few months understanding 
modern frameworks and architectures.

I am looking to invest my energy in a product where I don't have to relying on 
the monkeys which occupy this space  we call software development.

I have found one that meets my requirements.

Would Apache Spark be a good Tool for me or  do I need to be a member of a team 
to develop  products  using Apache Spark  ?




PLEASE READ: This message is for the named person's use only. It may contain 
confidential, proprietary or legally privileged information. No confidentiality 
or privilege is waived or lost by any mistransmission. If you receive this 
message in error, please delete it and all copies from your system, destroy any 
hard copies and notify the sender. You must not, directly or indirectly, use, 
disclose, distribute, print, or copy any part of this message if you are not 
the intended recipient. Nomura Holding America Inc., Nomura Securities 
International, Inc, and their respective subsidiaries each reserve the right to 
monitor all e-mail communications through its networks. Any views expressed in 
this message are those of the individual sender, except where the message 
states otherwise and the sender is authorized to state the views of such 
entity. Unless otherwise stated, any pricing information in this message is 
indicative only, is subject to change and does not constitute an offer to deal 
at any price quoted. Any reference to the terms of executed transactions should 
be treated as preliminary only and subject to our formal written confirmation.

Adding worker dynamically in standalone mode

[seemanto.barua]

Hi,

Is it possible to add a worker dynamically to the master in standalone mode. If 
so can you please share the steps on how to ?

-thanks
Seemanto Barua


PLEASE READ: This message is for the named person's use only. It may contain 
confidential, proprietary or legally privileged information. No confidentiality 
or privilege is waived or lost by any mistransmission. If you receive this 
message in error, please delete it and all copies from your system, destroy any 
hard copies and notify the sender. You must not, directly or indirectly, use, 
disclose, distribute, print, or copy any part of this message if you are not 
the intended recipient. Nomura Holding America Inc., Nomura Securities 
International, Inc, and their respective subsidiaries each reserve the right to 
monitor all e-mail communications through its networks. Any views expressed in 
this message are those of the individual sender, except where the message 
states otherwise and the sender is authorized to state the views of such 
entity. Unless otherwise stated, any pricing information in this message is 
indicative only, is subject to change and does not constitute an offer to deal 
at any price quoted. Any reference to the terms of executed transactions should 
be treated as preliminary only and subject to our formal written confirmation.

launching app using SparkLauncher

[seemanto.barua]

Hi,

I have a question on org.apache.spark.launcher.SparkLauncher

How is the JavaSparkContext made available to the main class.


-regards
Seemanto Barua




PLEASE READ: This message is for the named person's use only. It may contain 
confidential, proprietary or legally privileged information. No confidentiality 
or privilege is waived or lost by any mistransmission. If you receive this 
message in error, please delete it and all copies from your system, destroy any 
hard copies and notify the sender. You must not, directly or indirectly, use, 
disclose, distribute, print, or copy any part of this message if you are not 
the intended recipient. Nomura Holding America Inc., Nomura Securities 
International, Inc, and their respective subsidiaries each reserve the right to 
monitor all e-mail communications through its networks. Any views expressed in 
this message are those of the individual sender, except where the message 
states otherwise and the sender is authorized to state the views of such 
entity. Unless otherwise stated, any pricing information in this message is 
indicative only, is subject to change and does not constitute an offer to deal 
at any price quoted. Any reference to the terms of executed transactions should 
be treated as preliminary only and subject to our formal written confirmation.

RE: Question in rdd caching in memory using persist

[seemanto.barua]

Attached is the screen shot of the storage tab details for the cached rdd. The 
host highlighted and at the end of the list is driver machine.


[cid:image001.png@01D1496C.FD79AB40]

-regards
Seemanto Barua


From: Barua, Seemanto (US)
Sent: Thursday, January 07, 2016 12:43 PM
To: 'premsure...@gmail.com'
Cc: 'user@spark.apache.org'
Subject: Re: Question in rdd caching in memory using persist

I have a standalone cluster. spark version is 1.3.1


From: Prem Sure [mailto:premsure...@gmail.com]
Sent: Thursday, January 07, 2016 12:32 PM
To: Barua, Seemanto (US)
Cc: spark users >
Subject: Re: Question in rdd caching in memory using persist

are you running standalone - local mode or cluster mode. executor and driver 
existance differ based on setup type. snapshot of your env UI would be helpful 
to say

On Thu, Jan 7, 2016 at 11:51 AM, 
> wrote:
Hi,


After I called rdd.persist(MEMORY_ONLY_SER), I see the driver listed as one of 
the ‘executors’ participating in holding the partitions of the rdd in memory, 
the memory usage shown against the driver is 0. This I see in the storage tab 
of the spark ui.
Why is the driver shown on the ui ? Will it ever hold rdd partitions when 
caching.


-regards
Seemanto Barua



PLEASE READ: This message is for the named person's use only. It may contain 
confidential, proprietary or legally privileged information. No confidentiality 
or privilege is waived or lost by any mistransmission. If you receive this 
message in error, please delete it and all copies from your system, destroy any 
hard copies and notify the sender. You must not, directly or indirectly, use, 
disclose, distribute, print, or copy any part of this message if you are not 
the intended recipient. Nomura Holding America Inc., Nomura Securities 
International, Inc, and their respective subsidiaries each reserve the right to 
monitor all e-mail communications through its networks. Any views expressed in 
this message are those of the individual sender, except where the message 
states otherwise and the sender is authorized to state the views of such 
entity. Unless otherwise stated, any pricing information in this message is 
indicative only, is subject to change and does not constitute an offer to deal 
at any price quoted. Any reference to the terms of executed transactions should 
be treated as preliminary only and subject to our formal written confirmation.



PLEASE READ: This message is for the named person's use only. It may contain 
confidential, proprietary or legally privileged information. No confidentiality 
or privilege is waived or lost by any mistransmission. If you receive this 
message in error, please delete it and all copies from your system, destroy any 
hard copies and notify the sender. You must not, directly or indirectly, use, 
disclose, distribute, print, or copy any part of this message if you are not 
the intended recipient. Nomura Holding America Inc., Nomura Securities 
International, Inc, and their respective subsidiaries each reserve the right to 
monitor all e-mail communications through its networks. Any views expressed in 
this message are those of the individual sender, except where the message 
states otherwise and the sender is authorized to state the views of such 
entity. Unless otherwise stated, any pricing information in this message is 
indicative only, is subject to change and does not constitute an offer to deal 
at any price quoted. Any reference to the terms of executed transactions should 
be treated as preliminary only and subject to our formal written confirmation.