Unsubscribe

[Chen Qiming]

Unsubscribe

unsubscribe

[Bandi, Saidi R]

Unsubscribe

Regards,
Saidi

The information contained in this message is proprietary and/or confidential. 
If you are not the intended recipient, please: (i) delete the message and all 
copies; (ii) do not disclose, distribute or use the message in any manner; and 
(iii) notify the sender immediately. In addition, please be aware that any 
message addressed to our domain is subject to archiving and review by persons 
other than the intended recipient. Thank you.

Re: how to implement checkpointing and fault tolerance based fault tolerance in hadoop

[Hassan Asghar]

Thank you for the clarification. i am talking about fault toldrance in map
reduce, is there any algorithm implement in it?? For Checkpointing and
Replication???

On Mon, 26 Jun 2017 at 3:32 AM, Jasson Chenwei 
wrote:

> hi, Hassan.
>
> First, YARN( the scheduler) doesn't provide any fault tolerance
> techniques, But applications(e.g., MapReduce or Spark) do.
>
> For MapReduce, its fault tolerance is based on speculative execution,
> which simply re-launched failed tasks.
> For Spark, it does provide checkpoint API which users can leverage in
> their code by designating which RDDs should be checkpointed at what time.
> For more details about Spark checkpoint, you can refer to Spark docs. If
> checkpoint is not enabled, it fails back to speculative execution, the same
> as MapReduce.
>
>
>
> Wei
>
> On Sat, Jun 24, 2017 at 8:44 PM, Hassan Asghar 
> wrote:
>
>> Dear users,
>>
>> I am performing a comparative study on different fault tolerance
>> techniques, so, my question is that, how can we implement checkpointing
>> based and replication based fault tolerance in hadoop, is there any patch
>> already implement so that i can use that in my hadoop cluster??
>>
>>
>> Regards,
>> Hassan Asghar
>>
> -
>> To unsubscribe, e-mail: user-unsubscr...@hadoop.apache.org
>> For additional commands, e-mail: user-h...@hadoop.apache.org
>>
>>
> --
Dear,

Best Regards
--
Hassan Asghar
*M*:*+923400400374*
*E*: *haxxanasg...@gmail.com *

Re: Can hdfs client 2.6 read file of hadoop 2.7 ?

[Ravi Prakash]

Hi Jeff!

Yes. hadoop-2.6 clients are able to read files on a hadoop-2.7 cluster. The
document I could find is
http://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/Compatibility.html
.

"Both Client-Server and Server-Server compatibility is preserved within a
major release"

HTH
Ravi.

On Mon, Jun 26, 2017 at 5:21 AM, Jeff Zhang  wrote:

>
> It looks like it can. But is there any document about the compatibility
> between versions ? Thanks
>
>
>

Re: Restrict the number of container can be run in Parallel in Yarn?

[Akira Ajisaka]

Hi wuchang,

If you are using Hadoop 2.7+,
you can use the following parameters to limit the number of
simultaneously running map/reduce tasks per MapReduce application:

* mapreduce.job.running.map.limit (default: 0, for no limit)
* mapreduce.job.running.reduce.limit (default: 0, for no limit)

Regards,
Akira

On 2017/06/26 11:24, wuchang wrote:

For a map reduce task , I have restrict the memory of mapper and reducers by

mapreduce.map.memory.mb=1G
mapreduce.reduce.memory.mb=4G

But this parameter just restrict the memory for each mapper or reducer task 
,instead of the number of mappers or reducers which can be launched in 
parallel.when this task launched, it seems that it consumed about 50G memory , 
which is not what I want. I find that there are many containers launched in 
parallel。
What I want is to restrict the memory or cpu resource it can consume every 
moment, for example, the maximum memory of this application can use every 
moment is 10G.What can I do?


-
To unsubscribe, e-mail: user-unsubscr...@hadoop.apache.org
For additional commands, e-mail: user-h...@hadoop.apache.org

unsubscribe

[Venkatrama, Krishna]

unsubscribe

Krishna Venkatrama(KK)
9049052189
IT Shared Services
Sr. Information Architect
Hadoop Data Platform Architect

We comply with applicable Federal civil rights laws and do not discriminate on 
the basis of race, color, national origin, age, disability or sex. You may 
access the Non-Discrimination and Accessibility Notice here 
.

Language Assistance Available:

Español, Kreyol Ayisien, Tiếng Việt, Português, 中文, français, Tagalog, русский, 
italiano, Deutsche, 한국어, Polskie, Gujarati, ไทย, العربية, 日本語, فارسی 


Florida Blue is a trade name of Blue Cross and Blue Shield of Florida, Inc.  
Blue Cross and Blue Shield of Florida, Inc., and its subsidiary and affiliate 
companies are not responsible for errors or omissions in this e-mail message. 
Any personal comments made in this e-mail do not reflect the views of Blue 
Cross and Blue Shield of Florida, Inc.  The information contained in this 
document may be confidential and intended solely for the use of the individual 
or entity to whom it is addressed.  This document may contain material that is 
privileged or protected from disclosure under applicable law.  If you are not 
the intended recipient or the individual responsible for delivering to the 
intended recipient, please (1) be advised that any use, dissemination, 
forwarding, or copying of this document IS STRICTLY PROHIBITED; and (2) notify 
sender immediately by telephone and destroy the document. THANK YOU.

Can hdfs client 2.6 read file of hadoop 2.7 ?

[Jeff Zhang]

It looks like it can. But is there any document about the compatibility
between versions ? Thanks

unsubscribe

[Vitaliy Semochkin]

unsubscribe