Re: the default value for param "mapreduce.reduce.shuffle.input.buffer.percent"

2014-06-25 Thread wzc
hi Sudhakara: Thank you for your reply. I've lost the old job information which I thought "mapreduce.reduce.shuffle.input.buffer.percent" was set to 0.9 in. I cant make it to read hard coded value from MergeManagerImpl.java. So maybe I made a mistake. Thanks. 2014-05-17 1:45 GMT+08:00 sudhakar

Re: persisent services in Hadoop

2014-06-25 Thread Arun Murthy
John, We are excited to see ISVs like you get value from YARN, and appreciate the patience you've already shown in the past to work through the teething issues of YARN & hadoop-2.x. W.r.t long-running services, the most straight-forward option is to go through Apache Slider (http://slider.incub

MapReduce Streaming on Solaris

2014-06-25 Thread Rich Haase
Hi all, I have a 20 node cluster that is running on Solaris x86 (OpenIndian). I'm not really familiar with OpenIndiana having moved from Solaris to Linux many years ago, but it's the OS of choice for the systems administrator at my company. Each worker has 24 700xGB drives, 24 cores and 96 GB of

persisent services in Hadoop

2014-06-25 Thread John Lilley
We are an ISV that currently ships a data-quality/integration suite running as a native YARN application. We are finding several use cases that would benefit from being able to manage a per-node persistent service. MapReduce has its "shuffle auxiliary service", but it isn't straightforward to

group similar items using pairwise similar items

2014-06-25 Thread parnab kumar
Hi, I have a set of items and a pairwise similar items. I want to group together items that are mutually similar. For ex : if *A B C D E F G* are the items I have the following pairwise similar items *A B* *A C* *B C * *D E * *C G* *E F* I want the output as *A B C G* *D E F* Can someone su

Splitting map and reduce

2014-06-25 Thread Chris MacKenzie
Hi, This is my first mail to this user group. I hope that the email is well formed and enables me to learn a great deal about Hadoop. I have to carry out sequence alignment using Hadoop with the aid of a critical subsequence. A potential critical subsequence is derived from the longest unique sub