Hi Hadoop user,
I have graph data file in the form of edge list
Source Vertex_id Sink Vertex_id
I want to assign each edge a unique ID. In the map function I emit
(key,value) as (Source Vertex_id, Sink Vertex_id)
In the reducer, for each value , I am using a combination of static count
You asked a similar question earlier also so I will copy those comments
here with what I replied then:
http://hadoop-common.472056.n3.nabble.com/how-to-assign-unique-ID-Long-Value-in-mapper-td4078062.html
Basically, to summarize, you shouldn't incorporate common sharable state
among reducers. You
Thanks a lot Shahab.
On Sun, Jun 28, 2015 at 8:38 PM, Shahab Yunus shahab.yu...@gmail.com
wrote:
You asked a similar question earlier also so I will copy those comments
here with what I replied then:
Hi Hadoop user,
I have two map classes processing two different input files. Both map
functions have same key,value format to emit.
But Reducer called twice for same key , one for value from first map while
one for value from other map.
I am printing (key ,value) pairs in reducer :
Can you describe the hotspot problem in a bit more detail ?
Which scheduler are you using ?
Cheers
On Mon, Jun 15, 2015 at 5:42 PM, codercooler codercoo...@163.com wrote:
hey guys,
how is yarn assign its containers? is that a completely random behavior? I
use hadoop 2.7.0 and I got a
As per Map Reduce, it is not possible that two different reducers will get
same keys.
I think you have created some custom key type? If that is the case then
there should be some issue with the comparator.
On Mon, Jun 29, 2015 at 12:40 AM, Ravikant Dindokar ravikant.i...@gmail.com
wrote:
Hi
Adding source code for more clarity
Problem statement is simple
PartitionFileMapper : it takes input file which has tab separated value V ,
P
It emits (V, -1#P)
ALFileMapper : It takes input file which has tab separated values V, EL
It emits (V, E#-1)
in reducer I want to emit
(V,E#P)
Thanks
