Re: how to assign unique ID (Long Value) in mapper

2015-06-30 Thread Ravikant Dindokar
Thanks Gabriel. On Tue, Jun 30, 2015 at 1:04 AM, gabriel balan gabriel.ba...@oracle.com wrote: Hi Rather than trying to figure out the line number of the current line, you can use the byte offset of the current line. It's just as unique as the line number, and much easier to obtain:

Re: how to assign unique ID (Long Value) in mapper

2015-06-29 Thread gabriel balan
Hi Rather than trying to figure out the line number of the current line, you can use the byte offset of the current line. It's just as unique as the line number, and much easier to obtain: TextInputFormat (FileInputFormat) uses it as the key. Keys are the position in the file, and values

Re: how to assign unique ID (Long Value) in mapper

2015-06-26 Thread Ravikant Dindokar
yes , there can be loop in the graph On Fri, Jun 26, 2015 at 9:09 AM, Harshit Mathur mathursh...@gmail.com wrote: Are there loops in your graph? On Thu, Jun 25, 2015 at 10:39 PM, Ravikant Dindokar ravikant.i...@gmail.com wrote: Hi Hadoop user, I have a file containing one line for each

Re: how to assign unique ID (Long Value) in mapper

2015-06-26 Thread Ravikant Dindokar
The problem can be thought as assigning line number for each line. Is there any inbuilt functionality in hadoop which can do this? On Fri, Jun 26, 2015 at 1:11 PM, Ravikant Dindokar ravikant.i...@gmail.com wrote: yes , there can be loop in the graph On Fri, Jun 26, 2015 at 9:09 AM, Harshit

Re: how to assign unique ID (Long Value) in mapper

2015-06-26 Thread Shahab Yunus
I see 2 issues here which go kind of against the architecture and idea of M/R (or distributed and parallel programming models.) 1- The map and reduce tasks are suppose to be shared-nothing and independent tasks. If you add a functionality like this where you need more sure that some data is

how to assign unique ID (Long Value) in mapper

2015-06-25 Thread Ravikant Dindokar
Hi Hadoop user, I have a file containing one line for each edge in the graph with two vertex ids (source sink). sample: 12 (here 1 is source and 2 is sink node for the edge) 15 23 42 43 I want to assign a unique Id (Long value )to each edge i.e for each line of the file. How

Re: how to assign unique ID (Long Value) in mapper

2015-06-25 Thread Harshit Mathur
Are there loops in your graph? On Thu, Jun 25, 2015 at 10:39 PM, Ravikant Dindokar ravikant.i...@gmail.com wrote: Hi Hadoop user, I have a file containing one line for each edge in the graph with two vertex ids (source sink). sample: 12 (here 1 is source and 2 is sink node for the