RE: streaming + binary input/output data?

2008-04-12 Thread Runping Qi
Actually, there is an old jira about the same issue: https://issues.apache.org/jira/browse/HADOOP-1722 Runping > -Original Message- > From: John Menzer [mailto:[EMAIL PROTECTED] > Sent: Saturday, April 12, 2008 2:45 PM > To: core-user@hadoop.apache.org > Subject: RE: streaming + binary

RE: streaming + binary input/output data?

2008-04-12 Thread John Menzer
so you mean you changed the hadoop streaming source code? actually i am not really willing to change the source code if it's not necessary. so i thought about simply encoding the input binary data to txt (e.g. with base64) and then adding a '\n' after each line to make it splittable for streaming

time measuring

2008-04-12 Thread John Menzer
hello, Since I have to compare hadoop performance to other technologies like MPI i need to know how and what to measure. in hadoop the web ui for the jobtracker provides for each successful job information like: Started at: Sat Apr 12 22:34:24 CEST 2008 Finished at: Sat Apr 12 22:35:03 CEST 200

Re: "could only be replicated to 0 nodes, instead of 1"

2008-04-12 Thread lohit
Can you check the datanode and namenode logs and see if all are up and running? I am assuming you are running this on single host hence replication of 1. Thanks, Lohit - Original Message From: John Menzer <[EMAIL PROTECTED]> To: core-user@hadoop.apache.org Sent: Saturday, April 12, 2008

Re: "could only be replicated to 0 nodes, instead of 1"

2008-04-12 Thread John Menzer
i had the same error message... can you describe when and how this error occurs? Jayant Durgad wrote: > > I am faced with the exact same problem described here, does anybody know > how > to resolve this? > > -- View this message in context: http://www.nabble.com/Re%3A-%22could-only-be-repl

Re: 答复: Problem with key aggregation when number of reduce tasks is more than 1

2008-04-12 Thread Harish Mallipeddi
Hey thanks a lot. That's basically what I needed. 2008/4/11 Zhang, jian <[EMAIL PROTECTED]>: > Hi, > > Please read this, you need to implement partitioner. > It controls which key is sent to which reducer, if u want to get unique > key result, you need to implement partitioner and the compareTO f