Re: Any one successfully ran the c++ pipes example?
Hi, Owen, Did you mean that the example with a C++ record reader is not complete? I have to run this example with the class file of that WordCountInputFormat.java. Also, it seemed that the semantics of the C++ Pipes API is different from Java. A InputSplit is a chunk of a file in Java, while in C++, it seemed to be the name of a file. I got this conclution from that wordcount example. Thanks, Mike - Original Message From: Owen O'Malley <[EMAIL PROTECTED]> To: core-user@hadoop.apache.org Sent: Friday, October 17, 2008 2:02:20 PM Subject: Re: Any one successfully ran the c++ pipes example? On Oct 16, 2008, at 1:40 PM, Zhengguo 'Mike' SUN wrote: > I was trying to write an application using the pipes api. But it > seemed the serialization part is not working correctly. More > specifically, I can't deserialize a string from an StringInStream > constructed from context.getInputSplit(). Even with the examples > bundled in the distribution archive(wordcount-nopipe.cc), it threw > exceptions. If anyone had experience on that, please kindly give > some advise. So you mean the example with a C++ record reader? You have to use the InputFormat that generates input splits that consists of strings. Look at src/test/org/apache/hadoop/pipes/WordCountInputFormat.java It would be useful to have a C++ impl of FileInputSplit too... -- Owen __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
Re: Any one successfully ran the c++ pipes example?
On Oct 16, 2008, at 1:40 PM, Zhengguo 'Mike' SUN wrote: I was trying to write an application using the pipes api. But it seemed the serialization part is not working correctly. More specifically, I can't deserialize a string from an StringInStream constructed from context.getInputSplit(). Even with the examples bundled in the distribution archive(wordcount-nopipe.cc), it threw exceptions. If anyone had experience on that, please kindly give some advise. So you mean the example with a C++ record reader? You have to use the InputFormat that generates input splits that consists of strings. Look at src/test/org/apache/hadoop/pipes/WordCountInputFormat.java It would be useful to have a C++ impl of FileInputSplit too... -- Owen
Any one successfully ran the c++ pipes example?
Hi, I was trying to write an application using the pipes api. But it seemed the serialization part is not working correctly. More specifically, I can't deserialize a string from an StringInStream constructed from context.getInputSplit(). Even with the examples bundled in the distribution archive(wordcount-nopipe.cc), it threw exceptions. If anyone had experience on that, please kindly give some advise. P.S. The code that I am suspecting: HadoopUtils::StringInStream stream(context.getInputSplit()); HadoopUtils::deserializeString(fname, stream); fname should be deserialized from that stream, but it actually wasn't.
