Hadoop PIPES job using C++ and binary data results in data locality problem.

2012-01-10 Thread GorGo
Hi everyone. I am running C++ code using the PIPES wrapper and I am looking for some tutorials, examples or any kind of help with regards to using binary data. My problems is that I am working with large chunks of binary data and converting it to text not an option. My first question is thus,

Re: Hadoop PIPES job using C++ and binary data results in data locality problem.

2012-01-10 Thread Robert Evans
I think what you want to try and do is to use JNI rather then pipes or streaming. PIPES has known issues and it is my understanding that its use is now discouraged. The ideal way to do this is to use JNI to send your data to the C code. Be aware that moving large amounts of data through JNI