Is pipes really supposed to be a serious API? Is it being actively developed?

2008-09-19 Thread Marc Vaillant
Only about 5 pipes/c++ related posts since mid July, and basically no responses. Is anyone really using or actively developing pipes? We've invested some time to make it platform independent (ported bsd sockets to boost sockets, and the xdr serialization to boost serialization), but it's still la

Is Hadoop not particularly well suited for smaller parallel processing jobs?

2008-09-05 Thread Marc Vaillant
We have a potential parallel application in which a task consists of say 5 subcomponents, each of which can run concurrently. Each component takes on the order of 3 seconds to complete. We would like to get performance enhancement from parallel processing of a single task (task runs 5 parallel su

pipes examples, trouble with wordcount-nopipe

2008-08-28 Thread Marc Vaillant
We have set up a single node cluster to test pipes examples. The wordcount-simple example works. However, the wordcount-nopipe fails. The wordcount-nopipe example was invoked with the following commandline: "bin/hadoop pipes -conf src/examples/pipes/conf/word-nopipe.xml -input input/ -output out"

building C++ API for windows, is it just bsd sockets that is incompatible with a native build?

2008-07-21 Thread Marc Vaillant
I see that cygwin is the only supported option for building Hadoop Pipes for windows. I'm trying a mingw build and it looks like the only thing needing porting is the communications from bsd sockets to say winsock? Is that correct? Thanks, Marc