Re: Hadoop 2.2.0 MR tasks failing

2013-11-01 Thread Robert Dyer
So does anyone have any ideas how to track this down? Is it perhaps an exception somewhere in an output committer that is being swallowed and not showing up in the logs? On Tue, Oct 22, 2013 at 2:19 AM, Robert Dyer wrote: > The logs for the maps and reduces show nothing useful. There are a ton

Re: Why is my output directory owned by yarn?

2013-11-01 Thread Harsh J
The DefaultContainerExecutor isn't the one that can do setuid. The LinuxContainerExecutor can do that. On Fri, Nov 1, 2013 at 8:00 PM, Bill Sparks wrote: > We'll I thought I've set all this up correctly and on the NodeManager > nodes can change to my user id, so general user authentication is wor

Re: Path exception when running from inside IDE.

2013-11-01 Thread Chris Mawata
What does the code look like? Chances are you are using a file:/// url Chris On 11/1/2013 12:59 PM, Omar@Gmail wrote: Hi, Running from inside IDE (intellij idea) getting exception, see below: In the program arguments I specify 'input output' Of course 'input' does exist in HDFS with data file

Re: best solution for data ingestion

2013-11-01 Thread Marcel Mitsuto F. S.
I've done some testing with flume, but ended up using syslog-ng, more flexible, reliable, and with a lower fingerprint. On Fri, Nov 1, 2013 at 3:57 PM, Mirko Kämpf wrote: > Have a look on Sqoop for data from RDBMS or Flume, if data flows and > multiple sources have to be used. > Best wishes > M

Re: best solution for data ingestion

2013-11-01 Thread Mirko Kämpf
Have a look on Sqoop for data from RDBMS or Flume, if data flows and multiple sources have to be used. Best wishes Mirko 2013/11/1 Siddharth Tiwari > hi team > > seeking your advice on what could be best way to ingest a lot of data to > hadoop. Also what are views about fuse ? > > > **-

best solution for data ingestion

2013-11-01 Thread Siddharth Tiwari
hi team seeking your advice on what could be best way to ingest a lot of data to hadoop. Also what are views about fuse ? ** Cheers !!! Siddharth Tiwari Have a refreshing day !!! "Every duty is holy, and devotion to duty is the highest form of worship of God.” "Maybe

Path exception when running from inside IDE.

2013-11-01 Thread Omar@Gmail
Hi, Running from inside IDE (intellij idea) getting exception, see below: In the program arguments I specify 'input output' Of course 'input' does exist in HDFS with data file in it. But the code is trying access a directory form local project file system location not form HDFS. Please let me

Re: Why is my output directory owned by yarn?

2013-11-01 Thread Bill Sparks
We'll I thought I've set all this up correctly and on the NodeManager nodes can change to my user id, so general user authentication is working. But still the output is written as yarn. I guess my question is how to enable secure mode - I thought that was the default mode. When the containers are