RE: Incrementally adding to existing output directory

It seems, It is not taking the CutomOutputFormat for the Job. You need to set the custom output format class using the org.apache.hadoop.mapred.JobConf.setOutputFormat(Class theClass) API for your Job. If we don't set OutputFormat for Job, it takes the default as TextOutputFormat which interna

Re:

Hello Anit, Could you show me the exact error log? Warm Regards, Tariq cloudfront.blogspot.com On Tue, Jul 16, 2013 at 8:45 AM, Anit Alexander wrote: > yes i did recompile. But i seem to face the same problem. I am running the > map reduce with a custom input format. I am not sure if there is

Re: Incrementally adding to existing output directory

Hi Devaraj, Thank you very much for your help. I've created a CustomOutputFormat which is almost identical to FileOutputFormat as seen here

JACKY_LIN�]Chih-Wei Lin �L�Ӱ��^ is out of the office.

I will be out of the office starting 2013/07/18 and will not return until 2013/07/19. I will respond to your message when I return. --- TSMC PROPERTY This em

Re: header of a tuple/bag

hi, you got the error because you did not arrange name to the fields, pig did not know which column is called cookie_id. you'd better do like below: Data = LOAD '/user/xx/20130523/*'; x = FOREACH Data GENERATE (chararray) $i AS cookie_id; i is the index of the cookie_id in the tuple counting fro

Re: QJM for federation

Yeah ideally you needn't configure a local edits directory, unless you want extra safety. There was an issue where it still gets configured and logs local edits though but am not sure if this was fixed. If you are not able to turn off local edit logging and if it still always happens, do file a HDF

Re: QJM for federation

I have another question for QJM. If I use QJM for HA, do I need to save edit log on the local filesystem? I think the QJM is high availability for edit log, so I don't need to config the "dfs.namenode.edits.dir". Thanks, LiuLei 2013/7/17 lei liu > Thanks Harsh. > > > > > 2013/7/17 Harsh J

Re: QJM for federation

Thanks Harsh. 2013/7/17 Harsh J > This has been asked previously. Use suffixes to solve your issue. See > http://search-hadoop.com/m/Fingkg6Dk91 > > On Wed, Jul 17, 2013 at 1:33 PM, lei liu wrote: > > I have two namespaces, example below: > > > > > > dfs.nameservices > > ns1,ns2 > >

Re: QJM for federation

This has been asked previously. Use suffixes to solve your issue. See http://search-hadoop.com/m/Fingkg6Dk91 On Wed, Jul 17, 2013 at 1:33 PM, lei liu wrote: > I have two namespaces, example below: > > > dfs.nameservices > ns1,ns2 > > > Can I config the "dfs.namenode.shared.edits.dir" t

QJM for federation

I have two namespaces, example below: dfs.nameservices ns1,ns2 Can I config the "dfs.namenode.shared.edits.dir" to below content? dfs.namenode.shared.edits.dir qjournal://10.232.98.61:20022;10.232.98.62:20022;10.232.98.63:20022/nn1,nn2 Thanks, LiuLei

Re: java.io.IOException: error=2, No such file or directory

Sure, there were some problems with postgresql installation. (Actually, this was the solution that you said in your mail but I did not notice.) On Tue, Jul 16, 2013 at 6:37 PM, Shahab Yunus wrote: > Great. Can you please share, if possible, what was the problem and how you > solved it? Thanks. >