Re: mapred Configuration Vs mapReduce configuration in Ooozie

[Robert Kanter]

Hi,

You can use the new api in your action if you set these two properties in
the action's  section:

  mapred.mapper.new-api
  true


  mapred.reducer.new-api
  true


The JobConf.setUseNewMapper(boolean flag) simply sets the same property;
same with JobConf.setUseNewReducer(boolean flag).


 - Robert


On Mon, Sep 23, 2013 at 2:39 PM, KayVajj  wrote:

> I am implementing an oozie workflow MR job . I was not sure if I have to
> use the mapred properties vs the mapreduce properties.
>
> I have seen that if mapreduce properties were to be used, it has to be set
> explicitly like
>
> JobConf.setUseNewMapper(boolean flag)
>
>
> How do I do the same in Oozie?
>
> Also another question, if I were to use the new properties where do I find
> the property names like "mapreduce.job.inputformat.class" These seem to be
> hidden in some private API.
>
> Thanks your answers are appreciated.
>
>
>  --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "CDH Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to cdh-user+unsubscr...@cloudera.org.
> For more options, visit
> https://groups.google.com/a/cloudera.org/groups/opt_out.
>

Oozie Meetup on March 12

[Robert Kanter]

Hi everyone,

I apologize for cross-posting this

Cloudera will be hosting an Oozie meetup on March 12 from 2:30pm to
5:00pm in our Palo Alto office.
Please join us to meet fellow Oozie users and developers and have some
free food.  Interested users from other projects are welcome to join us
too.

Agenda:
1) Yahoo! Oozie team to talk about their recent HCatalog integration work
2) Cloudera Hue team to talk about their Hue Oozie UI
3) Yahoo! Oozie team to lead a discussion/brainstorming session on the
Job notifications and SLA notification/email alerting work described in
OOZIE-929 
4) Open discussion

Please RSVP and we hope to see you all there!
http://www.meetup.com/Oozie-Contributors/events/107147852/

thanks
- Robert

Re: Re: aggregated logs on HDFS garbled

[Robert Kanter]

As a shameless self-promotion :), you can try this program that I wrote
which will convert the log files back to plain text files:
https://github.com/rkanter/ALFExploder


- Robert

On Thu, May 10, 2018 at 12:44 AM,  wrote:

> Thanks for your advice！
>
> I tried the tflie reader you provided in [2]，but there's still some garble
> appear at the beginnig and the end, looks like this：
> stdout0
>
> and I found another tool that works fine:
> https://github.com/shanyu/hadooplogparser
>
> 在2018-05-10 11:13:38,Hariharan Iyer写道：
> > Application logs are not saved in text format. They are saved in a
> > binary format called TFile. You can use the "yarn logs"[1] command to
> > view them. Alternatively you can use a tool like this tfile reader[2].
> >
> > [1] - https://hadoop.apache.org/docs/r2.7.4/hadoop-yarn/
> hadoop-yarn-site/YarnCommands.html#logs
> > [2] - https://github.com/alexandruanghel/hadoop-
> snippets/tree/master/read-tfile
> >
> > Thanks,
> > Hari
> >
> >
> > On Thu, May 10, 2018 at 6:06 AM,   wrote:
> > > Hello，
> > >
> > >
> > >
> > > I set “yarn.log-aggregation-enable” to true, and got logs on HDFS, but
> when
> > > I use
> > >
> > > “bin/hdfs dfs -cat” or download files from the HDFS website, some
> garble
> > > appeared, and it looks like this:
> > >
> > >
> > >
> > > \D1 \D3h\91\B5׶9\DFA@\92\BA\E1P
> > > \00 VERSION \00\00\00  \00 APPLICATION_ACL$\00 VIEW_APP\00 root \00
> > >
> > > MODIFY_APP\00 root
> > >  \00 APPLICATION_OWNER \00 root(\00&container_1525851594379_0001_01_03
> \00 stderr\00 0\00 stdout\00 0\00 \00\00 \00
> > >
> > > \00 VERSION*(\00&container_1525851594379_0001_01_03  none  ̡̡
> data:BCFile.index none\CC\ED   data:TFile.index none̷66 data:TFile.meta
> none̱  \00\00\00\00\00\00\00\F8\00 \00\00\D1 \D3h\91\B5׶9\DFA@\92\BA\E1P
> > >
> > >
> > > What can I do to fix this?
> > >
> > >
> > >
> > > Thanks in advance!
> >
> > -
> > To unsubscribe, e-mail: user-unsubscr...@hadoop.apache.org
> > For additional commands, e-mail: user-h...@hadoop.apache.org
> >
>
>