Re: Oozie - Handling of the activation of Dayligh Saving Time

2016-03-11 Thread Alejandro Abdelnur
No it would not. a day follows DST rules, a minute does not. thx On Fri, Mar 11, 2016 at 1:04 PM, Alvin Chyan wrote: > As a follow up question, is this auto-handling of daylight savings only if > we use "${coord:days(1)}"? Would frequency="${coord:minutes(160)}" also > work? > > Thanks! > > > *

Re: [ANNOUNCE] New Oozie PMC Chair - Robert Kanter

2015-10-13 Thread Alejandro Abdelnur
Robert, congratulations!! On Tue, Oct 13, 2015 at 8:50 PM, Aravindakshan Srinivasan < s_arav...@yahoo.com.invalid> wrote: > Congratulations Robert! Great news. Regards,Aravind > From: Mohammad Islam > To: Oozie ; User > Sent: Wednesday, October 7, 2015 12:25 AM > Subject: [ANNOUNCE] Ne

Re: Apache Oozie book published

2015-05-28 Thread Alejandro Abdelnur
ect: Re: Apache Oozie book published > > Mohammad, Aravind,Awesome, congratulations! > just wondering what the animal on cover page represents :)Ryota > > > > > On Thursday, May 28, 2015 6:36 AM, Alejandro Abdelnur < > tuc...@gmail.com> wrote: > > > M

Re: Apache Oozie book published

2015-05-28 Thread Alejandro Abdelnur
rsday, May 28, 2015 6:36 AM, Alejandro Abdelnur < > tuc...@gmail.com> wrote: > > > Mohammad, Aravind, > > This is Great, a 1000 Congratulations! > > Alejandro > > On Thu, May 28, 2015 at 12:40 AM, Mohammad Islam > > wrote: > > > Dear Oozie us

Re: Apache Oozie book published

2015-05-28 Thread Alejandro Abdelnur
Mohammad, Aravind, This is Great, a 1000 Congratulations! Alejandro On Thu, May 28, 2015 at 12:40 AM, Mohammad Islam wrote: > Dear Oozie users/developers, > My co-author Aravind and I are happy to announce the official launch of > our book on Oozie. We thank O'Reilly and all the reviewers for

Re: Oozie Load distribution

2015-03-31 Thread Alejandro Abdelnur
oozie simply submits the job to run the script to the cluster, it is up to the cluster scheduler what nodes are allocated. hth Alejandro > On Mar 31, 2015, at 22:56, Bhagaban Khatai wrote: > > Hi, > > Can someone explain me, if oozie doing the load distribution in the > cluster. I have to s

Re: Oozie works with YARN?

2015-02-18 Thread Alejandro Abdelnur
doing that is not possible, you cannot suspend MR jobs mid-flight. When you suspend an Oozie WF, Oozie will run the action (MR in this case) till completion and won't start then next one. HTH On Wed, Feb 18, 2015 at 12:56 PM, xeonmailinglist wrote: > I want to suspend a mapreduce job when the m

Re: Welcome new Oozie Committer - Shwetha GS

2014-07-22 Thread Alejandro Abdelnur
Congrats, well deserved. On Tue, Jul 22, 2014 at 2:34 PM, Robert Kanter wrote: > Congrats! > > > On Tue, Jul 22, 2014 at 1:59 PM, Purshotam Shah < > purus...@yahoo-inc.com.invalid> wrote: > > > Congrats Shwetha! > > > > > > Puru. > > > > On 7/22/14, 1:56 PM, "Mona Chitnis" > wrote: > > > > >Co

Re: Welcoming new Oozie PMC member Rohini Palaniswamy

2014-07-01 Thread Alejandro Abdelnur
Welcome Rohini, well deserved! On Tue, Jul 1, 2014 at 3:41 PM, Mona Chitnis wrote: > Hello Oozie community, > > > It's my pleasure to announce that Rohini Palaniswamy became the newest > addition to the Oozie PMC. > Rohini has been strongly contributing to Oozie and providing expert > guidance.

Re: Welcoming Bowen Zhang as Oozie committer

2014-06-12 Thread Alejandro Abdelnur
Congrats Bowen, well deserved! On Thu, Jun 12, 2014 at 10:16 AM, Robert Kanter wrote: > It is my pleasure to announce that Oozie PMC has invited Bowen to become > anOozie committer and he has accepted our invitation. > Bowen has already made several contributions to Oozie and we are sure > many

Re: why is sudo disallowed in oozie

2014-04-22 Thread Alejandro Abdelnur
Could you be a bit more specific on what you are trying to do? Thx > On Apr 22, 2014, at 7:39 AM, Scott Preddy wrote: > > I understand security considerations, but this can be controlled. It is > unnecessarily limiting for uses where it is needed and can be controlled.

Re: Any plan for Java 7 or 8 implementation of Oozie?

2014-04-08 Thread Alejandro Abdelnur
for each soon. > > Is there Apache infrastructure which can be used for Regression testing. > Sooner or later Hadoop community should accept Java 8 for the set of > functional programming features. > > Thanks, > > Jagat Singh > > > > > On Wed, Apr 9, 2014

Re: Any plan for Java 7 or 8 implementation of Oozie?

2014-04-08 Thread Alejandro Abdelnur
java6, no plans for 7 or 8 yet. thx Alejandro (phone typing) > On Apr 8, 2014, at 16:03, "Segerlind, Nathan L" > wrote: > > Hi All. > > I looked for a FAQ but I couldn't find one. Oozie is currently implemented in > Java 6 - are there any plans to move to Jave 7 or Java 8? > > Thanks, > N

Re: Delegation Token

2014-02-21 Thread Alejandro Abdelnur
i suggest trying hdp mailing list. thx Alejandro (phone typing) > On Feb 21, 2014, at 5:14, Roshan Punnoose wrote: > > We installed Oozie/Kerberos with Ambari and HDP 2.0.6. I don't believe > kerberos is configured incorrectly, but maybe there is something I'm > missing that Ambari won't add

Re: Dynamic parallel tasks in Oozie

2014-02-20 Thread Alejandro Abdelnur
This is what I refer as sharding, it can be seen as a special type of fork/join where all shards are doing the same actions on different datasets and the number of shards depends on the number of datasets. A while ago I've rewritten the workflow lib, cleanning it up a bit and adding this capabilit

Re: [ANNOUNCE] Oozie source repo has moved to GIT

2013-12-26 Thread Alejandro Abdelnur
I've forgot to mention 'master' branch is the new 'trunk' On Thu, Dec 26, 2013 at 8:49 AM, Alejandro Abdelnur wrote: > Oozie developers, > > The move from SVN to GIT is complete. > > I've already updated the jenkins jobs to work out of the GIT repo.

[ANNOUNCE] Oozie source repo has moved to GIT

2013-12-26 Thread Alejandro Abdelnur
Oozie developers, The move from SVN to GIT is complete. I've already updated the jenkins jobs to work out of the GIT repo. Only the Apache Oozie website remains in Subversion: https://svn.apache.org/repos/asf/oozie/site/trunk I still need to update the site and wiki pages where SVN is being

Re: Shell and SSH actions dispatch mechanism

2013-11-22 Thread Alejandro Abdelnur
hi Misha, you are correct shell actions run in a random node of the cluster. there is no way to control that today. eventually we could add support for locality affinity leveraging yarn. ssh action run in the specified host, bode cluster or not thx Alejandro (phone typing) On Nov 22, 2013,

Re: Re: hi,how can I complete this DAG in a workflow using oozie?

2013-11-09 Thread Alejandro Abdelnur
; actions at the begging of each coordinator will > occupied 300+ map slots. > 4.Can Oozie server hold 300+ concurrency? > > I want to know better ways to configure the 1000+ scripts in this project, > and the reason why I can't put the "M" type dependency in a workfl

Re: hi,how can I complete this DAG in a workflow using oozie?

2013-11-08 Thread Alejandro Abdelnur
the best parallelism you can get for this in a wf is: fork 1, 2 join fork 3, 4, 5 join thx Alejandro (phone typing) On Nov 8, 2013, at 5:45, renguihe wrote: > hi, > Add the error info I have got here when I try to implement this DAG in a > workflow: > 1->3 > 1->4 > 2->4 > 2->5 > > I set oo

Re: Is there a way to ensure two actions run on the same datanode?

2013-10-30 Thread Alejandro Abdelnur
Michael, Currently it is not possible to make to actions to run in the same node. Eventually, when Oozie starts leveraging Yarn capabilities, this could be possible. Today you can overcome this limitation by using HDFS as the filesystem for your actions to leave and pick up data from. Thanks.

Re: Cascading and Oozie

2013-10-08 Thread Alejandro Abdelnur
I would suggest looking at how Pig/Hive/Sqoop/Distcp actions works if you want to have a custom action. Which, BTW, it would be a great contribution to Oozie. If you are going this path, you'll have to write a CascadingActionExecutor class that runs in the Oozie server and you'll e corhave to wri

Re: Oozie and hadoop

2013-09-19 Thread Alejandro Abdelnur
gt; > > > > > Oh, that documentation of -hadoop option is removed now. > > > > The problem with copying hadooplibs to libext is that if the hadoop > > classpath has multiple directories, all jars from all directories need to > > be copied. Instead, if the hadoop

Re: Oozie and hadoop

2013-09-17 Thread Alejandro Abdelnur
Hi Shwetha, You just need to copy the JARs from your Hadoop version in libext/ and run oozie prepare-war, you don't need to specify any version of hadoop http://oozie.apache.org/docs/4.0.0/DG_QuickStart.html#Server_Installation thx On Tue, Sep 17, 2013 at 3:01 PM, Shwetha GS wrote: > Hi, >

Oozie PM chair change, new chair Mohammad Islam

2013-09-16 Thread Alejandro Abdelnur
First of all, my apologies for the delay getting the word out. Last month, Oozie PMC has voted Mohammad Islam as the new Oozie PMC chair. Join me congratulating Mohammad on his new post. Cheers. Alejandro. PS: Mohammad, you'll have to update the ASF LDAP and pages to reflect the changes. Please

Re: Problem with the workflow in a single node

2013-08-19 Thread Alejandro Abdelnur
Most likely you cluster is running out of slots to run jobs. Thanks. On Mon, Aug 19, 2013 at 2:27 AM, Đức Trung Nguyễn wrote: > Hello every body, > > I am a student. I tried to run a workflow, which contains two parallel jobs > (using fork) in a single node (installed Pseudo mode of hadoop) but

Re: [VOTE] Release Oozie 4.0.0 (RC1)

2013-08-15 Thread Alejandro Abdelnur
I think we need to do a new RC without the workflow generator (as we did before in the 3.x series). On Wed, Aug 14, 2013 at 1:12 PM, Ryota Egashira wrote: > +1 to RC1 (my previous email was vote for candidate 0, my bad) > > Thanks > Ryota > > On 8/14/13 1:00 PM, "Virag Kothari" wrote: > > >+1 >

Re: Oozie and Lzo on Hortonworks 1.2 - ClassNotFoundException

2013-08-08 Thread Alejandro Abdelnur
i'm not familiar with the hw oozie setup, but if you put the lzo jar in the same dirs the oozie-core jar is, it should work. if not i'd suggest reaching out the hw alias for their recommended way of doing it thx Alejandro (phone typing) On Aug 8, 2013, at 6:10, w00t w00t wrote: > Hello, >

Re: Support for workflow actions running across multiple big data frameworks in parallel?

2013-06-19 Thread Alejandro Abdelnur
or if you don't want to go into writing a custom action executor for now, you could use the Oozie shell or ssh actions to start your mesos job. On Wed, Jun 19, 2013 at 1:58 PM, Mona Chitnis wrote: > That is an interesting use case Som! Oozie is tightly coupled with Hadoop > right now. You might

Re: Allowing Impersonation

2013-06-03 Thread Alejandro Abdelnur
Lenin, It should the unix username that is running the oozie server. thanks On Mon, Jun 3, 2013 at 8:35 PM, Lenin Raj wrote: > Thanks Roman. Can you please suggest the recommended way to do > impersonation in this Hadoop version? > > What should I replace * with? I tried replacing * with root

Re: Welcome Mona Chitnis as new Oozie PMC

2013-06-03 Thread Alejandro Abdelnur
Congratulations Mona, well deserved. On Mon, Jun 3, 2013 at 3:57 PM, Mohammad Islam wrote: > Hi Oozie community, > It is our pleasure to announce that Mona Chitnis has joined Oozie PMC. > Mona has consistently contributing a lot of significant features and > helping to grow the community. > >

Re: Pipeline multiple workflows?

2013-05-30 Thread Alejandro Abdelnur
Felix, there is a sub-workflow action you can use from a workflow to call another workflow. Thanks On Thu, May 30, 2013 at 10:29 PM, Felix.徐 wrote: > Hi all, > > How to coordinate multiple workflows into a single job. I do not really > want to merge all the codes into a single workflow.xml ,

Re: Oozie 2.3.2 - What is the impact if Instrumentation service is disabled?

2013-05-29 Thread Alejandro Abdelnur
we never seen perf issues because instrumentation service. said this, the safest way to disable instrumentation would be having a nop subclass and using that instead thx Alejandro (phone typing) On May 28, 2013, at 19:53, The Trainer wrote: > Hi All, > > Just curious to know what is the im

Re: Oozie 2.3.2 - Long running coordinators - purge service not deleting historical actions

2013-05-29 Thread Alejandro Abdelnur
as mohammad says, oozie 2 has issues in its purge logic. still what you are seeing it is the expected behavior. no data of running jobs is deleted. one way of solving this is to have back to back coord jobs every few months. then the completed ones will qualify for purging. thx Alejandro

Welcoming Rohini Palaniswamy as Oozie committer

2013-05-28 Thread Alejandro Abdelnur
It is my pleasure to announce that Oozie PMC has invited Rohini to become an Oozie committer and she has accepted our invitation. Rohini has already made several contributions to Oozie and we are sure many more will come in her new role. Please join me congratulating her. Regards. Alejandro, on

Welcoming Ryota Egashira as Oozie committer

2013-05-28 Thread Alejandro Abdelnur
It is my pleasure to announce that Oozie PMC has invited Ryota to become an Oozie committer and he has accepted our invitation. Ryota has already made several contributions to Oozie and we are sure many more will come in his new role. Please join me congratulating him. Regards. Alejandro, on be

Re: Binary distribution of oozie

2013-05-28 Thread Alejandro Abdelnur
erver_Installation> > ) > > > Thanks, > Lenin > > > On Tue, May 28, 2013 at 10:49 PM, Alejandro Abdelnur >wrote: > > > ha!, that may be, try Sun JDK 1.6 > > > > thx > > > > > > On Tue, May 28, 2013 at 10:17 AM, Lenin Raj > wrote: > >

Re: Binary distribution of oozie

2013-05-28 Thread Alejandro Abdelnur
ot-allocate-memory-calling-run > > Actually, I am using Open JDK right now. Should I installed Sun JDK 6 or > Oracle JDK 7? Which is more compatible with Hadoop? > > > Thanks, > Lenin > > > On Tue, May 28, 2013 at 10:25 PM, Alejandro Abdelnur >wrote: > > > Lenin

Re: Binary distribution of oozie

2013-05-28 Thread Alejandro Abdelnur
ion about the errors and possible solutions, > please read the following articles: > [ERROR] [Help 1] > http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException > [ERROR] > [ERROR] After correcting the problems, you can resume the build with the > command >

Welcome Robert Kanter as new Oozie PMC

2013-05-28 Thread Alejandro Abdelnur
Since Robert got involved in Oozie he has made significant contributions to the project, no only code-wise by also helping grow our community. It is my pleasure to announce that Robert Kanter has joined Oozie PMC. Regards. Alejandro, on behalf of the Oozie PMC.

Re: Binary distribution of oozie

2013-05-28 Thread Alejandro Abdelnur
mvn clean package > assembly:single ${MVN_OPTS} "$@" > > ERROR, Oozie distro creation failed > * > Shall I try with -DskipTests? > > > Thanks, > Lenin > > > On Tue, May 28, 2013 at 8:36 PM, Alejandro Abdelnur >wrote: > > > Len

Re: Binary distribution of oozie

2013-05-28 Thread Alejandro Abdelnur
86", family: "unix" > > where can I locate .m3 cache dir? > > > Thanks, > Lenin > > > On Tue, May 28, 2013 at 8:27 PM, Alejandro Abdelnur >wrote: > > > odd, would you try using the latest version of maven and also trashing > your > >

Re: Binary distribution of oozie

2013-05-28 Thread Alejandro Abdelnur
cal/oozie/oozie# echo $MAVEN_OPTS > -Xmx1024m* > > Any other pointers? > > > > Thanks, > Lenin > > > On Tue, May 28, 2013 at 1:31 AM, Alejandro Abdelnur >wrote: > > > Hi Lenin, > > > > We don't have binary distributions availa

Re: Binary distribution of oozie

2013-05-27 Thread Alejandro Abdelnur
Hi Lenin, We don't have binary distributions available. Would you try raising the memory for Maven before running your build? $ export MAVEN_OPTS=-Xmx1024m Thanks On Mon, May 27, 2013 at 11:27 AM, Lenin Raj wrote: > Hello, > > I was searching for the binary distribution of oozie here: > > ht

Re: Not able to find the binary distribution for apache oozie-3.3.0

2013-05-22 Thread Alejandro Abdelnur
this is topic keeps coming in several projects. my understanding is that according to ASF rules, release artifacts must be source only artifacts. and, while companion binary artifacts are allowed, they should not be signed. furthermore, they should not be hosted in the ASF dist server? roman, d

Re: Exception while trying to send email using emailactionexecutor

2013-05-21 Thread Alejandro Abdelnur
Kiran, It seems a limitation in Oozie EmailActionExecutor which does not allow setting additional (not envisioned) properties for the SMTP client. Would you please open a JIRA for it? And if you want to take a stab at it I'll gladly help Thanks. On Mon, May 20, 2013 at 11:37 PM, Kiran Nagasubra

Re: any limit on how long a action runs

2013-05-07 Thread Alejandro Abdelnur
Tousif, currently Oozie does not support timing out a running action. If this is what you are referring to, please open a JIRA (and if you want to take a stab at it it would be great) Thx On Fri, May 3, 2013 at 5:43 AM, Tousif wrote: > Hey, > > > I would like to know if there is any limit set

[REPORT] Apache Oozie - May Report

2013-05-06 Thread Alejandro Abdelnur
DESCRIPTION Apache Oozie is a system for managing and scheduling workflows that run different types of Hadoop jobs (such as MapReduce, Pig, Hive and Sqoop) as well as system specific jobs (such as Java programs and shell scripts). NEW RELEASES * Apache Oozie version 3.3.2, released on MAR/25/2013

Re: Wait for File dependency in Oozie workflow

2013-03-28 Thread Alejandro Abdelnur
Yes, oozie coordinator app is what you want. Thx On Mar 28, 2013, at 6:13 AM, Carlos Morillo wrote: > Hi Everybody, > > > Does anybody know if you can have in an Oozie workflow a dependency on a file > present on a certain directory? > > Something similar to the Wait For File stage in a Dat

Re: ANNOUNCE Apache Oozie 3.3.2 release

2013-03-26 Thread Alejandro Abdelnur
great job Robert, thanks for driving this release. And thanks to all for making it possible. Alejandro (phone typing) On Mar 26, 2013, at 9:24 AM, Robert Kanter wrote: > Hi All, > > The Apache Oozie team is pleased to announce the release of Oozie 3.3.2. > Oozie 3.3.2 has many improvements

Re: Oozie job failed with RemoteException

2013-03-15 Thread Alejandro Abdelnur
Savitha, is hduser a group in the JT/NN and the hduser belongs to the hduser group? Thanks. On Fri, Mar 15, 2013 at 5:02 AM, Savitha Devi Inbasekaran < savith...@smartek21.com> wrote: > Hi, > I am using Hadoop 1.0.4 and Oozie 3.3.0. I tried to run first sample job > in oozie but got the follow

Re: Oozie 3.3.1 distribution package - availability?

2013-03-14 Thread Alejandro Abdelnur
Jun, that sounds like a good idea. how would you do that? thx? Alejandro (phone typing) On Mar 13, 2013, at 10:54 AM, jun aoki wrote: > It will be great if we have binaries for oozie, and moreover, published on > mvn public repo. > > If it is the case, users as long as have jdk and mvn instal

Re: unauthorized connection for super-user

2013-03-14 Thread Alejandro Abdelnur
what are the versions of hadoop and oozie you are using? what unix user is running oozie? from which host? how the proxyuser settings look like in the jobtracker and namenode? have you restarted your cluster after setting the proxyusers? wjat is the full error trace you are getting? what errors you

Re: Capture the output of a Hive query?

2013-03-12 Thread Alejandro Abdelnur
Felix, On #1, you typically don't want to run user code (apps) in the oozie server. Also, by using an MR job for the shell action we get automatic load distribution. If you need to run an action outside of the cluster, then look at the SSH action. On #2, whatever you print in an action will be in

Re: Capture the output of a Hive query?

2013-03-12 Thread Alejandro Abdelnur
Felix, On #1, Hive actions don't produce output back to oozie. If you need the next action to handle the output, you should point to the next action the output directory of the hive action. ON #2, definitely, you can add ${VAR} in your workflow configs, and provide the the VAR=VALUE in the job.pr

Re: Web Service API: Job Submission example

2013-03-11 Thread Alejandro Abdelnur
The header should be -H 'Content-type: application/xml' Thx On Mon, Mar 11, 2013 at 3:06 PM, Praveen M wrote: > I'm running into the exact same issue. Can someone please provide some > insight in this? > > Thanks, > Praveen > > > On Fri, Mar 8, 2013 at 11:17 AM, Parrot Silk wrote: > > > Hi,

Re: Submit RHadoop job using Ozzie in Cloudera Manager

2013-03-08 Thread Alejandro Abdelnur
[moving thread to user@oozie.a.o, BCCing common-user@hadoop.a.o] Oozie web UI is read only, it does not do job submissions. If you want to do that you should look at Hue. Thx On Fri, Mar 8, 2013 at 2:53 AM, rohit sarewar wrote: > Hi > > I have R and RHadoop packages installed on all the nodes.

Re: Submitting Hadoop jobs from a shell action leads to the lost of the user identity

2013-02-22 Thread Alejandro Abdelnur
Clément, any luck? On Thu, Feb 21, 2013 at 9:35 AM, Alejandro Abdelnur wrote: > Hi Clément, > > The HADOOP_USER_NAME trick will work only in a non-secure cluster where > you can be whoever you want just by setting that property. it will not work > in a secure setup. > > T

Re: Can I propagate errors from sub-workflows to the parent workflow?

2013-02-22 Thread Alejandro Abdelnur
Tim, No, it is not possible today. You may want to open a JIRA to initiate a discussion on how we could do this. Thx On Thu, Feb 21, 2013 at 9:50 PM, Tim Goodman wrote: > I have a workflow action with many sub-workflows. If it's possible, I'd > like to handle errors in the parent workflow on

Re: Submitting Hadoop jobs from a shell action leads to the lost of the user identity

2013-02-21 Thread Alejandro Abdelnur
, 2013 at 9:22 AM, Clément MATHIEU wrote: > On 2013-02-21 18:01, Alejandro Abdelnur wrote: > > Thanks Alejandro, > > Oozie provides MR and Pig actions that propagate the user identity >> correctly. If you are using a shell action your shell action ends up >> running as the

Re: Submitting Hadoop jobs from a shell action leads to the lost of the user identity

2013-02-21 Thread Alejandro Abdelnur
Clément, Oozie provides MR and Pig actions that propagate the user identity correctly. If you are using a shell action your shell action ends up running as the Unix user running the task, in an unsecure cluster is typically the mapred user, in a secure cluster is the same user that submitted the j

Re: Question about secure Impersonation of oozie -- Are real Unix users still needed?

2013-02-14 Thread Alejandro Abdelnur
Jun, With Kerberos enabled, you need in your Hadoop cluster (all nodes) a unix id for every user submitting jobs to the cluster (via proxy user -like Oozie- or directly). You can configure your nodes to use LDAP as their source of user provisioning. Thx On Thu, Feb 14, 2013 at 12:29 PM, Jun Yu

Re: Oozie Web Console & cookies: Stop responding until cleared.

2013-02-13 Thread Alejandro Abdelnur
8/05/20/ext-21-state-managment-issues-dont-use-it/ > > I don't think we really need to remember the sizes of the windows, so it > would be best if we found a way to simply disable those cookies. > > - Robert > > > > On Wed, Feb 13, 2013 at 8:41 AM, Alejandro Abdelnur

Re: Oozie - accessing files from local file system

2013-02-13 Thread Alejandro Abdelnur
I guess the HCAT integration (in hive and oozie, both WIP) would be the answer for that. On Wed, Feb 13, 2013 at 10:04 AM, Harish Krishnan < harish.t.krish...@gmail.com> wrote: > I have also faced similar issue. Would appreciate a solution for this > issue. > > Thanks & Regards, > Harish.T.K > >

Re: Oozie Web Console & cookies: Stop responding until cleared.

2013-02-13 Thread Alejandro Abdelnur
Eduardo, I have not seen that problem, would you mind posting all the cookies (key/values) you are deleting in order to get the console responsive again? Thx On Wed, Feb 13, 2013 at 8:23 AM, Eduardo Afonso Ferreira wrote: > Hi there, > > I'm sure somebody else experience this problem as well

Re: Is there any way to use parameterized values for input-events?

2013-02-12 Thread Alejandro Abdelnur
yes it can, just don't use the ${} in the embedded var. thx Alejandro (phone typing) On Feb 12, 2013, at 1:45 PM, Tim Chan wrote: > I would like to used values specified in my properties files to populate > > ${coord:current(-7)} > > Can this -7 value be specified by a property? I've tried:

Re: Seeking advice: Actions stuck in PREP using Oozie 3.2.0 (cdh4)

2013-02-06 Thread Alejandro Abdelnur
Matthew, please use the CDH aliases for issues that are for CDH releases (moving it there, BCCing user@oozie.a.o) Is the Oozie log reporting any kind of error/warning? Have you followed the upgrade instructions (DB, sharelibs?) Thx On Wed, Feb 6, 2013 at 2:47 PM, Matthew Rathbone wrote: > He

Re: Oozie coordinator action rerun.

2013-02-04 Thread Alejandro Abdelnur
On Sat, Feb 2, 2013 at 11:05 PM, Shangzhong zhu wrote: > Hi All, > > We are developing a wrapper on top of oozie to automate failed/killed > coordinator action rerun. > > To rerun a coordinator action, seems I have two options. > > 1. Using coordinator action rerun: > oozie job -rerun <-dat

Re: Hadoop proxyuser problem

2013-02-01 Thread Alejandro Abdelnur
s merely for troubleshooting. Although it still does not work > without it. > > > hadoop.proxyuser.oozie. > hosts > > > > > hadoop.proxyuser.oozie.groups > oozie,hadoop > > > Thx. > > > > On Fri, Feb 1, 2013 at 10:29 AM, Alejan

Re: [REPORT] Apache Oozie - February Report

2013-02-01 Thread Alejandro Abdelnur
Andy, Does the oozie user belong to the oozie group in the JT and NN hosts? On Fri, Feb 1, 2013 at 10:48 AM, Alejandro Abdelnur wrote: > DESCRIPTION > > Apache Oozie is a system for managing and scheduling workflows that run > different types of Hadoop jobs (such as MapReduce, P

[REPORT] Apache Oozie - February Report

2013-02-01 Thread Alejandro Abdelnur
DESCRIPTION Apache Oozie is a system for managing and scheduling workflows that run different types of Hadoop jobs (such as MapReduce, Pig, Hive and Sqoop) as well as system specific jobs (such as Java programs and shell scripts). NEW RELEASES * Apache Oozie version 3.3.0, released on DEC/03/2012

Re: Hadoop proxyuser problem

2013-02-01 Thread Alejandro Abdelnur
AM, aasfo kxi wrote: > Sorry. Accidentally hit send before pasting my config > > > hadoop.proxyuser.oozie.hosts > > > > > hadoop.proxyuser.oozie.groups > oozie,hadoop > > > Thanks. > > Andy > > > On Thu, Jan 31, 2013 at 8:44 PM,

Re: oozie error output

2013-02-01 Thread Alejandro Abdelnur
no map-reduce task launched. > > -Original Message- > From: Alejandro Abdelnur [mailto:tuc...@gmail.com] > Sent: den 1 februari 2013 17:54 > To: user@oozie.apache.org > Subject: Re: oozie error output > > moving thread to user@ (bcc dev@) > > look at the co

Re: oozie error output

2013-02-01 Thread Alejandro Abdelnur
moving thread to user@ (bcc dev@) look at the corresponding oozie launcher job in hadoop, and check the logs of its single map task. there you should have all the output from pig thx Alejandro (phone typing) On Feb 1, 2013, at 5:51 AM, Jonas Hartwig wrote: > Hi, > > I have a workflow con

Re: Hadoop proxyuser problem

2013-01-31 Thread Alejandro Abdelnur
Hadoop 1.0.4 does not support wildcards '*' in the proxyuser hosts/groups properties. Starting with Hadoop 1.1.1 this is supported. Hadoop 2.0.2-alpha supports it. You cannot patch Oozie to ignore this. This is on Hadoop side. And it has its very good reason, is to be able to control who can imper

Re: Oozie workflow error - renewing token issue

2013-01-30 Thread Alejandro Abdelnur
Cobert, [Moving thread to user@oozie.a.o, BCCing common-user@hadoop.a.o] * What version of Oozie are you using? * Is the cluster a secure setup (Kerberos enabled)? * Would you mind posting the complete launcher logs? Thx On Wed, Jan 30, 2013 at 6:14 AM, Corbett Martin wrote: > Thanks for th

Re: Is it possible to use Oozie Java/Shell action to put a file into hdfs?

2013-01-29 Thread Alejandro Abdelnur
Jinye Luo, I'm afraid it is not possible to do what you want to do in the way you want to do it. Another approach would be that you have a java action that starts a connection to your data source and writes it to HDFS. For example, if your datasource is exposed with an HTTP server this would be q

Re: [VOTE] Release Oozie 3.3.1 (candidate 1)

2013-01-17 Thread Alejandro Abdelnur
+1 verified MD5 and signature. removed release-log.txt file from rc0 and rc1 and run "find . -type f | sort | xargs -I % cat | md5" in both, got same MD5 in both (so build, install, config, run checks apply) On Thu, Jan 17, 2013 at 3:50 PM, Virag Kothari wrote: > Hi, > > The release candidate

Re: [VOTE] Release Oozie 3.3.1 (candidate 0)

2013-01-17 Thread Alejandro Abdelnur
Verified MD5 Verified signature Verified rat report Built successfully Installed and configured with Hadoop 1 Run all WF examples against Hadoop 1 There is a small NIT, the release-log.txt should say release, currently says (unreleased) After that fix +1 Thx On Thu, Jan 17, 2013 at 11:34 AM, V

Re: Oozie Timezone and Workflow Date Time

2012-12-20 Thread Alejandro Abdelnur
Hi Junaid, Coordinator job datetimes are always expressed in UTC. The timezone attribute for the coordinator-app and for datasets is only used to perform DST adjustments. For example, if you want to run something always at midnight where midnight may change -/+ 1hr because of a DST observing time

Re: Installation and Configuration of Oozie 3.3

2012-12-20 Thread Alejandro Abdelnur
nd advice. > > -- > Matt > > On Thu, Dec 20, 2012 at 3:39 AM, Alejandro Abdelnur >wrote: > > > Hi Mat, > > > > Your 11min delay seems like Oozie is using its fail safe polling > mechanism > > to detect that a job completed in the cluster. > > &

Re: Installation and Configuration of Oozie 3.3

2012-12-20 Thread Alejandro Abdelnur
> > > > > > > oozie.service.ProxyUserService.proxyuser.hadoop.hosts* > > > > > > > > > oozie.service.ProxyUserService.proxyuser.hadoop.groups* > > > > > > If I recall correctly this authorization is only checked at the the > JT/NN

Re: Accessing MR Job Counters via Oozie Client

2012-12-18 Thread Alejandro Abdelnur
Hi Siva, With Oozie 2.3.2 this is not possible. You could get the Hadoop job ids (external ID) from the client API and then for each job Id contact Hadoop and get the counters for the job. thx On Tue, Dec 18, 2012 at 8:21 PM, Siva Subramanian wrote: > Hi There, > > I need to get event count

Re: Installation and Configuration of Oozie 3.3

2012-12-13 Thread Alejandro Abdelnur
release with a -bin like most other apache projects? > 2) Are there any issues I might expect to run into when trying to run this > on EC2 backed by EMR? > > -- > Matt > > > On Thu, Dec 13, 2012 at 5:48 PM, Alejandro Abdelnur >wrote: > > > Matt, > > >

Re: Installation and Configuration of Oozie 3.3

2012-12-13 Thread Alejandro Abdelnur
Matt, Apache Oozie release artifacts are sources only. The easiest way to build the TARBALL is: * install Maven * run bin/mkdistro.sh -DskipTests Then follow the Quick Start instructions. I'll open a JIRA to add this to the docs. On Thu, Dec 13, 2012 at 3:36 PM, Matt Goeke wrote: > All, > >

Re: Publish schemas on the web site ?

2012-12-06 Thread Alejandro Abdelnur
gt; >> https://issues.apache.org/jira/browse/OOZIE-1120 >> >> >> - jun >> >> >> On Thu, Dec 6, 2012 at 12:05 PM, Mohammad Islam >> wrote: >> > Good proposal. >> > >> > --Mohammad >> > >> > >> >

Re: Publish schemas on the web site ?

2012-12-06 Thread Alejandro Abdelnur
ticket and will see what I >> can do. >> >> >> > xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; >> xsi:schemaLocation="uri:oozie:workflow:0.4 >> http://oozie.apache.org/schemas/3.1.3/oozie-workflow-0.4.xsd"; >> name="fl

Re: Publish schemas on the web site ?

2012-12-06 Thread Alejandro Abdelnur
Hi Clément, They are avail in the Oozie docs: http://oozie.apache.org/docs/3.3.0/WorkflowFunctionalSpec.html#Appendix_A_Oozie_XML-Schema All version are there. Thx On Thu, Dec 6, 2012 at 6:43 AM, Clément MATHIEU wrote: > Hi all, > > I'm wondering if the XML schemas defined by Oozie are off

Re: oozie action jars are copied to FS dervied from workflow app-path

2012-12-05 Thread Alejandro Abdelnur
Idris, [moving to user@o.a.o, bcc-ing dev@o.a.o] The Oozie app (coord and wf XMLs, files and JARs) must be in the same NN configured in the JT. Thx On Wed, Dec 5, 2012 at 2:18 AM, Idris Ali wrote: > Hi. > > Env oozie-3.2.2 > > I am running oozie's aggregator coordinator example to test one o

ANNOUNCE Apache Oozie 3.3.0 release

2012-12-03 Thread Alejandro Abdelnur
The Apache Oozie team is pleased to announce the release of Oozie 3.3.0. [This release has been driven my Mohammad Islam, I'm just ] [completing his work by pushing the bits, updating the site ] [ and making this announcement. ] The details of the release and downloa

Re: E0803: IO error,

2012-12-03 Thread Alejandro Abdelnur
Jabir, What version of Oozie are using? What database are using for Oozie? Does your workflow fail? Or it continues after you see this exception? This normally happens when a deadlock is detected and a DB connection is rolledback. Typically this is retried and your workflow job continues workin

Re: SQL Server as an alternative to derby/mySQL/Oracle?postgreSQL?

2012-12-01 Thread Alejandro Abdelnur
In addition, currently Oozie supports derby/mysql/postgres/oracle DBs, it does not support ms sqlserver. It will require modifying the persistence.xml file and creating a custom ORM file for ms sqlserver. I'd suggest opening a JIRA, I'll be glad to help adding support for another DB. Thx On

Re: [VOTE] Release Oozie 3.3.0 (candidate 0)

2012-11-26 Thread Alejandro Abdelnur
Hi Jinwei Zhu, That will happen soon after a proposed RC passes the vote. A vote on an RC typically runs for 3 days. And I'd qualify 'soon after' as more day until bits are pushed to the dist servers. Thanks. On Mon, Nov 26, 2012 at 9:19 PM, jinwei zhu wrote: > Hi, Mohammad, > You ar

Re: Coordinator Application Execution Notification

2012-11-26 Thread Alejandro Abdelnur
Hi Yongcheng Li, You could use an email action at the beginning and end of your workflow. Thx. On Wed, Nov 21, 2012 at 1:30 PM, Yongcheng Li wrote: > Hi, > > Does Oozie provide an easy way to get notification through email about > recurrent coordinator application execution? For example, I wa

Re: Oozie actions' success and failure

2012-11-20 Thread Alejandro Abdelnur
correct. exit = 0 means success, any other value error. keep in mind that no system. exit call means exit =0. thx Alejandro On Nov 19, 2012, at 11:48 PM, Ajay Srivastava wrote: > Hi, > > Any idea how does oozie decide whether an action in workflow is success ( ok > to= ) or failure ( err

Re: More Oozie 'deadlocking'

2012-11-15 Thread Alejandro Abdelnur
Chris, Oozie2 has some issues with coordinator jobs when there is backlog, repetitive internal commands fill up the processing queue. In Oozie3 this has been address by adding an ID to the commands and deduping them in the processing queue (if already in, ignore the new queueing). Because of signif

[REPORT] Apache Oozie - November Report

2012-11-12 Thread Alejandro Abdelnur
DESCRIPTION Apache Oozie is a system for managing and scheduling workflows that run different types of Hadoop jobs (such as MapReduce, Pig, Hive and Sqoop) as well as system specific jobs (such as Java programs and shell scripts). RELEASES * The last release of Apache Oozie was version 3.2.0-incu

Re: What is left for us to tackle in order to push the 3.3.0 release out?

2012-11-10 Thread Alejandro Abdelnur
hi roman, branch-3 Alejandro On Nov 10, 2012, at 1:29 PM, Roman Shaposhnik wrote: > Ping! Can anybody, please, let me know the answer to the > question I was asking bellow? > > Thanks, > Roman. > > On Fri, Nov 9, 2012 at 8:33 AM, Roman Shaposhnik wrote: >> On Thu, Nov 8, 2012 at 8:36 PM, Moh

Re: Oozie 3.2.0 and Hadoop 1.0.4

2012-11-08 Thread Alejandro Abdelnur
yep. hadoop 1.0.x does not support wildcards for proxyuser settings Alejandro On Nov 8, 2012, at 12:19 PM, Saiph Kappa wrote: > Actually I solved this by not using '*' thereby replacing by localhost and > a group which my user belongs to. > > On Thu, Nov 8, 2012 at 7:31 PM, Saiph Kappa wrote:

Re: Welcoming Robert Kanter as Oozie committer

2012-11-08 Thread Alejandro Abdelnur
congrats robert. well deserved Alejandro On Nov 7, 2012, at 11:57 PM, Mohammad Islam wrote: > Hi, > On behalf of the Oozie PPMC, I am pleased to announce that Robert Kanter has > been elected as a committer in the Apache Oozie > project. We appreciate all the work Robert has put into the proj

  1   2   >