[jira] Subscription: Oozie Patch Available

[jira]

Issue Subscription
Filter: Oozie Patch Available (53 issues)

Subscriber: ooziedaily

Key Summary
OOZIE-2272  Use Hadoop's CredentialProvider for passwords in oozie-site
https://issues.apache.org/jira/browse/OOZIE-2272
OOZIE-2271  Upgrade Tomcat to 6.0.44
https://issues.apache.org/jira/browse/OOZIE-2271
OOZIE-2266  [core] Fix 'total' actions returned in coordinator job info
https://issues.apache.org/jira/browse/OOZIE-2266
OOZIE-2262  Fix log streaming from other server with start/end filter
https://issues.apache.org/jira/browse/OOZIE-2262
OOZIE-2258  Introducing a new counter in the instrumentation log to distinguish 
between the reasons for launcher failure
https://issues.apache.org/jira/browse/OOZIE-2258
OOZIE-2251  Expose instrumental matrices in Realtime Graphing tool
https://issues.apache.org/jira/browse/OOZIE-2251
OOZIE-2250  Show log for WAITING and TIMEDOUT coord actions
https://issues.apache.org/jira/browse/OOZIE-2250
OOZIE-2245  Service to periodically check database schema
https://issues.apache.org/jira/browse/OOZIE-2245
OOZIE-2244  Oozie should mask passwords in the logs when logging command 
arguments
https://issues.apache.org/jira/browse/OOZIE-2244
OOZIE-2243  Kill Command does not kill the child job for java action
https://issues.apache.org/jira/browse/OOZIE-2243
OOZIE-2203  Fix the login example
https://issues.apache.org/jira/browse/OOZIE-2203
OOZIE-2196  Create Local Client for Bundle
https://issues.apache.org/jira/browse/OOZIE-2196
OOZIE-2187  Add a way to specify a default JT/RM and NN
https://issues.apache.org/jira/browse/OOZIE-2187
OOZIE-2168  Oozie flow and action names have 50 char limit 
https://issues.apache.org/jira/browse/OOZIE-2168
OOZIE-2159  'oozie validate' command should be moved server-side
https://issues.apache.org/jira/browse/OOZIE-2159
OOZIE-2134  Remove references to Services.get().getConf() in code
https://issues.apache.org/jira/browse/OOZIE-2134
OOZIE-2106  Make tomcat download url configurable in the pom file
https://issues.apache.org/jira/browse/OOZIE-2106
OOZIE-2105  Make version of submodules configurable with parent version 
https://issues.apache.org/jira/browse/OOZIE-2105
OOZIE-2099  Add test-patch support for patches generated without --no-prefix
https://issues.apache.org/jira/browse/OOZIE-2099
OOZIE-2081  WorkflowJob notification to include coordinator action id 
https://issues.apache.org/jira/browse/OOZIE-2081
OOZIE-2060  Incorrect documentation of Java action config XML filename
https://issues.apache.org/jira/browse/OOZIE-2060
OOZIE-2044  ssh action succeed with a not exists command which should be fail.
https://issues.apache.org/jira/browse/OOZIE-2044
OOZIE-2020  Rerun all Failed/killed/timedout coordinator actions rather than 
specifying action numbers
https://issues.apache.org/jira/browse/OOZIE-2020
OOZIE-1980  Sql error should not fail coord job
https://issues.apache.org/jira/browse/OOZIE-1980
OOZIE-1977  Display patch analysis issues
https://issues.apache.org/jira/browse/OOZIE-1977
OOZIE-1936  Queuedump command should display queue information for all server.
https://issues.apache.org/jira/browse/OOZIE-1936
OOZIE-1931  Admin command to print all locks held by server(s)
https://issues.apache.org/jira/browse/OOZIE-1931
OOZIE-1927  Use StoreStatusFilter for WorkflowsJobGetJPAExecutor 
https://issues.apache.org/jira/browse/OOZIE-1927
OOZIE-1922  MemoryLocksService fails if lock is acquired multiple times in same 
thread and released
https://issues.apache.org/jira/browse/OOZIE-1922
OOZIE-1918  ActionXCommand refactoring for code reuse
https://issues.apache.org/jira/browse/OOZIE-1918
OOZIE-1860  Oozie job mapper launch fails due to null value returned from 
action file
https://issues.apache.org/jira/browse/OOZIE-1860
OOZIE-1855  TestPriorityDelayQueue#testPoll failed intermittently in Jenkins
https://issues.apache.org/jira/browse/OOZIE-1855
OOZIE-1816  LogInfo uses action name instead of id
https://issues.apache.org/jira/browse/OOZIE-1816
OOZIE-1810  Workflow cannot get into Failed state when kill control node cannot 
resolve variable in message 
https://issues.apache.org/jira/browse/OOZIE-1810
OOZIE-1802  Support workflow action log
https://issues.apache.org/jira/browse/OOZIE-1802
OOZIE-1793  Improve find bugs reporting for Oozie
https://issues.apache.org/jira/browse/OOZIE-1793
OOZIE-1782  Workflow path not found is thrown as SC_UNAUTHORIZED
https://issues.apache.org/jira/browse/OOZIE-1782
OOZIE-1779  Add parent-id as a filter option
https://issues.apache.org/jira/browse/OOZIE-1779
OOZIE-1705  Enable gc

[jira] [Commented] (OOZIE-2253) Spark Job is failing when it is running in standalone server

[Srikanth Sundarrajan (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14587506#comment-14587506
 ] 

Srikanth Sundarrajan commented on OOZIE-2253:
-

Looks good. +1

> Spark Job is failing when it is running in standalone server
> 
>
> Key: OOZIE-2253
> URL: https://issues.apache.org/jira/browse/OOZIE-2253
> Project: Oozie
>  Issue Type: Bug
>Reporter: pavan kumar kolamuri
>Assignee: pavan kumar kolamuri
> Attachments: OOZIE-2253.patch
>
>
> When Spark Job is running in spark standalone cluster the job is getting 
> launched and succedded and infinite jobs are getting launched in spark 
> cluster. Oozie workflow will be in running state forever as spark is 
> launching job infinite times. 
> This might be because in spark when job succeeds and it always do 
> System.exit(0) . In LauncherSecurityManager  exception is thrown for this. It 
> looks like spark(through akka framework)  is catching that and launching one 
> more attempt for the same job. It is happening infinitely .
> {noformat}
> Sending launch command to spark://inmobi-Precision-T3610:7077
> Driver successfully submitted as driver-20150526105806-
> ... waiting before polling master for driver state
> ... polling master for driver state
> State of driver-20150526105806- is SUBMITTED
> Sending launch command to spark://inmobi-Precision-T3610:7077
> Driver successfully submitted as driver-20150526105811-0001
> ... waiting before polling master for driver state
> ... polling master for driver state
> State of driver-20150526105811-0001 is SUBMITTED
> Sending launch command to spark://inmobi-Precision-T3610:7077
> Driver successfully submitted as driver-20150526105816-0002
> ... waiting before polling master for driver state
> ... polling master for driver state
> State of driver-20150526105816-0002 is SUBMITTED
> Sending launch command to spark://inmobi-Precision-T3610:7077
> Driver successfully submitted as driver-20150526105821-0003
> ... waiting before polling master for driver state
> ... polling master for driver state
> State of driver-20150526105821-0003 is SUBMITTED
> Sending launch command to spark://inmobi-Precision-T3610:7077
> Driver successfully submitted as driver-20150526105826-0004
> ... waiting before polling master for driver state
> {noformat}
> {noformat}
> 2015-05-26 10:58:11,573 ERROR [driverClient-akka.actor.default-dispatcher-4] 
> akka.actor.OneForOneStrategy: Intercepted System.exit(0)
> java.lang.SecurityException: Intercepted System.exit(0)
>   at 
> org.apache.oozie.action.hadoop.LauncherSecurityManager.checkExit(LauncherMapper.java:601)
>   at java.lang.Runtime.exit(Runtime.java:107)
>   at java.lang.System.exit(System.java:962)
>   at 
> org.apache.spark.deploy.ClientActor.pollAndReportStatus(Client.scala:115)
>   at 
> org.apache.spark.deploy.ClientActor$$anonfun$receiveWithLogging$1.applyOrElse(Client.scala:123)
>   at 
> scala.runtime.AbstractPartialFunction$mcVL$sp.apply$mcVL$sp(AbstractPartialFunction.scala:33)
>   at 
> scala.runtime.AbstractPartialFunction$mcVL$sp.apply(AbstractPartialFunction.scala:33)
>   at 
> scala.runtime.AbstractPartialFunction$mcVL$sp.apply(AbstractPartialFunction.scala:25)
>   at 
> org.apache.spark.util.ActorLogReceive$$anon$1.apply(ActorLogReceive.scala:53)
>   at 
> org.apache.spark.util.ActorLogReceive$$anon$1.apply(ActorLogReceive.scala:42)
>   at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:118)
>   at 
> org.apache.spark.util.ActorLogReceive$$anon$1.applyOrElse(ActorLogReceive.scala:42)
>   at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)
>   at akka.actor.ActorCell.invoke(ActorCell.scala:456)
>   at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
>   at akka.dispatch.Mailbox.run(Mailbox.scala:219)
>   at 
> akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)
>   at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
>   at 
> scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
>   at 
> scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
>   at 
> scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 35488: Fixed spark action in yarn mode and standalone mode

[Srikanth Sundarrajan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35488/#review88030
---

Ship it!


Ship It!

- Srikanth Sundarrajan


On June 16, 2015, 5:09 a.m., pavan kumar kolamuri wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35488/
> ---
> 
> (Updated June 16, 2015, 5:09 a.m.)
> 
> 
> Review request for oozie and Srikanth Sundarrajan.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> Spark job launched through spark action is not running as expected in 
> standalone mode and  it is not running in yarn mode
> 
> 
> Diffs
> -
> 
>   pom.xml ee26366 
>   sharelib/spark/pom.xml 3fe9c71 
>   src/main/assemblies/sharelib.xml ea95c2e 
> 
> Diff: https://reviews.apache.org/r/35488/diff/
> 
> 
> Testing
> ---
> 
> Basic testing done. Deployed spark job on yarn, standalone cluster and local 
> mode . It is running successfully in all these modes.
> 
> 
> Thanks,
> 
> pavan kumar kolamuri
> 
>

[jira] [Commented] (OOZIE-2253) Spark Job is failing when it is running in standalone server

[pavan kumar kolamuri (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14587493#comment-14587493
 ] 

pavan kumar kolamuri commented on OOZIE-2253:
-

[~sriksun]  Updated to review board ,  attached the link here.

> Spark Job is failing when it is running in standalone server
> 
>
> Key: OOZIE-2253
> URL: https://issues.apache.org/jira/browse/OOZIE-2253
> Project: Oozie
>  Issue Type: Bug
>Reporter: pavan kumar kolamuri
>Assignee: pavan kumar kolamuri
> Attachments: OOZIE-2253.patch
>
>
> When Spark Job is running in spark standalone cluster the job is getting 
> launched and succedded and infinite jobs are getting launched in spark 
> cluster. Oozie workflow will be in running state forever as spark is 
> launching job infinite times. 
> This might be because in spark when job succeeds and it always do 
> System.exit(0) . In LauncherSecurityManager  exception is thrown for this. It 
> looks like spark(through akka framework)  is catching that and launching one 
> more attempt for the same job. It is happening infinitely .
> {noformat}
> Sending launch command to spark://inmobi-Precision-T3610:7077
> Driver successfully submitted as driver-20150526105806-
> ... waiting before polling master for driver state
> ... polling master for driver state
> State of driver-20150526105806- is SUBMITTED
> Sending launch command to spark://inmobi-Precision-T3610:7077
> Driver successfully submitted as driver-20150526105811-0001
> ... waiting before polling master for driver state
> ... polling master for driver state
> State of driver-20150526105811-0001 is SUBMITTED
> Sending launch command to spark://inmobi-Precision-T3610:7077
> Driver successfully submitted as driver-20150526105816-0002
> ... waiting before polling master for driver state
> ... polling master for driver state
> State of driver-20150526105816-0002 is SUBMITTED
> Sending launch command to spark://inmobi-Precision-T3610:7077
> Driver successfully submitted as driver-20150526105821-0003
> ... waiting before polling master for driver state
> ... polling master for driver state
> State of driver-20150526105821-0003 is SUBMITTED
> Sending launch command to spark://inmobi-Precision-T3610:7077
> Driver successfully submitted as driver-20150526105826-0004
> ... waiting before polling master for driver state
> {noformat}
> {noformat}
> 2015-05-26 10:58:11,573 ERROR [driverClient-akka.actor.default-dispatcher-4] 
> akka.actor.OneForOneStrategy: Intercepted System.exit(0)
> java.lang.SecurityException: Intercepted System.exit(0)
>   at 
> org.apache.oozie.action.hadoop.LauncherSecurityManager.checkExit(LauncherMapper.java:601)
>   at java.lang.Runtime.exit(Runtime.java:107)
>   at java.lang.System.exit(System.java:962)
>   at 
> org.apache.spark.deploy.ClientActor.pollAndReportStatus(Client.scala:115)
>   at 
> org.apache.spark.deploy.ClientActor$$anonfun$receiveWithLogging$1.applyOrElse(Client.scala:123)
>   at 
> scala.runtime.AbstractPartialFunction$mcVL$sp.apply$mcVL$sp(AbstractPartialFunction.scala:33)
>   at 
> scala.runtime.AbstractPartialFunction$mcVL$sp.apply(AbstractPartialFunction.scala:33)
>   at 
> scala.runtime.AbstractPartialFunction$mcVL$sp.apply(AbstractPartialFunction.scala:25)
>   at 
> org.apache.spark.util.ActorLogReceive$$anon$1.apply(ActorLogReceive.scala:53)
>   at 
> org.apache.spark.util.ActorLogReceive$$anon$1.apply(ActorLogReceive.scala:42)
>   at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:118)
>   at 
> org.apache.spark.util.ActorLogReceive$$anon$1.applyOrElse(ActorLogReceive.scala:42)
>   at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)
>   at akka.actor.ActorCell.invoke(ActorCell.scala:456)
>   at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
>   at akka.dispatch.Mailbox.run(Mailbox.scala:219)
>   at 
> akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)
>   at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
>   at 
> scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
>   at 
> scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
>   at 
> scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Review Request 35488: Fixed spark action in yarn mode and standalone mode

[pavan kumar kolamuri]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35488/
---

Review request for oozie and Srikanth Sundarrajan.


Repository: oozie-git


Description
---

Spark job launched through spark action is not running as expected in 
standalone mode and  it is not running in yarn mode


Diffs
-

  pom.xml ee26366 
  sharelib/spark/pom.xml 3fe9c71 
  src/main/assemblies/sharelib.xml ea95c2e 

Diff: https://reviews.apache.org/r/35488/diff/


Testing
---

Basic testing done. Deployed spark job on yarn, standalone cluster and local 
mode . It is running successfully in all these modes.


Thanks,

pavan kumar kolamuri

[jira] [Commented] (OOZIE-2253) Spark Job is failing when it is running in standalone server

[Srikanth Sundarrajan (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14587482#comment-14587482
 ] 

Srikanth Sundarrajan commented on OOZIE-2253:
-

Can you please upload to review board ?


> Spark Job is failing when it is running in standalone server
> 
>
> Key: OOZIE-2253
> URL: https://issues.apache.org/jira/browse/OOZIE-2253
> Project: Oozie
>  Issue Type: Bug
>Reporter: pavan kumar kolamuri
>Assignee: pavan kumar kolamuri
> Attachments: OOZIE-2253.patch
>
>
> When Spark Job is running in spark standalone cluster the job is getting 
> launched and succedded and infinite jobs are getting launched in spark 
> cluster. Oozie workflow will be in running state forever as spark is 
> launching job infinite times. 
> This might be because in spark when job succeeds and it always do 
> System.exit(0) . In LauncherSecurityManager  exception is thrown for this. It 
> looks like spark(through akka framework)  is catching that and launching one 
> more attempt for the same job. It is happening infinitely .
> {noformat}
> Sending launch command to spark://inmobi-Precision-T3610:7077
> Driver successfully submitted as driver-20150526105806-
> ... waiting before polling master for driver state
> ... polling master for driver state
> State of driver-20150526105806- is SUBMITTED
> Sending launch command to spark://inmobi-Precision-T3610:7077
> Driver successfully submitted as driver-20150526105811-0001
> ... waiting before polling master for driver state
> ... polling master for driver state
> State of driver-20150526105811-0001 is SUBMITTED
> Sending launch command to spark://inmobi-Precision-T3610:7077
> Driver successfully submitted as driver-20150526105816-0002
> ... waiting before polling master for driver state
> ... polling master for driver state
> State of driver-20150526105816-0002 is SUBMITTED
> Sending launch command to spark://inmobi-Precision-T3610:7077
> Driver successfully submitted as driver-20150526105821-0003
> ... waiting before polling master for driver state
> ... polling master for driver state
> State of driver-20150526105821-0003 is SUBMITTED
> Sending launch command to spark://inmobi-Precision-T3610:7077
> Driver successfully submitted as driver-20150526105826-0004
> ... waiting before polling master for driver state
> {noformat}
> {noformat}
> 2015-05-26 10:58:11,573 ERROR [driverClient-akka.actor.default-dispatcher-4] 
> akka.actor.OneForOneStrategy: Intercepted System.exit(0)
> java.lang.SecurityException: Intercepted System.exit(0)
>   at 
> org.apache.oozie.action.hadoop.LauncherSecurityManager.checkExit(LauncherMapper.java:601)
>   at java.lang.Runtime.exit(Runtime.java:107)
>   at java.lang.System.exit(System.java:962)
>   at 
> org.apache.spark.deploy.ClientActor.pollAndReportStatus(Client.scala:115)
>   at 
> org.apache.spark.deploy.ClientActor$$anonfun$receiveWithLogging$1.applyOrElse(Client.scala:123)
>   at 
> scala.runtime.AbstractPartialFunction$mcVL$sp.apply$mcVL$sp(AbstractPartialFunction.scala:33)
>   at 
> scala.runtime.AbstractPartialFunction$mcVL$sp.apply(AbstractPartialFunction.scala:33)
>   at 
> scala.runtime.AbstractPartialFunction$mcVL$sp.apply(AbstractPartialFunction.scala:25)
>   at 
> org.apache.spark.util.ActorLogReceive$$anon$1.apply(ActorLogReceive.scala:53)
>   at 
> org.apache.spark.util.ActorLogReceive$$anon$1.apply(ActorLogReceive.scala:42)
>   at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:118)
>   at 
> org.apache.spark.util.ActorLogReceive$$anon$1.applyOrElse(ActorLogReceive.scala:42)
>   at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)
>   at akka.actor.ActorCell.invoke(ActorCell.scala:456)
>   at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
>   at akka.dispatch.Mailbox.run(Mailbox.scala:219)
>   at 
> akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)
>   at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
>   at 
> scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
>   at 
> scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
>   at 
> scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Build failed in Jenkins: oozie-trunk-precommit-build #2416

[Apache Jenkins Server]

See 

Changes:

[purushah] OOZIE-2264 Fix coord:offset(n,DAY) to resolve correct data set

--
[...truncated 9168 lines...]
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @ oozie-tools ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:2.0-beta-6:attach-descriptor (attach-descriptor) @ 
oozie-tools ---
[INFO] 
[INFO] --- maven-assembly-plugin:2.2.1:single (default-cli) @ oozie-tools ---
[INFO] Reading assembly descriptor: ../src/main/assemblies/tools.xml
[WARNING] The following patterns were never triggered in this artifact 
exclusion filter:
o  '*:*:pom:*'

[INFO] Copying files to 

[WARNING] Assembly file: 

 is not a regular file (it may be a directory). It cannot be attached to the 
project build for installation or deployment.
[INFO] 
[INFO] 
[INFO] Building Apache Oozie MiniOozie 4.3.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ oozie-mini ---
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
oozie-mini ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ oozie-mini 
---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
oozie-mini ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 3 resources
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
oozie-mini ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ oozie-mini ---
[INFO] Tests are skipped.
[INFO] 
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @ oozie-mini ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:2.0-beta-6:attach-descriptor (attach-descriptor) @ 
oozie-mini ---
[INFO] 
[INFO] --- maven-assembly-plugin:2.2.1:single (default-cli) @ oozie-mini ---
[INFO] Reading assembly descriptor: src/main/assemblies/empty.xml
[INFO] 
[INFO] 
[INFO] Building Apache Oozie Distro 4.3.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ oozie-distro 
---
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
oozie-distro ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ oozie-distro 
---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
oozie-distro ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
oozie-distro ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ oozie-distro ---
[INFO] Tests are skipped.
[INFO] 
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @ oozie-distro ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:2.0-beta-6:attach-descriptor (attach-descriptor) @ 
oozie-distro ---
[INFO] 
[INFO] --- maven-antrun-plugin:1.6:run (default) @ oozie-distro ---
[INFO] Executing tasks

main:
  [get] Getting: 
http://archive.apache.org/dist/tomcat/tomcat-6/v6.0.43/bin/apache-tomcat-6.0.43.tar.gz
  [get] To

[jira] [Updated] (OOZIE-2274) Last_only can execute older actions

[Purshotam Shah (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Purshotam Shah updated OOZIE-2274:
--
Description: 
Ex.
If job concurrency = 1.
If action 1 takes long time to complete and meanwhile if action 2 and action 3 
nominal time is reached.
While action 1 is still running, action 2 and 3 will move to ready state after 
input check. 
At end of coord action 1, Oozie will execute coord action 3. But at end of 
coord action 3, Oozie might execute action 2.


> Last_only can execute older actions
> ---
>
> Key: OOZIE-2274
> URL: https://issues.apache.org/jira/browse/OOZIE-2274
> Project: Oozie
>  Issue Type: Bug
>Reporter: Purshotam Shah
>
> Ex.
> If job concurrency = 1.
> If action 1 takes long time to complete and meanwhile if action 2 and action 
> 3 nominal time is reached.
> While action 1 is still running, action 2 and 3 will move to ready state 
> after input check. 
> At end of coord action 1, Oozie will execute coord action 3. But at end of 
> coord action 3, Oozie might execute action 2.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (OOZIE-2274) Last_only can execute older actions

[Purshotam Shah (JIRA)]

Purshotam Shah created OOZIE-2274:
-

 Summary: Last_only can execute older actions
 Key: OOZIE-2274
 URL: https://issues.apache.org/jira/browse/OOZIE-2274
 Project: Oozie
  Issue Type: Bug
Reporter: Purshotam Shah






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Build failed in Jenkins: oozie-trunk-precommit-build #2415

[Apache Jenkins Server]

See 

--
[...truncated 14512 lines...]
[INFO] --- maven-site-plugin:2.0-beta-6:attach-descriptor (attach-descriptor) @ 
oozie-tools ---
[INFO] 
[INFO] --- maven-assembly-plugin:2.2.1:single (default-cli) @ oozie-tools ---
[INFO] Reading assembly descriptor: ../src/main/assemblies/tools.xml
[WARNING] The following patterns were never triggered in this artifact 
exclusion filter:
o  '*:*:pom:*'

[INFO] Copying files to 

[WARNING] Assembly file: 

 is not a regular file (it may be a directory). It cannot be attached to the 
project build for installation or deployment.
[INFO] 
[INFO] 
[INFO] Building Apache Oozie MiniOozie 4.3.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ oozie-mini ---
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
oozie-mini ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ oozie-mini 
---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
oozie-mini ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 3 resources
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
oozie-mini ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ oozie-mini ---
[INFO] Tests are skipped.
[INFO] 
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @ oozie-mini ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:2.0-beta-6:attach-descriptor (attach-descriptor) @ 
oozie-mini ---
[INFO] 
[INFO] --- maven-assembly-plugin:2.2.1:single (default-cli) @ oozie-mini ---
[INFO] Reading assembly descriptor: src/main/assemblies/empty.xml
[INFO] 
[INFO] 
[INFO] Building Apache Oozie Distro 4.3.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ oozie-distro 
---
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
oozie-distro ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ oozie-distro 
---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
oozie-distro ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
oozie-distro ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ oozie-distro ---
[INFO] Tests are skipped.
[INFO] 
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @ oozie-distro ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:2.0-beta-6:attach-descriptor (attach-descriptor) @ 
oozie-distro ---
[INFO] 
[INFO] --- maven-antrun-plugin:1.6:run (default) @ oozie-distro ---
[INFO] Executing tasks

main:
  [get] Getting: 
http://archive.apache.org/dist/tomcat/tomcat-6/v6.0.43/bin/apache-tomcat-6.0.43.tar.gz
  [get] To: 

..

[jira] [Updated] (OOZIE-2266) [core] Fix 'total' actions returned in coordinator job info

[Sai Chirravuri (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sai Chirravuri updated OOZIE-2266:
--
Attachment: OOZIE-2266.patch

Added unit tests

> [core] Fix 'total' actions returned in coordinator job info
> ---
>
> Key: OOZIE-2266
> URL: https://issues.apache.org/jira/browse/OOZIE-2266
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Affects Versions: 4.2.0
>Reporter: Sai Chirravuri
>Assignee: Sai Chirravuri
> Attachments: 
> 0001-OOZIE-2266-core-Fix-total-actions-in-coord-info.patch, OOZIE-2266.patch, 
> OOZIE-2266.patch
>
>
> Coordinator job info doesn't take filtering into account when returning 
> 'total' actions. It always returns total actions.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2264) Fix coord:offset(n,"DAY") to resolve correct data set

[Purshotam Shah (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14586721#comment-14586721
 ] 

Purshotam Shah commented on OOZIE-2264:
---

+1. Thanks [~kailongs] for fixing this. Congrats for your first Oozie patch.

> Fix coord:offset(n,"DAY") to resolve correct data set
> -
>
> Key: OOZIE-2264
> URL: https://issues.apache.org/jira/browse/OOZIE-2264
> Project: Oozie
>  Issue Type: Bug
>Reporter: Kailong Sheng
>Assignee: Kailong Sheng
> Attachments: OOZIE-2264.patch
>
>
> Time unit was hardcoded as "Minute" when materializing the instances.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Review Request 35472: OOZIE-2245: Service to periodically check database schema

[Robert Kanter]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35472/
---

Review request for oozie.


Bugs: OOZIE-2245
https://issues.apache.org/jira/browse/OOZIE-2245


Repository: oozie-git


Description
---

See 
https://issues.apache.org/jira/browse/OOZIE-2245?focusedCommentId=14555005&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14555005


Diffs
-

  core/src/main/java/org/apache/oozie/command/SchemaCheckXCommand.java 
PRE-CREATION 
  core/src/main/java/org/apache/oozie/service/SchemaCheckerService.java 
PRE-CREATION 
  core/src/main/resources/oozie-default.xml 8960073 

Diff: https://reviews.apache.org/r/35472/diff/


Testing
---


Thanks,

Robert Kanter

[jira] [Created] (OOZIE-2273) MiniOozie does not work outside of Oozie

[Robert Kanter (JIRA)]

Robert Kanter created OOZIE-2273:


 Summary: MiniOozie does not work outside of Oozie
 Key: OOZIE-2273
 URL: https://issues.apache.org/jira/browse/OOZIE-2273
 Project: Oozie
  Issue Type: Bug
  Components: tests
Affects Versions: 4.2.0
Reporter: Robert Kanter


{{MiniOozie}} doesn't appear to work correctly outside of Oozie, due to some 
hardcoded paths/assumptions.  We should fix this.

As a quick workaround, I added the "core" folder and sub folders/files.  For 
example:
{noformat}
project-root
|- src
|- pom.xml
|- (other typical maven stuff)
|- core
|- |- src
|- |- |- test
|- |- |- |- resources
|- |- |- |- |- hadoop-config.xml (a copy of this file [1])
{noformat}

Also, the Metastore doesn't work properly either (not sure why).  It can be 
disabled by adding {{System.setProperty("oozie.test.metastore.server", 
"false");}} in the {{setUp()}} method, as done at \[2], but we should fix that 
as well.

\[1] 
https://raw.githubusercontent.com/apache/oozie/master/core/src/test/resources/hadoop-config.xml
\[2] 
https://github.com/apache/oozie/blob/master/minitest/src/test/java/org/apache/oozie/test/WorkflowTest.java#L45



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2030) Configuration properties from global section is not getting set in Hadoop job conf when using sub-workflow action in Oozie workflow.xml

[Rohini Palaniswamy (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14586535#comment-14586535
 ] 

Rohini Palaniswamy commented on OOZIE-2030:
---

Agree. If we are serializing whole global section into conf, need to ensure 
that the serialized conf does not end up in multiple places - each action conf, 
job.xml on hdfs, etc which will bloat size a lot and that inefficiency will end 
up being a problem at scale.  So [~jaydeepvishwakarma], in the implementation 
take care to only process that serialized config in LiteWorkflowAppParser and 
throw it away before the actionconf is constructed and stored in database.  
[~rkanter] has done some cleanup of the global section in OOZIE-2187. Would be 
good to base your patch on top of that.

Also please base64 encode the setting value, in case it causes some issues with 
the xml format.

> Configuration properties from global section is not getting set in Hadoop job 
> conf when using sub-workflow action in Oozie workflow.xml 
> 
>
> Key: OOZIE-2030
> URL: https://issues.apache.org/jira/browse/OOZIE-2030
> Project: Oozie
>  Issue Type: Bug
>  Components: action
>Reporter: Peeyush Bishnoi
>Assignee: Jaydeep Vishwakarma
> Attachments: OOZIE-2030-v2.patch, OOZIE-2030-v3.patch, 
> OOZIE-2030-v4.patch, OOZIE-2030.patch
>
>
> When submitting Oozie workflow with sub-workflow action and with global 
> section, configuration properties defined in global section is not getting 
> set in launched Hadoop job conf. But when we use Pig or MR action in 
> workflow.xml, configuration properties from global section set properly into 
> Hadoop job conf.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2020) Rerun all Failed/killed/timedout coordinator actions rather than specifying action numbers

[Robert Kanter (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14586405#comment-14586405
 ] 

Robert Kanter commented on OOZIE-2020:
--

Hue is probably the biggest consumer of our REST API.  Some of our Cloudera 
Manager stuff might be using some of the APIs.
In any case, I don't think we can change the API incompatibly at this point.  
We could move to a v3 API though... I think I have some old JIRA that is 
waiting for review somewhere that wanted to go to v3 as well.

> Rerun all Failed/killed/timedout coordinator actions rather than specifying 
> action numbers
> --
>
> Key: OOZIE-2020
> URL: https://issues.apache.org/jira/browse/OOZIE-2020
> Project: Oozie
>  Issue Type: New Feature
>  Components: action
>Reporter: Sreedish P S
>Assignee: Narayan Periwal
>Priority: Minor
> Attachments: OOZIE-2020-v10.patch, OOZIE-2020-v11.patch, 
> OOZIE-2020-v12.patch, OOZIE-2020-v13.patch, OOZIE-2020-v14.patch, 
> OOZIE-2020-v15.patch, OOZIE-2020-v16.patch, OOZIE-2020-v17.patch, 
> OOZIE-2020-v8.patch, OOZIE-2020-v9.patch
>
>
> Currently rerun of coordinator actions are made through coordinator id and 
> action numbers, this feature request is for rerunning all coordinator actions 
> by mentioning a particular state
> for example :
> oozie job -rerun coord-id -state killed



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 34854: OOZIE-2251 : Expose instrumental matrices in Realtime Graphing tool

[Srikanth Sundarrajan]

> On June 15, 2015, 6:25 a.m., Shwetha GS wrote:
> > core/src/main/java/org/apache/oozie/util/MetricsInstrumentation.java, line 
> > 108
> > 
> >
> > change log level to debug

Since this is logged only once at the startup of the oozie service, it would be 
ok to leave this at Info level, as it is useful to know what the metrics sync 
is.


- Srikanth


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34854/#review87894
---


On June 15, 2015, 12:49 p.m., Narayan Periwal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34854/
> ---
> 
> (Updated June 15, 2015, 12:49 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-2251
> https://issues.apache.org/jira/browse/OOZIE-2251
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> We have been logging so many important matrices in oozie-instrumentation.log 
> . These information is very useful for oozie functional monitoring. But it is 
> always difficult to get the meaning from flat file. If we expose this 
> information on some graphing tool, We can get the lot of meaning out of it 
> and can take some actions based on it.
> 
> 
> Diffs
> -
> 
>   core/pom.xml 783 
>   
> core/src/main/java/org/apache/oozie/service/MetricsInstrumentationService.java
>  2a00a73 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 4eb6386 
>   core/src/main/java/org/apache/oozie/util/MetricsInstrumentation.java 
> 185b67e 
>   core/src/main/resources/oozie-default.xml 4dc127e 
>   docs/src/site/twiki/AG_Install.twiki 0ce2609 
>   webapp/pom.xml e42e219 
> 
> Diff: https://reviews.apache.org/r/34854/diff/
> 
> 
> Testing
> ---
> 
> Done
> 
> 
> Thanks,
> 
> Narayan Periwal
> 
>

[jira] [Commented] (OOZIE-2251) Expose instrumental matrices in Realtime Graphing tool

[Robert Kanter (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14586338#comment-14586338
 ] 

Robert Kanter commented on OOZIE-2251:
--

[~nperiwal] I left you some comments on RB.

Also, are any of the license-incompatible dependencies that we're excluding 
required at runtime?  Or just at compile time?  If we also need them at 
runtime, can you add some docs on what to add so users can do it?

> Expose instrumental matrices in Realtime Graphing tool
> --
>
> Key: OOZIE-2251
> URL: https://issues.apache.org/jira/browse/OOZIE-2251
> Project: Oozie
>  Issue Type: New Feature
>  Components: monitoring
>Reporter: Jaydeep Vishwakarma
>Assignee: Narayan Periwal
> Attachments: OOZIE-2251-v0.patch, OOZIE-2251-v1.patch, 
> OOZIE-2251-v2.patch, OOZIE-2251-v3.patch, OOZIE-2251-v4.patch, 
> OOZIE-2251-v5.patch, OOZIE-2251-v6.patch, OOZIE-2251-v7.patch, 
> OOZIE-2251-v8.patch
>
>
> We have been logging so many important matrices in oozie-instrumentation.log 
> . These information is very useful for oozie functional monitoring. But it is 
> always difficult to get the meaning from flat file. If we expose this 
> information on some graphing tool, We can get the lot of meaning out of it 
> and can take some actions based on it. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 34854: OOZIE-2251 : Expose instrumental matrices in Realtime Graphing tool

[Robert Kanter]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34854/#review87934
---



core/pom.xml


Versions should go in the root pom.

Also, io.dropwizard.metrics are the same thing as com.codahale.metrics 
(presumably newer because the latest version is higher and it's now what's 
mentioned on their website).  We already have some dependencides on the latter, 
and it would be good to keep this all consistent.  Can you replace the existing 
com.codahale.metrics dependencies with io.dropwizard.metrics?  

And given that we'll have a bunch of these dependencies, it's probably a 
good idea to create a variable for the version (like we do with some others 
such as tomcat).  For example, "dropwizard.metrics.version" or something.



core/src/main/java/org/apache/oozie/util/MetricsInstrumentation.java


All of these property names should be public static final String class 
variables



core/src/main/java/org/apache/oozie/util/MetricsInstrumentation.java


Let's just create a String for metricsServerName.trim().toLowerCase() so we 
don't have to do it so many times



core/src/main/java/org/apache/oozie/util/MetricsInstrumentation.java


All configs show up in the admin/configuration endpoint, so there's no 
reason to log it this way.  If you want to have debug logging with this info, 
that's still fine, but let's make it more consistent with other log statements. 
 e.g.
"Publishing external monitoring to [{0}] every [{1}] seconds" etc



core/src/main/java/org/apache/oozie/util/MetricsInstrumentation.java


This can just be LOG.error(



core/src/main/java/org/apache/oozie/util/MetricsInstrumentation.java


LOG.error(



webapp/pom.xml


From the JIRA, we're excluding this because of a license problem.  Can you 
add a comment in the XML here mentioning that so we don't accidently un-exclude 
it later?


- Robert Kanter


On June 15, 2015, 12:49 p.m., Narayan Periwal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34854/
> ---
> 
> (Updated June 15, 2015, 12:49 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-2251
> https://issues.apache.org/jira/browse/OOZIE-2251
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> We have been logging so many important matrices in oozie-instrumentation.log 
> . These information is very useful for oozie functional monitoring. But it is 
> always difficult to get the meaning from flat file. If we expose this 
> information on some graphing tool, We can get the lot of meaning out of it 
> and can take some actions based on it.
> 
> 
> Diffs
> -
> 
>   core/pom.xml 783 
>   
> core/src/main/java/org/apache/oozie/service/MetricsInstrumentationService.java
>  2a00a73 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 4eb6386 
>   core/src/main/java/org/apache/oozie/util/MetricsInstrumentation.java 
> 185b67e 
>   core/src/main/resources/oozie-default.xml 4dc127e 
>   docs/src/site/twiki/AG_Install.twiki 0ce2609 
>   webapp/pom.xml e42e219 
> 
> Diff: https://reviews.apache.org/r/34854/diff/
> 
> 
> Testing
> ---
> 
> Done
> 
> 
> Thanks,
> 
> Narayan Periwal
> 
>

[jira] [Commented] (OOZIE-2251) Expose instrumental matrices in Realtime Graphing tool

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14586137#comment-14586137
 ] 

Hadoop QA commented on OOZIE-2251:
--

Testing JIRA OOZIE-2251

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:red}-1{color} the patch does not add/modify any testcase
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 1653
.Tests failed: 1
.Tests errors: 5

.The patch failed the following testcases:

.  testForNoDuplicates(org.apache.oozie.event.TestEventGeneration)

{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}


The full output of the test-patch run is available at

.   https://builds.apache.org/job/oozie-trunk-precommit-build/2414/

> Expose instrumental matrices in Realtime Graphing tool
> --
>
> Key: OOZIE-2251
> URL: https://issues.apache.org/jira/browse/OOZIE-2251
> Project: Oozie
>  Issue Type: New Feature
>  Components: monitoring
>Reporter: Jaydeep Vishwakarma
>Assignee: Narayan Periwal
> Attachments: OOZIE-2251-v0.patch, OOZIE-2251-v1.patch, 
> OOZIE-2251-v2.patch, OOZIE-2251-v3.patch, OOZIE-2251-v4.patch, 
> OOZIE-2251-v5.patch, OOZIE-2251-v6.patch, OOZIE-2251-v7.patch, 
> OOZIE-2251-v8.patch
>
>
> We have been logging so many important matrices in oozie-instrumentation.log 
> . These information is very useful for oozie functional monitoring. But it is 
> always difficult to get the meaning from flat file. If we expose this 
> information on some graphing tool, We can get the lot of meaning out of it 
> and can take some actions based on it. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Build failed in Jenkins: oozie-trunk-precommit-build #2414

[Apache Jenkins Server]

See 

--
[...truncated 8404 lines...]
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ oozie-tools ---
[INFO] Tests are skipped.
[INFO] 
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @ oozie-tools ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:2.0-beta-6:attach-descriptor (attach-descriptor) @ 
oozie-tools ---
[INFO] 
[INFO] --- maven-assembly-plugin:2.2.1:single (default-cli) @ oozie-tools ---
[INFO] Reading assembly descriptor: ../src/main/assemblies/tools.xml
[WARNING] The following patterns were never triggered in this artifact 
exclusion filter:
o  '*:*:pom:*'

[INFO] Copying files to 

[WARNING] Assembly file: 

 is not a regular file (it may be a directory). It cannot be attached to the 
project build for installation or deployment.
[INFO] 
[INFO] 
[INFO] Building Apache Oozie MiniOozie 4.3.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ oozie-mini ---
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
oozie-mini ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ oozie-mini 
---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
oozie-mini ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 3 resources
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
oozie-mini ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ oozie-mini ---
[INFO] Tests are skipped.
[INFO] 
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @ oozie-mini ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:2.0-beta-6:attach-descriptor (attach-descriptor) @ 
oozie-mini ---
[INFO] 
[INFO] --- maven-assembly-plugin:2.2.1:single (default-cli) @ oozie-mini ---
[INFO] Reading assembly descriptor: src/main/assemblies/empty.xml
[INFO] 
[INFO] 
[INFO] Building Apache Oozie Distro 4.3.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ oozie-distro 
---
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
oozie-distro ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ oozie-distro 
---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
oozie-distro ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
oozie-distro ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ oozie-distro ---
[INFO] Tests are skipped.
[INFO] 
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @ oozie-distro ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:2.0-beta-6:attach-descriptor (attach-descriptor) @ 
oozie-distro ---
[INFO] 
[INFO] --- maven-antrun-plugin:1.6:run (default) @ oozie-distro ---
[INFO] Executing tasks

main:
  [get] Getting: 
http://archive.apache.org/dist/tomcat/tomcat-6/v6.0.43/bin/apache-tomcat-6.0.43.t

[jira] [Updated] (OOZIE-2251) Expose instrumental matrices in Realtime Graphing tool

[Narayan Periwal (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Narayan Periwal updated OOZIE-2251:
---
Attachment: OOZIE-2251-v8.patch

> Expose instrumental matrices in Realtime Graphing tool
> --
>
> Key: OOZIE-2251
> URL: https://issues.apache.org/jira/browse/OOZIE-2251
> Project: Oozie
>  Issue Type: New Feature
>  Components: monitoring
>Reporter: Jaydeep Vishwakarma
>Assignee: Narayan Periwal
> Attachments: OOZIE-2251-v0.patch, OOZIE-2251-v1.patch, 
> OOZIE-2251-v2.patch, OOZIE-2251-v3.patch, OOZIE-2251-v4.patch, 
> OOZIE-2251-v5.patch, OOZIE-2251-v6.patch, OOZIE-2251-v7.patch, 
> OOZIE-2251-v8.patch
>
>
> We have been logging so many important matrices in oozie-instrumentation.log 
> . These information is very useful for oozie functional monitoring. But it is 
> always difficult to get the meaning from flat file. If we expose this 
> information on some graphing tool, We can get the lot of meaning out of it 
> and can take some actions based on it. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2251) Expose instrumental matrices in Realtime Graphing tool

[Narayan Periwal (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Narayan Periwal updated OOZIE-2251:
---
Attachment: (was: OOZIE-2251-v8.patch)

> Expose instrumental matrices in Realtime Graphing tool
> --
>
> Key: OOZIE-2251
> URL: https://issues.apache.org/jira/browse/OOZIE-2251
> Project: Oozie
>  Issue Type: New Feature
>  Components: monitoring
>Reporter: Jaydeep Vishwakarma
>Assignee: Narayan Periwal
> Attachments: OOZIE-2251-v0.patch, OOZIE-2251-v1.patch, 
> OOZIE-2251-v2.patch, OOZIE-2251-v3.patch, OOZIE-2251-v4.patch, 
> OOZIE-2251-v5.patch, OOZIE-2251-v6.patch, OOZIE-2251-v7.patch
>
>
> We have been logging so many important matrices in oozie-instrumentation.log 
> . These information is very useful for oozie functional monitoring. But it is 
> always difficult to get the meaning from flat file. If we expose this 
> information on some graphing tool, We can get the lot of meaning out of it 
> and can take some actions based on it. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 34854: OOZIE-2251 : Expose instrumental matrices in Realtime Graphing tool

[Narayan Periwal]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34854/
---

(Updated June 15, 2015, 12:49 p.m.)


Review request for oozie.


Bugs: OOZIE-2251
https://issues.apache.org/jira/browse/OOZIE-2251


Repository: oozie-git


Description
---

We have been logging so many important matrices in oozie-instrumentation.log . 
These information is very useful for oozie functional monitoring. But it is 
always difficult to get the meaning from flat file. If we expose this 
information on some graphing tool, We can get the lot of meaning out of it and 
can take some actions based on it.


Diffs (updated)
-

  core/pom.xml 783 
  
core/src/main/java/org/apache/oozie/service/MetricsInstrumentationService.java 
2a00a73 
  core/src/main/java/org/apache/oozie/util/Instrumentation.java 4eb6386 
  core/src/main/java/org/apache/oozie/util/MetricsInstrumentation.java 185b67e 
  core/src/main/resources/oozie-default.xml 4dc127e 
  docs/src/site/twiki/AG_Install.twiki 0ce2609 
  webapp/pom.xml e42e219 

Diff: https://reviews.apache.org/r/34854/diff/


Testing
---

Done


Thanks,

Narayan Periwal

[jira] [Commented] (OOZIE-2251) Expose instrumental matrices in Realtime Graphing tool

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14585848#comment-14585848
 ] 

Hadoop QA commented on OOZIE-2251:
--

Testing JIRA OOZIE-2251

Cleaning local git workspace



{color:red}-1{color} Patch failed to apply to head of branch



> Expose instrumental matrices in Realtime Graphing tool
> --
>
> Key: OOZIE-2251
> URL: https://issues.apache.org/jira/browse/OOZIE-2251
> Project: Oozie
>  Issue Type: New Feature
>  Components: monitoring
>Reporter: Jaydeep Vishwakarma
>Assignee: Narayan Periwal
> Attachments: OOZIE-2251-v0.patch, OOZIE-2251-v1.patch, 
> OOZIE-2251-v2.patch, OOZIE-2251-v3.patch, OOZIE-2251-v4.patch, 
> OOZIE-2251-v5.patch, OOZIE-2251-v6.patch, OOZIE-2251-v7.patch, 
> OOZIE-2251-v8.patch
>
>
> We have been logging so many important matrices in oozie-instrumentation.log 
> . These information is very useful for oozie functional monitoring. But it is 
> always difficult to get the meaning from flat file. If we expose this 
> information on some graphing tool, We can get the lot of meaning out of it 
> and can take some actions based on it. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Build failed in Jenkins: oozie-trunk-precommit-build #2413

[Apache Jenkins Server]

See 

--
[...truncated 11271 lines...]
[INFO] Compiling 6 source files to 

[WARNING] 
:[165,20]
 [deprecation] getConf() in org.apache.oozie.service.Services has been 
deprecated
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ oozie-tools ---
[INFO] Tests are skipped.
[INFO] 
[INFO] 
[INFO] Building Apache Oozie MiniOozie 4.3.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ oozie-mini ---
[INFO] Deleting 

[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ oozie-mini ---
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
oozie-mini ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ oozie-mini 
---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
oozie-mini ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 3 resources
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
oozie-mini ---
[INFO] Compiling 1 source file to 

[INFO] 
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ oozie-mini ---
[INFO] Tests are skipped.
[INFO] 
[INFO] 
[INFO] Building Apache Oozie WorkflowGenerator 4.3.0-SNAPSHOT
[INFO] 
Downloading: 
http://gwt-links.googlecode.com/svn/repository/com/orange/links/gwt-links/1.3-SNAPSHOT/maven-metadata.xml
Downloading: 
http://repository.apache.org/snapshots/com/orange/links/gwt-links/1.3-SNAPSHOT/maven-metadata.xml
Downloading: 
https://repository.apache.org/content/groups/snapshots/com/orange/links/gwt-links/1.3-SNAPSHOT/maven-metadata.xml
 
 
 
[WARNING] Could not transfer metadata 
com.orange.links:gwt-links:1.3-SNAPSHOT/maven-metadata.xml from/to 
apache.snapshots.repo (https://repository.apache.org/content/groups/snapshots): 
java.lang.RuntimeException: Could not generate DH keypair
[WARNING] Failure to transfer 
com.orange.links:gwt-links:1.3-SNAPSHOT/maven-metadata.xml from 
https://repository.apache.org/content/groups/snapshots was cached in the local 
repository, resolution will not be reattempted until the update interval of 
apache.snapshots.repo has elapsed or updates are forced. Original error: Could 
not transfer metadata 
com.orange.links:gwt-links:1.3-SNAPSHOT/maven-metadata.xml from/to 
apache.snapshots.repo (https://repository.apache.org/content/groups/snapshots): 
java.lang.RuntimeException: Could not generate DH keypair
Downloading: 
https://repository.apache.org/content/groups/snapshots/com/orange/links/gwt-links-parent/1.3-SNAPSHOT/maven-metadata.xml
Downloading: 
http://gwt-links.googlecode.com/svn/repository/com/orange/links/gwt-links-parent/1.3-SNAPSHOT/maven-metadata.xml
Downloading: 
http://repository.apache.org/snapshots/com/orange/links/gwt-links-parent/1.3-SNAPSHOT/maven-metadata.xml
 
 
605/605 B

Downloaded: 
http://gwt-links.googlecode.com/svn/repository/com/orange/links/gwt-links-parent/1.3-SNAPSHOT/maven-metadata.xml
 (605 B at 1.4 KB/sec)
[WARNING] Could not transfer metadata 
com.orange.links:gwt-links-parent:1.3-SNAPSHOT/maven-metadata.xml from/to 
apache.snapshots.repo (https://repository.apache.org/content/groups/snapshots): 
java.lang.RuntimeException: Could not generate DH keypair
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ 
oozie-workflowgenerator ---
[INFO] Deleting 

[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ 
oozie-workflowgenerator ---
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
oozie-workflowgenerator ---
[debug] execute con

[jira] [Updated] (OOZIE-2251) Expose instrumental matrices in Realtime Graphing tool

[Narayan Periwal (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Narayan Periwal updated OOZIE-2251:
---
Attachment: (was: OOZIE-2251-v8.patch)

> Expose instrumental matrices in Realtime Graphing tool
> --
>
> Key: OOZIE-2251
> URL: https://issues.apache.org/jira/browse/OOZIE-2251
> Project: Oozie
>  Issue Type: New Feature
>  Components: monitoring
>Reporter: Jaydeep Vishwakarma
>Assignee: Narayan Periwal
> Attachments: OOZIE-2251-v0.patch, OOZIE-2251-v1.patch, 
> OOZIE-2251-v2.patch, OOZIE-2251-v3.patch, OOZIE-2251-v4.patch, 
> OOZIE-2251-v5.patch, OOZIE-2251-v6.patch, OOZIE-2251-v7.patch, 
> OOZIE-2251-v8.patch
>
>
> We have been logging so many important matrices in oozie-instrumentation.log 
> . These information is very useful for oozie functional monitoring. But it is 
> always difficult to get the meaning from flat file. If we expose this 
> information on some graphing tool, We can get the lot of meaning out of it 
> and can take some actions based on it. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2251) Expose instrumental matrices in Realtime Graphing tool

[Narayan Periwal (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Narayan Periwal updated OOZIE-2251:
---
Attachment: OOZIE-2251-v8.patch

> Expose instrumental matrices in Realtime Graphing tool
> --
>
> Key: OOZIE-2251
> URL: https://issues.apache.org/jira/browse/OOZIE-2251
> Project: Oozie
>  Issue Type: New Feature
>  Components: monitoring
>Reporter: Jaydeep Vishwakarma
>Assignee: Narayan Periwal
> Attachments: OOZIE-2251-v0.patch, OOZIE-2251-v1.patch, 
> OOZIE-2251-v2.patch, OOZIE-2251-v3.patch, OOZIE-2251-v4.patch, 
> OOZIE-2251-v5.patch, OOZIE-2251-v6.patch, OOZIE-2251-v7.patch, 
> OOZIE-2251-v8.patch
>
>
> We have been logging so many important matrices in oozie-instrumentation.log 
> . These information is very useful for oozie functional monitoring. But it is 
> always difficult to get the meaning from flat file. If we expose this 
> information on some graphing tool, We can get the lot of meaning out of it 
> and can take some actions based on it. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 34854: OOZIE-2251 : Expose instrumental matrices in Realtime Graphing tool

[Narayan Periwal]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34854/
---

(Updated June 15, 2015, 12:21 p.m.)


Review request for oozie.


Bugs: OOZIE-2251
https://issues.apache.org/jira/browse/OOZIE-2251


Repository: oozie-git


Description
---

We have been logging so many important matrices in oozie-instrumentation.log . 
These information is very useful for oozie functional monitoring. But it is 
always difficult to get the meaning from flat file. If we expose this 
information on some graphing tool, We can get the lot of meaning out of it and 
can take some actions based on it.


Diffs (updated)
-

  core/pom.xml 783 
  
core/src/main/java/org/apache/oozie/service/MetricsInstrumentationService.java 
29f6294 
  core/src/main/java/org/apache/oozie/util/Instrumentation.java 3dfb67a 
  core/src/main/java/org/apache/oozie/util/MetricsInstrumentation.java e56bfda 
  core/src/main/resources/oozie-default.xml 8960073 
  docs/src/site/twiki/AG_Install.twiki 0ce2609 
  webapp/pom.xml e42e219 

Diff: https://reviews.apache.org/r/34854/diff/


Testing
---

Done


Thanks,

Narayan Periwal

[jira] [Updated] (OOZIE-2251) Expose instrumental matrices in Realtime Graphing tool

[Narayan Periwal (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Narayan Periwal updated OOZIE-2251:
---
Attachment: OOZIE-2251-v8.patch

> Expose instrumental matrices in Realtime Graphing tool
> --
>
> Key: OOZIE-2251
> URL: https://issues.apache.org/jira/browse/OOZIE-2251
> Project: Oozie
>  Issue Type: New Feature
>  Components: monitoring
>Reporter: Jaydeep Vishwakarma
>Assignee: Narayan Periwal
> Attachments: OOZIE-2251-v0.patch, OOZIE-2251-v1.patch, 
> OOZIE-2251-v2.patch, OOZIE-2251-v3.patch, OOZIE-2251-v4.patch, 
> OOZIE-2251-v5.patch, OOZIE-2251-v6.patch, OOZIE-2251-v7.patch, 
> OOZIE-2251-v8.patch
>
>
> We have been logging so many important matrices in oozie-instrumentation.log 
> . These information is very useful for oozie functional monitoring. But it is 
> always difficult to get the meaning from flat file. If we expose this 
> information on some graphing tool, We can get the lot of meaning out of it 
> and can take some actions based on it. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2251) Expose instrumental matrices in Realtime Graphing tool

[Narayan Periwal (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14585831#comment-14585831
 ] 

Narayan Periwal commented on OOZIE-2251:


[~shwethags], have made the changes as per review. Please have a look

> Expose instrumental matrices in Realtime Graphing tool
> --
>
> Key: OOZIE-2251
> URL: https://issues.apache.org/jira/browse/OOZIE-2251
> Project: Oozie
>  Issue Type: New Feature
>  Components: monitoring
>Reporter: Jaydeep Vishwakarma
>Assignee: Narayan Periwal
> Attachments: OOZIE-2251-v0.patch, OOZIE-2251-v1.patch, 
> OOZIE-2251-v2.patch, OOZIE-2251-v3.patch, OOZIE-2251-v4.patch, 
> OOZIE-2251-v5.patch, OOZIE-2251-v6.patch, OOZIE-2251-v7.patch
>
>
> We have been logging so many important matrices in oozie-instrumentation.log 
> . These information is very useful for oozie functional monitoring. But it is 
> always difficult to get the meaning from flat file. If we expose this 
> information on some graphing tool, We can get the lot of meaning out of it 
> and can take some actions based on it. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 34854: OOZIE-2251 : Expose instrumental matrices in Realtime Graphing tool

[Narayan Periwal]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34854/
---

(Updated June 15, 2015, 12:07 p.m.)


Review request for oozie.


Bugs: OOZIE-2251
https://issues.apache.org/jira/browse/OOZIE-2251


Repository: oozie-git


Description
---

We have been logging so many important matrices in oozie-instrumentation.log . 
These information is very useful for oozie functional monitoring. But it is 
always difficult to get the meaning from flat file. If we expose this 
information on some graphing tool, We can get the lot of meaning out of it and 
can take some actions based on it.


Diffs (updated)
-

  core/pom.xml 783 
  
core/src/main/java/org/apache/oozie/service/MetricsInstrumentationService.java 
29f6294 
  core/src/main/java/org/apache/oozie/util/Instrumentation.java 3dfb67a 
  core/src/main/java/org/apache/oozie/util/MetricsInstrumentation.java e56bfda 
  core/src/main/resources/oozie-default.xml 8960073 
  docs/src/site/twiki/AG_Install.twiki 0ce2609 
  webapp/pom.xml e42e219 

Diff: https://reviews.apache.org/r/34854/diff/


Testing
---

Done


Thanks,

Narayan Periwal

[jira] [Commented] (OOZIE-2253) Spark Job is failing when it is running in standalone server

[pavan kumar kolamuri (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14585811#comment-14585811
 ] 

pavan kumar kolamuri commented on OOZIE-2253:
-

This patch will fix the issues of spark job running on yarn . For standalone 
mode spark rest url should be used as master url to avoid the above exceptions. 

[~rkanter]  [~shwethags]   please review the patch . 

> Spark Job is failing when it is running in standalone server
> 
>
> Key: OOZIE-2253
> URL: https://issues.apache.org/jira/browse/OOZIE-2253
> Project: Oozie
>  Issue Type: Bug
>Reporter: pavan kumar kolamuri
>Assignee: pavan kumar kolamuri
> Attachments: OOZIE-2253.patch
>
>
> When Spark Job is running in spark standalone cluster the job is getting 
> launched and succedded and infinite jobs are getting launched in spark 
> cluster. Oozie workflow will be in running state forever as spark is 
> launching job infinite times. 
> This might be because in spark when job succeeds and it always do 
> System.exit(0) . In LauncherSecurityManager  exception is thrown for this. It 
> looks like spark(through akka framework)  is catching that and launching one 
> more attempt for the same job. It is happening infinitely .
> {noformat}
> Sending launch command to spark://inmobi-Precision-T3610:7077
> Driver successfully submitted as driver-20150526105806-
> ... waiting before polling master for driver state
> ... polling master for driver state
> State of driver-20150526105806- is SUBMITTED
> Sending launch command to spark://inmobi-Precision-T3610:7077
> Driver successfully submitted as driver-20150526105811-0001
> ... waiting before polling master for driver state
> ... polling master for driver state
> State of driver-20150526105811-0001 is SUBMITTED
> Sending launch command to spark://inmobi-Precision-T3610:7077
> Driver successfully submitted as driver-20150526105816-0002
> ... waiting before polling master for driver state
> ... polling master for driver state
> State of driver-20150526105816-0002 is SUBMITTED
> Sending launch command to spark://inmobi-Precision-T3610:7077
> Driver successfully submitted as driver-20150526105821-0003
> ... waiting before polling master for driver state
> ... polling master for driver state
> State of driver-20150526105821-0003 is SUBMITTED
> Sending launch command to spark://inmobi-Precision-T3610:7077
> Driver successfully submitted as driver-20150526105826-0004
> ... waiting before polling master for driver state
> {noformat}
> {noformat}
> 2015-05-26 10:58:11,573 ERROR [driverClient-akka.actor.default-dispatcher-4] 
> akka.actor.OneForOneStrategy: Intercepted System.exit(0)
> java.lang.SecurityException: Intercepted System.exit(0)
>   at 
> org.apache.oozie.action.hadoop.LauncherSecurityManager.checkExit(LauncherMapper.java:601)
>   at java.lang.Runtime.exit(Runtime.java:107)
>   at java.lang.System.exit(System.java:962)
>   at 
> org.apache.spark.deploy.ClientActor.pollAndReportStatus(Client.scala:115)
>   at 
> org.apache.spark.deploy.ClientActor$$anonfun$receiveWithLogging$1.applyOrElse(Client.scala:123)
>   at 
> scala.runtime.AbstractPartialFunction$mcVL$sp.apply$mcVL$sp(AbstractPartialFunction.scala:33)
>   at 
> scala.runtime.AbstractPartialFunction$mcVL$sp.apply(AbstractPartialFunction.scala:33)
>   at 
> scala.runtime.AbstractPartialFunction$mcVL$sp.apply(AbstractPartialFunction.scala:25)
>   at 
> org.apache.spark.util.ActorLogReceive$$anon$1.apply(ActorLogReceive.scala:53)
>   at 
> org.apache.spark.util.ActorLogReceive$$anon$1.apply(ActorLogReceive.scala:42)
>   at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:118)
>   at 
> org.apache.spark.util.ActorLogReceive$$anon$1.applyOrElse(ActorLogReceive.scala:42)
>   at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)
>   at akka.actor.ActorCell.invoke(ActorCell.scala:456)
>   at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
>   at akka.dispatch.Mailbox.run(Mailbox.scala:219)
>   at 
> akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)
>   at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
>   at 
> scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
>   at 
> scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
>   at 
> scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2253) Spark Job is failing when it is running in standalone server

[pavan kumar kolamuri (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

pavan kumar kolamuri updated OOZIE-2253:

Attachment: OOZIE-2253.patch

> Spark Job is failing when it is running in standalone server
> 
>
> Key: OOZIE-2253
> URL: https://issues.apache.org/jira/browse/OOZIE-2253
> Project: Oozie
>  Issue Type: Bug
>Reporter: pavan kumar kolamuri
>Assignee: pavan kumar kolamuri
> Attachments: OOZIE-2253.patch
>
>
> When Spark Job is running in spark standalone cluster the job is getting 
> launched and succedded and infinite jobs are getting launched in spark 
> cluster. Oozie workflow will be in running state forever as spark is 
> launching job infinite times. 
> This might be because in spark when job succeeds and it always do 
> System.exit(0) . In LauncherSecurityManager  exception is thrown for this. It 
> looks like spark(through akka framework)  is catching that and launching one 
> more attempt for the same job. It is happening infinitely .
> {noformat}
> Sending launch command to spark://inmobi-Precision-T3610:7077
> Driver successfully submitted as driver-20150526105806-
> ... waiting before polling master for driver state
> ... polling master for driver state
> State of driver-20150526105806- is SUBMITTED
> Sending launch command to spark://inmobi-Precision-T3610:7077
> Driver successfully submitted as driver-20150526105811-0001
> ... waiting before polling master for driver state
> ... polling master for driver state
> State of driver-20150526105811-0001 is SUBMITTED
> Sending launch command to spark://inmobi-Precision-T3610:7077
> Driver successfully submitted as driver-20150526105816-0002
> ... waiting before polling master for driver state
> ... polling master for driver state
> State of driver-20150526105816-0002 is SUBMITTED
> Sending launch command to spark://inmobi-Precision-T3610:7077
> Driver successfully submitted as driver-20150526105821-0003
> ... waiting before polling master for driver state
> ... polling master for driver state
> State of driver-20150526105821-0003 is SUBMITTED
> Sending launch command to spark://inmobi-Precision-T3610:7077
> Driver successfully submitted as driver-20150526105826-0004
> ... waiting before polling master for driver state
> {noformat}
> {noformat}
> 2015-05-26 10:58:11,573 ERROR [driverClient-akka.actor.default-dispatcher-4] 
> akka.actor.OneForOneStrategy: Intercepted System.exit(0)
> java.lang.SecurityException: Intercepted System.exit(0)
>   at 
> org.apache.oozie.action.hadoop.LauncherSecurityManager.checkExit(LauncherMapper.java:601)
>   at java.lang.Runtime.exit(Runtime.java:107)
>   at java.lang.System.exit(System.java:962)
>   at 
> org.apache.spark.deploy.ClientActor.pollAndReportStatus(Client.scala:115)
>   at 
> org.apache.spark.deploy.ClientActor$$anonfun$receiveWithLogging$1.applyOrElse(Client.scala:123)
>   at 
> scala.runtime.AbstractPartialFunction$mcVL$sp.apply$mcVL$sp(AbstractPartialFunction.scala:33)
>   at 
> scala.runtime.AbstractPartialFunction$mcVL$sp.apply(AbstractPartialFunction.scala:33)
>   at 
> scala.runtime.AbstractPartialFunction$mcVL$sp.apply(AbstractPartialFunction.scala:25)
>   at 
> org.apache.spark.util.ActorLogReceive$$anon$1.apply(ActorLogReceive.scala:53)
>   at 
> org.apache.spark.util.ActorLogReceive$$anon$1.apply(ActorLogReceive.scala:42)
>   at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:118)
>   at 
> org.apache.spark.util.ActorLogReceive$$anon$1.applyOrElse(ActorLogReceive.scala:42)
>   at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)
>   at akka.actor.ActorCell.invoke(ActorCell.scala:456)
>   at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
>   at akka.dispatch.Mailbox.run(Mailbox.scala:219)
>   at 
> akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)
>   at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
>   at 
> scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
>   at 
> scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
>   at 
> scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 34854: OOZIE-2251 : Expose instrumental matrices in Realtime Graphing tool

[Narayan Periwal]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34854/
---

(Updated June 15, 2015, 11:05 a.m.)


Review request for oozie.


Bugs: OOZIE-2251
https://issues.apache.org/jira/browse/OOZIE-2251


Repository: oozie-git


Description
---

We have been logging so many important matrices in oozie-instrumentation.log . 
These information is very useful for oozie functional monitoring. But it is 
always difficult to get the meaning from flat file. If we expose this 
information on some graphing tool, We can get the lot of meaning out of it and 
can take some actions based on it.


Diffs (updated)
-

  core/pom.xml 783 
  
core/src/main/java/org/apache/oozie/service/MetricsInstrumentationService.java 
29f6294 
  core/src/main/java/org/apache/oozie/util/Instrumentation.java 3dfb67a 
  core/src/main/java/org/apache/oozie/util/MetricsInstrumentation.java e56bfda 
  core/src/main/resources/oozie-default.xml 8960073 
  docs/src/site/twiki/AG_Install.twiki 0ce2609 

Diff: https://reviews.apache.org/r/34854/diff/


Testing
---

Done


Thanks,

Narayan Periwal

[jira] [Commented] (OOZIE-2030) Configuration properties from global section is not getting set in Hadoop job conf when using sub-workflow action in Oozie workflow.xml

[Jaydeep Vishwakarma (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14585684#comment-14585684
 ] 

Jaydeep Vishwakarma commented on OOZIE-2030:


Yes, [~sriksun] solution looks more clean.

> Configuration properties from global section is not getting set in Hadoop job 
> conf when using sub-workflow action in Oozie workflow.xml 
> 
>
> Key: OOZIE-2030
> URL: https://issues.apache.org/jira/browse/OOZIE-2030
> Project: Oozie
>  Issue Type: Bug
>  Components: action
>Reporter: Peeyush Bishnoi
>Assignee: Jaydeep Vishwakarma
> Attachments: OOZIE-2030-v2.patch, OOZIE-2030-v3.patch, 
> OOZIE-2030-v4.patch, OOZIE-2030.patch
>
>
> When submitting Oozie workflow with sub-workflow action and with global 
> section, configuration properties defined in global section is not getting 
> set in launched Hadoop job conf. But when we use Pig or MR action in 
> workflow.xml, configuration properties from global section set properly into 
> Hadoop job conf.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2020) Rerun all Failed/killed/timedout coordinator actions rather than specifying action numbers

[Shwetha G S (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14585682#comment-14585682
 ] 

Shwetha G S commented on OOZIE-2020:


[~rkanter], [~puru], [~bowenzhangusa], does anyone use the oozie REST APIs 
directly? Instead of keep adding more arguments to re-run command, I was 
thinking we can merge -actions, -date, -status into one option -filter. We can 
still expose these in CLI and OozieClient for backward compatibility. But 
internally, all these can map to just one argument 'filter' at the client side 
and pass just this to the server. Or we can merge this in server as well, if 
some users use REST APIs directly.

> Rerun all Failed/killed/timedout coordinator actions rather than specifying 
> action numbers
> --
>
> Key: OOZIE-2020
> URL: https://issues.apache.org/jira/browse/OOZIE-2020
> Project: Oozie
>  Issue Type: New Feature
>  Components: action
>Reporter: Sreedish P S
>Assignee: Narayan Periwal
>Priority: Minor
> Attachments: OOZIE-2020-v10.patch, OOZIE-2020-v11.patch, 
> OOZIE-2020-v12.patch, OOZIE-2020-v13.patch, OOZIE-2020-v14.patch, 
> OOZIE-2020-v15.patch, OOZIE-2020-v16.patch, OOZIE-2020-v17.patch, 
> OOZIE-2020-v8.patch, OOZIE-2020-v9.patch
>
>
> Currently rerun of coordinator actions are made through coordinator id and 
> action numbers, this feature request is for rerunning all coordinator actions 
> by mentioning a particular state
> for example :
> oozie job -rerun coord-id -state killed



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2030) Configuration properties from global section is not getting set in Hadoop job conf when using sub-workflow action in Oozie workflow.xml

[Srikanth Sundarrajan (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14585679#comment-14585679
 ] 

Srikanth Sundarrajan commented on OOZIE-2030:
-

Didn't realize that [~shwethags] has already proposed the same approach. Sorry

> Configuration properties from global section is not getting set in Hadoop job 
> conf when using sub-workflow action in Oozie workflow.xml 
> 
>
> Key: OOZIE-2030
> URL: https://issues.apache.org/jira/browse/OOZIE-2030
> Project: Oozie
>  Issue Type: Bug
>  Components: action
>Reporter: Peeyush Bishnoi
>Assignee: Jaydeep Vishwakarma
> Attachments: OOZIE-2030-v2.patch, OOZIE-2030-v3.patch, 
> OOZIE-2030-v4.patch, OOZIE-2030.patch
>
>
> When submitting Oozie workflow with sub-workflow action and with global 
> section, configuration properties defined in global section is not getting 
> set in launched Hadoop job conf. But when we use Pig or MR action in 
> workflow.xml, configuration properties from global section set properly into 
> Hadoop job conf.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2030) Configuration properties from global section is not getting set in Hadoop job conf when using sub-workflow action in Oozie workflow.xml

[Srikanth Sundarrajan (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14585671#comment-14585671
 ] 

Srikanth Sundarrajan commented on OOZIE-2030:
-

There is perhaps a simpler way to tackle this issue. If we modify the 
LiteWorkflowAppParser to serialize persist the contents of global in conf and 
have handleGlobal() also consult conf for the handling section, this will 
ensure that global is propagated correctly with no further changes to any other 
section of the code, honoring the right overlay priorities. While the code in 
LWAP wouldn't be specific to Subworkflows either. Luckily the conf itself is 
propagates into subflows on user's request. - [~shwethags], [~rohini], makes 
sense ?

> Configuration properties from global section is not getting set in Hadoop job 
> conf when using sub-workflow action in Oozie workflow.xml 
> 
>
> Key: OOZIE-2030
> URL: https://issues.apache.org/jira/browse/OOZIE-2030
> Project: Oozie
>  Issue Type: Bug
>  Components: action
>Reporter: Peeyush Bishnoi
>Assignee: Jaydeep Vishwakarma
> Attachments: OOZIE-2030-v2.patch, OOZIE-2030-v3.patch, 
> OOZIE-2030-v4.patch, OOZIE-2030.patch
>
>
> When submitting Oozie workflow with sub-workflow action and with global 
> section, configuration properties defined in global section is not getting 
> set in launched Hadoop job conf. But when we use Pig or MR action in 
> workflow.xml, configuration properties from global section set properly into 
> Hadoop job conf.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2030) Configuration properties from global section is not getting set in Hadoop job conf when using sub-workflow action in Oozie workflow.xml

[Shwetha G S (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14585630#comment-14585630
 ] 

Shwetha G S commented on OOZIE-2030:


[~rohini],
SubWorkflowActionExecutor launches new workflow with app path = suborkflow app 
path. Sub workflow will read the global conf from this subworkflow app path. 
Merging the global sections requires modifying the workflow.xml from this 
subworkflow app path which we don't want to touch, right? So, only other way is 
to set the global conf as a conf while launching subworkflow and workflow 
parser of sub workflow will do the merge. Makes sense? So, this requires 
changes in LiteWorkflowAppParser

> Configuration properties from global section is not getting set in Hadoop job 
> conf when using sub-workflow action in Oozie workflow.xml 
> 
>
> Key: OOZIE-2030
> URL: https://issues.apache.org/jira/browse/OOZIE-2030
> Project: Oozie
>  Issue Type: Bug
>  Components: action
>Reporter: Peeyush Bishnoi
>Assignee: Jaydeep Vishwakarma
> Attachments: OOZIE-2030-v2.patch, OOZIE-2030-v3.patch, 
> OOZIE-2030-v4.patch, OOZIE-2030.patch
>
>
> When submitting Oozie workflow with sub-workflow action and with global 
> section, configuration properties defined in global section is not getting 
> set in launched Hadoop job conf. But when we use Pig or MR action in 
> workflow.xml, configuration properties from global section set properly into 
> Hadoop job conf.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2251) Expose instrumental matrices in Realtime Graphing tool

[Shwetha G S (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14585606#comment-14585606
 ] 

Shwetha G S commented on OOZIE-2251:


You need gmetric4j at compile time, but don't include it in the war

> Expose instrumental matrices in Realtime Graphing tool
> --
>
> Key: OOZIE-2251
> URL: https://issues.apache.org/jira/browse/OOZIE-2251
> Project: Oozie
>  Issue Type: New Feature
>  Components: monitoring
>Reporter: Jaydeep Vishwakarma
>Assignee: Narayan Periwal
> Attachments: OOZIE-2251-v0.patch, OOZIE-2251-v1.patch, 
> OOZIE-2251-v2.patch, OOZIE-2251-v3.patch, OOZIE-2251-v4.patch, 
> OOZIE-2251-v5.patch, OOZIE-2251-v6.patch, OOZIE-2251-v7.patch
>
>
> We have been logging so many important matrices in oozie-instrumentation.log 
> . These information is very useful for oozie functional monitoring. But it is 
> always difficult to get the meaning from flat file. If we expose this 
> information on some graphing tool, We can get the lot of meaning out of it 
> and can take some actions based on it. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2251) Expose instrumental matrices in Realtime Graphing tool

[Narayan Periwal (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14585544#comment-14585544
 ] 

Narayan Periwal commented on OOZIE-2251:


[~shwethags], you said don't include info.ganglia.gmetric4j:gmetric4j as well. 
But I need this to initialise the ganglia server.

> Expose instrumental matrices in Realtime Graphing tool
> --
>
> Key: OOZIE-2251
> URL: https://issues.apache.org/jira/browse/OOZIE-2251
> Project: Oozie
>  Issue Type: New Feature
>  Components: monitoring
>Reporter: Jaydeep Vishwakarma
>Assignee: Narayan Periwal
> Attachments: OOZIE-2251-v0.patch, OOZIE-2251-v1.patch, 
> OOZIE-2251-v2.patch, OOZIE-2251-v3.patch, OOZIE-2251-v4.patch, 
> OOZIE-2251-v5.patch, OOZIE-2251-v6.patch, OOZIE-2251-v7.patch
>
>
> We have been logging so many important matrices in oozie-instrumentation.log 
> . These information is very useful for oozie functional monitoring. But it is 
> always difficult to get the meaning from flat file. If we expose this 
> information on some graphing tool, We can get the lot of meaning out of it 
> and can take some actions based on it. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 34854: OOZIE-2251 : Expose instrumental matrices in Realtime Graphing tool

[Narayan Periwal]

> On June 15, 2015, 6:25 a.m., Shwetha GS wrote:
> > core/src/main/java/org/apache/oozie/util/MetricsInstrumentation.java, line 
> > 98
> > 
> >
> > prefix all related configs with 'oozie.external_monitoring.'. For 
> > example, change oozie.metrics.report.interval.sec - 
> > oozie.external_monitoring.reporterIntervalSecs
> > oozie.metrics.host - oozie.external_monitoring.address (covers both 
> > host and port so that there are less configs that users need to set)
> > oozie.metrics.server.name - oozie.external_monitoring.type
> > oozie.metrics.prefix - oozie.external_monitoring.metricPrefix
> > 
> > Use . for hierarchical configs, not as word separator
> 
> Narayan Periwal wrote:
> When we initialize the graphite object, there we need to pass the host 
> and the port separately to make the InetSocketAddress object. So, to avoid 
> the parsing, I am keeping the host and the port as separate configs. Also, 
> now the user cannot fail to give the port number.
> 
> Narayan Periwal wrote:
> Please let me know if you have some other suggestion
> 
> Shwetha GS wrote:
> For the function, you can use URL and get host and port. We should 
> minimise the number of configs that users have to set. 
> 
> now the user cannot fail to give the port number - why can't they skip 
> setting another config?
> 
> Both graphite and ganglia servers have different default ports than the 
> default that you have set.

Ok...I will use URL to get the host and port. Graphite server have the port 
2020 which is the same that I have set in the default


- Narayan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34854/#review87894
---


On June 12, 2015, 7:53 p.m., Narayan Periwal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34854/
> ---
> 
> (Updated June 12, 2015, 7:53 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-2251
> https://issues.apache.org/jira/browse/OOZIE-2251
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> We have been logging so many important matrices in oozie-instrumentation.log 
> . These information is very useful for oozie functional monitoring. But it is 
> always difficult to get the meaning from flat file. If we expose this 
> information on some graphing tool, We can get the lot of meaning out of it 
> and can take some actions based on it.
> 
> 
> Diffs
> -
> 
>   core/pom.xml 783 
>   
> core/src/main/java/org/apache/oozie/service/MetricsInstrumentationService.java
>  29f6294 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 3dfb67a 
>   core/src/main/java/org/apache/oozie/util/MetricsInstrumentation.java 
> e56bfda 
>   core/src/main/resources/oozie-default.xml 8960073 
>   docs/src/site/twiki/AG_Install.twiki 0ce2609 
> 
> Diff: https://reviews.apache.org/r/34854/diff/
> 
> 
> Testing
> ---
> 
> Done
> 
> 
> Thanks,
> 
> Narayan Periwal
> 
>

Re: Review Request 34854: OOZIE-2251 : Expose instrumental matrices in Realtime Graphing tool

[Shwetha GS]

> On June 15, 2015, 6:25 a.m., Shwetha GS wrote:
> > core/src/main/java/org/apache/oozie/util/MetricsInstrumentation.java, line 
> > 98
> > 
> >
> > prefix all related configs with 'oozie.external_monitoring.'. For 
> > example, change oozie.metrics.report.interval.sec - 
> > oozie.external_monitoring.reporterIntervalSecs
> > oozie.metrics.host - oozie.external_monitoring.address (covers both 
> > host and port so that there are less configs that users need to set)
> > oozie.metrics.server.name - oozie.external_monitoring.type
> > oozie.metrics.prefix - oozie.external_monitoring.metricPrefix
> > 
> > Use . for hierarchical configs, not as word separator
> 
> Narayan Periwal wrote:
> When we initialize the graphite object, there we need to pass the host 
> and the port separately to make the InetSocketAddress object. So, to avoid 
> the parsing, I am keeping the host and the port as separate configs. Also, 
> now the user cannot fail to give the port number.
> 
> Narayan Periwal wrote:
> Please let me know if you have some other suggestion

For the function, you can use URL and get host and port. We should minimise the 
number of configs that users have to set. 

now the user cannot fail to give the port number - why can't they skip setting 
another config?

Both graphite and ganglia servers have different default ports than the default 
that you have set.


- Shwetha


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34854/#review87894
---


On June 12, 2015, 7:53 p.m., Narayan Periwal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34854/
> ---
> 
> (Updated June 12, 2015, 7:53 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-2251
> https://issues.apache.org/jira/browse/OOZIE-2251
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> We have been logging so many important matrices in oozie-instrumentation.log 
> . These information is very useful for oozie functional monitoring. But it is 
> always difficult to get the meaning from flat file. If we expose this 
> information on some graphing tool, We can get the lot of meaning out of it 
> and can take some actions based on it.
> 
> 
> Diffs
> -
> 
>   core/pom.xml 783 
>   
> core/src/main/java/org/apache/oozie/service/MetricsInstrumentationService.java
>  29f6294 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 3dfb67a 
>   core/src/main/java/org/apache/oozie/util/MetricsInstrumentation.java 
> e56bfda 
>   core/src/main/resources/oozie-default.xml 8960073 
>   docs/src/site/twiki/AG_Install.twiki 0ce2609 
> 
> Diff: https://reviews.apache.org/r/34854/diff/
> 
> 
> Testing
> ---
> 
> Done
> 
> 
> Thanks,
> 
> Narayan Periwal
> 
>

Re: Review Request 34854: OOZIE-2251 : Expose instrumental matrices in Realtime Graphing tool

[Narayan Periwal]

> On June 15, 2015, 6:25 a.m., Shwetha GS wrote:
> > core/src/main/java/org/apache/oozie/util/MetricsInstrumentation.java, line 
> > 98
> > 
> >
> > prefix all related configs with 'oozie.external_monitoring.'. For 
> > example, change oozie.metrics.report.interval.sec - 
> > oozie.external_monitoring.reporterIntervalSecs
> > oozie.metrics.host - oozie.external_monitoring.address (covers both 
> > host and port so that there are less configs that users need to set)
> > oozie.metrics.server.name - oozie.external_monitoring.type
> > oozie.metrics.prefix - oozie.external_monitoring.metricPrefix
> > 
> > Use . for hierarchical configs, not as word separator
> 
> Narayan Periwal wrote:
> When we initialize the graphite object, there we need to pass the host 
> and the port separately to make the InetSocketAddress object. So, to avoid 
> the parsing, I am keeping the host and the port as separate configs. Also, 
> now the user cannot fail to give the port number.

Please let me know if you have some other suggestion


- Narayan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34854/#review87894
---


On June 12, 2015, 7:53 p.m., Narayan Periwal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34854/
> ---
> 
> (Updated June 12, 2015, 7:53 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-2251
> https://issues.apache.org/jira/browse/OOZIE-2251
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> We have been logging so many important matrices in oozie-instrumentation.log 
> . These information is very useful for oozie functional monitoring. But it is 
> always difficult to get the meaning from flat file. If we expose this 
> information on some graphing tool, We can get the lot of meaning out of it 
> and can take some actions based on it.
> 
> 
> Diffs
> -
> 
>   core/pom.xml 783 
>   
> core/src/main/java/org/apache/oozie/service/MetricsInstrumentationService.java
>  29f6294 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 3dfb67a 
>   core/src/main/java/org/apache/oozie/util/MetricsInstrumentation.java 
> e56bfda 
>   core/src/main/resources/oozie-default.xml 8960073 
>   docs/src/site/twiki/AG_Install.twiki 0ce2609 
> 
> Diff: https://reviews.apache.org/r/34854/diff/
> 
> 
> Testing
> ---
> 
> Done
> 
> 
> Thanks,
> 
> Narayan Periwal
> 
>