date:20150605

[jira] Subscription: Oozie Patch Available

2015-06-05 Thread jira

Issue Subscription
Filter: Oozie Patch Available (53 issues)

Subscriber: ooziedaily

Key Summary
OOZIE-2262  Fix log streaming from other server with start/end filter
https://issues.apache.org/jira/browse/OOZIE-2262
OOZIE-2258  Introducing a new counter in the instrumentation log to distinguish 
between the reasons for launcher failure
https://issues.apache.org/jira/browse/OOZIE-2258
OOZIE-2257  BundleStatusTransitXCommand doesn't update last modified time.
https://issues.apache.org/jira/browse/OOZIE-2257
OOZIE-2251  Expose instrumental matrices in Realtime Graphing tool
https://issues.apache.org/jira/browse/OOZIE-2251
OOZIE-2250  Show log for WAITING and TIMEDOUT coord actions
https://issues.apache.org/jira/browse/OOZIE-2250
OOZIE-2245  Service to periodically check database schema
https://issues.apache.org/jira/browse/OOZIE-2245
OOZIE-2244  Oozie should mask passwords in the logs when logging command 
arguments
https://issues.apache.org/jira/browse/OOZIE-2244
OOZIE-2243  Kill Command does not kill the child job for java action
https://issues.apache.org/jira/browse/OOZIE-2243
OOZIE-2203  Fix the login example
https://issues.apache.org/jira/browse/OOZIE-2203
OOZIE-2196  Create Local Client for Bundle
https://issues.apache.org/jira/browse/OOZIE-2196
OOZIE-2187  Add a way to specify a default JT/RM and NN
https://issues.apache.org/jira/browse/OOZIE-2187
OOZIE-2178  fix javadoc to compile on JDK8
https://issues.apache.org/jira/browse/OOZIE-2178
OOZIE-2168  Oozie flow and action names have 50 char limit 
https://issues.apache.org/jira/browse/OOZIE-2168
OOZIE-2159  'oozie validate' command should be moved server-side
https://issues.apache.org/jira/browse/OOZIE-2159
OOZIE-2134  Remove references to Services.get().getConf() in code
https://issues.apache.org/jira/browse/OOZIE-2134
OOZIE-2106  Make tomcat download url configurable in the pom file
https://issues.apache.org/jira/browse/OOZIE-2106
OOZIE-2105  Make version of submodules configurable with parent version 
https://issues.apache.org/jira/browse/OOZIE-2105
OOZIE-2099  Add test-patch support for patches generated without --no-prefix
https://issues.apache.org/jira/browse/OOZIE-2099
OOZIE-2081  WorkflowJob notification to include coordinator action id 
https://issues.apache.org/jira/browse/OOZIE-2081
OOZIE-2060  Incorrect documentation of Java action config XML filename
https://issues.apache.org/jira/browse/OOZIE-2060
OOZIE-2044  ssh action succeed with a not exists command which should be fail.
https://issues.apache.org/jira/browse/OOZIE-2044
OOZIE-2020  Rerun all Failed/killed/timedout coordinator actions rather than 
specifying action numbers
https://issues.apache.org/jira/browse/OOZIE-2020
OOZIE-1980  Sql error should not fail coord job
https://issues.apache.org/jira/browse/OOZIE-1980
OOZIE-1977  Display patch analysis issues
https://issues.apache.org/jira/browse/OOZIE-1977
OOZIE-1936  Queuedump command should display queue information for all server.
https://issues.apache.org/jira/browse/OOZIE-1936
OOZIE-1931  Admin command to print all locks held by server(s)
https://issues.apache.org/jira/browse/OOZIE-1931
OOZIE-1927  Use StoreStatusFilter for WorkflowsJobGetJPAExecutor 
https://issues.apache.org/jira/browse/OOZIE-1927
OOZIE-1922  MemoryLocksService fails if lock is acquired multiple times in same 
thread and released
https://issues.apache.org/jira/browse/OOZIE-1922
OOZIE-1918  ActionXCommand refactoring for code reuse
https://issues.apache.org/jira/browse/OOZIE-1918
OOZIE-1884  ZKJobsConcurrencyService.checkJobIdForServer fails if jobs length > 
7
https://issues.apache.org/jira/browse/OOZIE-1884
OOZIE-1860  Oozie job mapper launch fails due to null value returned from 
action file
https://issues.apache.org/jira/browse/OOZIE-1860
OOZIE-1855  TestPriorityDelayQueue#testPoll failed intermittently in Jenkins
https://issues.apache.org/jira/browse/OOZIE-1855
OOZIE-1816  LogInfo uses action name instead of id
https://issues.apache.org/jira/browse/OOZIE-1816
OOZIE-1810  Workflow cannot get into Failed state when kill control node cannot 
resolve variable in message 
https://issues.apache.org/jira/browse/OOZIE-1810
OOZIE-1802  Support workflow action log
https://issues.apache.org/jira/browse/OOZIE-1802
OOZIE-1793  Improve find bugs reporting for Oozie
https://issues.apache.org/jira/browse/OOZIE-1793
OOZIE-1782  Workflow path not found is thrown as SC_UNAUTHORIZED
https://issues.apache.org/jira/browse/OOZIE-1782
OOZIE-1779  Add parent-id as a filter option
https://issues.apache.org/jira/browse/OOZIE-1779
O

Re: Review Request 33400: OOZIE-2215 Support glob in FS EL function

2015-06-05 Thread Ryota Egashira


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33400/#review86855
---


this behavior is observed in hadoop-1 only,  fixed in hadoop-2

- Ryota Egashira


On June 1, 2015, 5:49 p.m., Ryota Egashira wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33400/
> ---
> 
> (Updated June 1, 2015, 5:49 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/OOZIE-2215
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 
> f8af61c 
>   core/src/main/resources/oozie-default.xml 8960073 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsELFunctions.java 
> c81fc61 
>   docs/src/site/twiki/WorkflowFunctionalSpec.twiki 02dc65b 
> 
> Diff: https://reviews.apache.org/r/33400/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Ryota Egashira
> 
>

[jira] [Commented] (OOZIE-2195) Expose Pig read/write bytes details as counters in Launcher

2015-06-05 Thread Rohini Palaniswamy (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-2195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14574889#comment-14574889
 ] 

Rohini Palaniswamy commented on OOZIE-2195:
---

  That would be really good. 

 [~jaydeepvishwakarma],
Can you make it a generic aggregation framework to aggregate counters of 
child jobs for all types of actions into one counter group and do special 
aggregation or collection of input/output counters for pig/hive actions?  If 
you just want to work on the Pig special counters in this jira it is fine, but 
make it generic with empty implementation in JavaActionExecutor/LauncherMain 
(aggregation of all counters to be added later) and override in 
PigActionExecutor/PigMain to add the counters.

> Expose Pig read/write bytes details as counters in Launcher
> ---
>
> Key: OOZIE-2195
> URL: https://issues.apache.org/jira/browse/OOZIE-2195
> Project: Oozie
>  Issue Type: New Feature
>Reporter: Jaydeep Vishwakarma
>Assignee: Narayan Periwal
>
> Pig action spawns multiple mr jobs and all the aggregated read/write bytes 
> metrics are emitted in logs only. It would be really helpful if we expose 
> those information in Launcher as counters.  This should not be the default 
> behaviour. User can enable this by job level or global 
> configuration.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2251) Expose instrumental matrices in Realtime Graphing tool

2015-06-05 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-2251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14574842#comment-14574842
 ] 

Hadoop QA commented on OOZIE-2251:
--

Testing JIRA OOZIE-2251

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:red}-1{color} the patch does not add/modify any testcase
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
{color:red}-1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:red}-1{color} the patch seems to introduce 1 new javac warning(s)
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 1653
.Tests failed: 4
.Tests errors: 0

.The patch failed the following testcases:

.  
testNone(org.apache.oozie.command.coord.TestCoordActionInputCheckXCommandNonUTC)
.  
testPauseBundleAndCoordinator(org.apache.oozie.service.TestPauseTransitService)
.  testSamplers(org.apache.oozie.util.TestMetricsInstrumentation)
.  
testCoordStatus_Failed(org.apache.oozie.command.coord.TestCoordChangeXCommand)

{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}


The full output of the test-patch run is available at

.   https://builds.apache.org/job/oozie-trunk-precommit-build/2392/

> Expose instrumental matrices in Realtime Graphing tool
> --
>
> Key: OOZIE-2251
> URL: https://issues.apache.org/jira/browse/OOZIE-2251
> Project: Oozie
>  Issue Type: New Feature
>  Components: monitoring
>Reporter: Jaydeep Vishwakarma
>Assignee: Narayan Periwal
> Attachments: OOZIE-2251-v0.patch, OOZIE-2251-v1.patch, 
> OOZIE-2251-v2.patch, OOZIE-2251-v3.patch, OOZIE-2251-v4.patch
>
>
> We have been logging so many important matrices in oozie-instrumentation.log 
> . These information is very useful for oozie functional monitoring. But it is 
> always difficult to get the meaning from flat file. If we expose this 
> information on some graphing tool, We can get the lot of meaning out of it 
> and can take some actions based on it. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Build failed in Jenkins: oozie-trunk-precommit-build #2392

2015-06-05 Thread Apache Jenkins Server

See 

--
[...truncated 8519 lines...]
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @ oozie-tools ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:2.0-beta-6:attach-descriptor (attach-descriptor) @ 
oozie-tools ---
[INFO] 
[INFO] --- maven-assembly-plugin:2.2.1:single (default-cli) @ oozie-tools ---
[INFO] Reading assembly descriptor: ../src/main/assemblies/tools.xml
[WARNING] The following patterns were never triggered in this artifact 
exclusion filter:
o  '*:*:pom:*'

[INFO] Copying files to 

[WARNING] Assembly file: 

 is not a regular file (it may be a directory). It cannot be attached to the 
project build for installation or deployment.
[INFO] 
[INFO] 
[INFO] Building Apache Oozie MiniOozie 4.3.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ oozie-mini ---
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
oozie-mini ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ oozie-mini 
---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
oozie-mini ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 3 resources
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
oozie-mini ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ oozie-mini ---
[INFO] Tests are skipped.
[INFO] 
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @ oozie-mini ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:2.0-beta-6:attach-descriptor (attach-descriptor) @ 
oozie-mini ---
[INFO] 
[INFO] --- maven-assembly-plugin:2.2.1:single (default-cli) @ oozie-mini ---
[INFO] Reading assembly descriptor: src/main/assemblies/empty.xml
[INFO] 
[INFO] 
[INFO] Building Apache Oozie Distro 4.3.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ oozie-distro 
---
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
oozie-distro ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ oozie-distro 
---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
oozie-distro ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
oozie-distro ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ oozie-distro ---
[INFO] Tests are skipped.
[INFO] 
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @ oozie-distro ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:2.0-beta-6:attach-descriptor (attach-descriptor) @ 
oozie-distro ---
[INFO] 
[INFO] --- maven-antrun-plugin:1.6:run (default) @ oozie-distro ---
[INFO] Executing tasks

main:
  [get] Getting: 
http://archive.apache.org/dist/tomcat/tomcat-6/v6.0.43/bin/apache-tomcat-6.0.43.tar.gz
  [get] To:

Re: Review Request 34854: OOZIE-2251 : Expose instrumental matrices in Realtime Graphing tool

2015-06-05 Thread Narayan Periwal



> On June 4, 2015, 5:33 a.m., Srikanth Sundarrajan wrote:
> > core/src/main/java/org/apache/oozie/service/PopulatorMetricsReportingManager.java,
> >  line 55
> > 
> >
> > Perhaps this can be merged with start. There are duplicate checks for 
> > METRICS_SERVER_NAME in both start & init, and they can be avoided if 
> > merged. Also notice that in start(), value is checked against lower case 
> > and in init() the check is case sensitive.

I have merged the init method with the start method


> On June 4, 2015, 5:33 a.m., Srikanth Sundarrajan wrote:
> > core/src/main/java/org/apache/oozie/util/Instrumentation.java, line 56
> > 
> >
> > Should this be in Insturmentation or InstrumentationService.
> > 
> > I also dont see PopulatorMetricsReportingManager::stop() actually 
> > called unless manager's life cycle is wired up to a service
> 
> Narayan Periwal wrote:
> The life-time of the metrics reporter is the entire lifetime of the oozie 
> server. So, it will be stopped automatically when the oozie server stops. So, 
> we have no particular of stop method here.

Yes, PopulatorMetricsReportingManager::stop() is not called actually. So, I 
have removed it from the code


- Narayan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34854/#review86550
---


On June 4, 2015, 6:56 a.m., Narayan Periwal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34854/
> ---
> 
> (Updated June 4, 2015, 6:56 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-2251
> https://issues.apache.org/jira/browse/OOZIE-2251
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> We have been logging so many important matrices in oozie-instrumentation.log 
> . These information is very useful for oozie functional monitoring. But it is 
> always difficult to get the meaning from flat file. If we expose this 
> information on some graphing tool, We can get the lot of meaning out of it 
> and can take some actions based on it.
> 
> 
> Diffs
> -
> 
>   core/pom.xml 783 
>   core/src/main/java/org/apache/oozie/service/MetricsReporter.java 
> PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 3dfb67a 
>   core/src/main/java/org/apache/oozie/util/OozieMonitoringComponent.java 
> PRE-CREATION 
>   core/src/main/resources/oozie-default.xml 8960073 
> 
> Diff: https://reviews.apache.org/r/34854/diff/
> 
> 
> Testing
> ---
> 
> Done
> 
> 
> Thanks,
> 
> Narayan Periwal
> 
>

Re: Review Request 34854: OOZIE-2251 : Expose instrumental matrices in Realtime Graphing tool

2015-06-05 Thread Narayan Periwal


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34854/
---

(Updated June 5, 2015, 2:20 p.m.)


Review request for oozie.


Bugs: OOZIE-2251
https://issues.apache.org/jira/browse/OOZIE-2251


Repository: oozie-git


Description
---

We have been logging so many important matrices in oozie-instrumentation.log . 
These information is very useful for oozie functional monitoring. But it is 
always difficult to get the meaning from flat file. If we expose this 
information on some graphing tool, We can get the lot of meaning out of it and 
can take some actions based on it.


Diffs (updated)
-

  core/pom.xml 783 
  core/src/main/java/org/apache/oozie/service/MetricsReporter.java PRE-CREATION 
  core/src/main/java/org/apache/oozie/util/Instrumentation.java 3dfb67a 
  core/src/main/java/org/apache/oozie/util/OozieMonitoringComponent.java 
PRE-CREATION 
  core/src/main/resources/oozie-default.xml 8960073 

Diff: https://reviews.apache.org/r/34854/diff/


Testing
---

Done


Thanks,

Narayan Periwal

[jira] [Updated] (OOZIE-2251) Expose instrumental matrices in Realtime Graphing tool

2015-06-05 Thread Narayan Periwal (JIRA)


 [ 
https://issues.apache.org/jira/browse/OOZIE-2251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Narayan Periwal updated OOZIE-2251:
---
Attachment: OOZIE-2251-v4.patch

> Expose instrumental matrices in Realtime Graphing tool
> --
>
> Key: OOZIE-2251
> URL: https://issues.apache.org/jira/browse/OOZIE-2251
> Project: Oozie
>  Issue Type: New Feature
>  Components: monitoring
>Reporter: Jaydeep Vishwakarma
>Assignee: Narayan Periwal
> Attachments: OOZIE-2251-v0.patch, OOZIE-2251-v1.patch, 
> OOZIE-2251-v2.patch, OOZIE-2251-v3.patch, OOZIE-2251-v4.patch
>
>
> We have been logging so many important matrices in oozie-instrumentation.log 
> . These information is very useful for oozie functional monitoring. But it is 
> always difficult to get the meaning from flat file. If we expose this 
> information on some graphing tool, We can get the lot of meaning out of it 
> and can take some actions based on it. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2195) Expose Pig read/write bytes details as counters in Launcher

2015-06-05 Thread Laxman (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-2195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14574235#comment-14574235
 ] 

Laxman commented on OOZIE-2195:
---

+1 (non-binding) for the idea.
[~jaydeepvishwakarma], how about having some consolidated counters as well. 
For ex: total number of map tasks, total number of reduce tasks, total records 
written, total bytes written, bags spilled, records spilled and anything that 
makes sense of pig script performance.

Following are some counters emitted in log.
{noformat}
Counters:
Total records written : 1258632411
Total bytes written : 49810853810
Spillable Memory Manager spill count : 0
Total bags proactively spilled: 16
Total records proactively spilled: 447456
{noformat}

> Expose Pig read/write bytes details as counters in Launcher
> ---
>
> Key: OOZIE-2195
> URL: https://issues.apache.org/jira/browse/OOZIE-2195
> Project: Oozie
>  Issue Type: New Feature
>Reporter: Jaydeep Vishwakarma
>Assignee: Narayan Periwal
>
> Pig action spawns multiple mr jobs and all the aggregated read/write bytes 
> metrics are emitted in logs only. It would be really helpful if we expose 
> those information in Launcher as counters.  This should not be the default 
> behaviour. User can enable this by job level or global 
> configuration.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2259) Create a callback action

2015-06-05 Thread Jaydeep Vishwakarma (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-2259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14574224#comment-14574224
 ] 

Jaydeep Vishwakarma commented on OOZIE-2259:


[~sriksun], Please check my reply below
# Yes like others workflowactions retry will be available.
# Yeah, Idea is same, I will correct the syntax.
# Yes I will do same.
# Just want to keep it simple. 
# I understand why capture-output is useful. yes I will keep it. We can keep 
response code as capture output.
# Yes it is just callback notification. I am not sure if anyone will need the 
response body. We can enhance further if we get requirement.
# As mentioned in #5, We can get this information from capture output.
# We do not need launcher, The implementation would be same as 
FsActionExecutor/EmailActionExecutor.

> Create a callback action 
> -
>
> Key: OOZIE-2259
> URL: https://issues.apache.org/jira/browse/OOZIE-2259
> Project: Oozie
>  Issue Type: New Feature
>  Components: action
>Reporter: Jaydeep Vishwakarma
>Assignee: Jaydeep Vishwakarma
>
> Need an action to send notification to external server by oozie. We should be 
> able to do multiple types of callback, Currently I know jms and http call. It 
> should suppose to have capability to call diffrent types of methods along 
> with n number of arguments. 
> The sample workflow with callback action 
> {code:xml}
> 
> ...
> 
> 
>   [HOST]
>   [METHOD]
>   
>   [KEY][VALUE]
>   
> ...
> 
> ...
> 
> ...
> 
> {code}
> HOST : by the host system can figure out if it is http or jms callback 
> action. System will send the notification to that host.
> METHOD : it can be POST/GET/QUEUE/TOPIC



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2195) Expose Pig read/write bytes details as counters in Launcher

2015-06-05 Thread Jaydeep Vishwakarma (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-2195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14574152#comment-14574152
 ] 

Jaydeep Vishwakarma commented on OOZIE-2195:


Just went through more detail, It would be more helpful , If all read/write 
records and bytes will be expose as counters. 
Let say you had input I1  and output O1 , O2 than pig counters will be. 
pigcounters.I1.IN.RECORDS
pigcounters.I1.IN.BYTES 
pigcounters.O1.OUT.RECORDS
pigcounters.O1.OUT.BYTES
pigcounters.O2.OUT.RECORDS
pigcounters.O2.OUT.BYTES





> Expose Pig read/write bytes details as counters in Launcher
> ---
>
> Key: OOZIE-2195
> URL: https://issues.apache.org/jira/browse/OOZIE-2195
> Project: Oozie
>  Issue Type: New Feature
>Reporter: Jaydeep Vishwakarma
>Assignee: Jaydeep Vishwakarma
>
> Pig action spawns multiple mr jobs and all the aggregated read/write bytes 
> metrics are emitted in logs only. It would be really helpful if we expose 
> those information in Launcher as counters.  This should not be the default 
> behaviour. User can enable this by job level or global 
> configuration.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2195) Expose Pig read/write bytes details as counters in Launcher

2015-06-05 Thread Jaydeep Vishwakarma (JIRA)


 [ 
https://issues.apache.org/jira/browse/OOZIE-2195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jaydeep Vishwakarma updated OOZIE-2195:
---
Assignee: Narayan Periwal  (was: Jaydeep Vishwakarma)

> Expose Pig read/write bytes details as counters in Launcher
> ---
>
> Key: OOZIE-2195
> URL: https://issues.apache.org/jira/browse/OOZIE-2195
> Project: Oozie
>  Issue Type: New Feature
>Reporter: Jaydeep Vishwakarma
>Assignee: Narayan Periwal
>
> Pig action spawns multiple mr jobs and all the aggregated read/write bytes 
> metrics are emitted in logs only. It would be really helpful if we expose 
> those information in Launcher as counters.  This should not be the default 
> behaviour. User can enable this by job level or global 
> configuration.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Jenkins build is back to normal : oozie-trunk-find-patches-available #100306

2015-06-05 Thread Apache Jenkins Server

See

[jira] Subscription: Oozie Patch Available

2015-06-05 Thread jira

Issue Subscription
Filter: Oozie Patch Available (53 issues)

Subscriber: ooziedaily

Key Summary
OOZIE-2262  Fix log streaming from other server with start/end filter
https://issues.apache.org/jira/browse/OOZIE-2262
OOZIE-2258  Introducing a new counter in the instrumentation log to distinguish 
between the reasons for launcher failure
https://issues.apache.org/jira/browse/OOZIE-2258
OOZIE-2257  BundleStatusTransitXCommand doesn't update last modified time.
https://issues.apache.org/jira/browse/OOZIE-2257
OOZIE-2251  Expose instrumental matrices in Realtime Graphing tool
https://issues.apache.org/jira/browse/OOZIE-2251
OOZIE-2250  Show log for WAITING and TIMEDOUT coord actions
https://issues.apache.org/jira/browse/OOZIE-2250
OOZIE-2245  Service to periodically check database schema
https://issues.apache.org/jira/browse/OOZIE-2245
OOZIE-2244  Oozie should mask passwords in the logs when logging command 
arguments
https://issues.apache.org/jira/browse/OOZIE-2244
OOZIE-2243  Kill Command does not kill the child job for java action
https://issues.apache.org/jira/browse/OOZIE-2243
OOZIE-2203  Fix the login example
https://issues.apache.org/jira/browse/OOZIE-2203
OOZIE-2196  Create Local Client for Bundle
https://issues.apache.org/jira/browse/OOZIE-2196
OOZIE-2187  Add a way to specify a default JT/RM and NN
https://issues.apache.org/jira/browse/OOZIE-2187
OOZIE-2178  fix javadoc to compile on JDK8
https://issues.apache.org/jira/browse/OOZIE-2178
OOZIE-2168  Oozie flow and action names have 50 char limit 
https://issues.apache.org/jira/browse/OOZIE-2168
OOZIE-2159  'oozie validate' command should be moved server-side
https://issues.apache.org/jira/browse/OOZIE-2159
OOZIE-2134  Remove references to Services.get().getConf() in code
https://issues.apache.org/jira/browse/OOZIE-2134
OOZIE-2106  Make tomcat download url configurable in the pom file
https://issues.apache.org/jira/browse/OOZIE-2106
OOZIE-2105  Make version of submodules configurable with parent version 
https://issues.apache.org/jira/browse/OOZIE-2105
OOZIE-2099  Add test-patch support for patches generated without --no-prefix
https://issues.apache.org/jira/browse/OOZIE-2099
OOZIE-2081  WorkflowJob notification to include coordinator action id 
https://issues.apache.org/jira/browse/OOZIE-2081
OOZIE-2060  Incorrect documentation of Java action config XML filename
https://issues.apache.org/jira/browse/OOZIE-2060
OOZIE-2044  ssh action succeed with a not exists command which should be fail.
https://issues.apache.org/jira/browse/OOZIE-2044
OOZIE-2020  Rerun all Failed/killed/timedout coordinator actions rather than 
specifying action numbers
https://issues.apache.org/jira/browse/OOZIE-2020
OOZIE-1980  Sql error should not fail coord job
https://issues.apache.org/jira/browse/OOZIE-1980
OOZIE-1977  Display patch analysis issues
https://issues.apache.org/jira/browse/OOZIE-1977
OOZIE-1936  Queuedump command should display queue information for all server.
https://issues.apache.org/jira/browse/OOZIE-1936
OOZIE-1931  Admin command to print all locks held by server(s)
https://issues.apache.org/jira/browse/OOZIE-1931
OOZIE-1927  Use StoreStatusFilter for WorkflowsJobGetJPAExecutor 
https://issues.apache.org/jira/browse/OOZIE-1927
OOZIE-1922  MemoryLocksService fails if lock is acquired multiple times in same 
thread and released
https://issues.apache.org/jira/browse/OOZIE-1922
OOZIE-1918  ActionXCommand refactoring for code reuse
https://issues.apache.org/jira/browse/OOZIE-1918
OOZIE-1884  ZKJobsConcurrencyService.checkJobIdForServer fails if jobs length > 
7
https://issues.apache.org/jira/browse/OOZIE-1884
OOZIE-1860  Oozie job mapper launch fails due to null value returned from 
action file
https://issues.apache.org/jira/browse/OOZIE-1860
OOZIE-1855  TestPriorityDelayQueue#testPoll failed intermittently in Jenkins
https://issues.apache.org/jira/browse/OOZIE-1855
OOZIE-1816  LogInfo uses action name instead of id
https://issues.apache.org/jira/browse/OOZIE-1816
OOZIE-1810  Workflow cannot get into Failed state when kill control node cannot 
resolve variable in message 
https://issues.apache.org/jira/browse/OOZIE-1810
OOZIE-1802  Support workflow action log
https://issues.apache.org/jira/browse/OOZIE-1802
OOZIE-1793  Improve find bugs reporting for Oozie
https://issues.apache.org/jira/browse/OOZIE-1793
OOZIE-1782  Workflow path not found is thrown as SC_UNAUTHORIZED
https://issues.apache.org/jira/browse/OOZIE-1782
OOZIE-1779  Add parent-id as a filter option
https://issues.apache.org/jira/browse/OOZIE-1779
O

Build failed in Jenkins: oozie-trunk-find-patches-available #100305

2015-06-05 Thread Apache Jenkins Server

See 

--
Started by timer
Building remotely on H1 (Mapreduce Hadoop Pig Hdfs) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url 
 > https://git-wip-us.apache.org/repos/asf/oozie.git # timeout=10
Fetching upstream changes from https://git-wip-us.apache.org/repos/asf/oozie.git
 > git --version # timeout=10
 > git fetch --tags --progress 
 > https://git-wip-us.apache.org/repos/asf/oozie.git 
 > +refs/heads/*:refs/remotes/origin/*
 > git rev-parse refs/remotes/origin/master^{commit} # timeout=10
 > git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10
Checking out Revision 48b64bc9438137517e24b37b674c5a8893db67c3 
(refs/remotes/origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 48b64bc9438137517e24b37b674c5a8893db67c3
 > git rev-list 48b64bc9438137517e24b37b674c5a8893db67c3 # timeout=10
[oozie-trunk-find-patches-available] $ /bin/bash 
/tmp/hudson4447291893421088877.sh
mkdir: cannot create directory 
':
 File exists
  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed

  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
Warning: Transient problem: HTTP error Will retry in 1 seconds. 3 retries 
Warning: left.

  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
Warning: Transient problem: HTTP error Will retry in 2 seconds. 2 retries 
Warning: left.

  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
Warning: Transient problem: HTTP error Will retry in 4 seconds. 1 retries 
Warning: left.

  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
curl: (22) The requested URL returned error: 503 Service Temporarily Unavailable
Could not retrieve available patches from JIRA
Build step 'Execute shell' marked build as failure

Build failed in Jenkins: oozie-trunk-find-patches-available #100304

2015-06-05 Thread Apache Jenkins Server

See 

--
Started by timer
Building remotely on H1 (Mapreduce Hadoop Pig Hdfs) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url 
 > https://git-wip-us.apache.org/repos/asf/oozie.git # timeout=10
Fetching upstream changes from https://git-wip-us.apache.org/repos/asf/oozie.git
 > git --version # timeout=10
 > git fetch --tags --progress 
 > https://git-wip-us.apache.org/repos/asf/oozie.git 
 > +refs/heads/*:refs/remotes/origin/*
 > git rev-parse refs/remotes/origin/master^{commit} # timeout=10
 > git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10
Checking out Revision 48b64bc9438137517e24b37b674c5a8893db67c3 
(refs/remotes/origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 48b64bc9438137517e24b37b674c5a8893db67c3
 > git rev-list 48b64bc9438137517e24b37b674c5a8893db67c3 # timeout=10
[oozie-trunk-find-patches-available] $ /bin/bash 
/tmp/hudson1544400174831976249.sh
mkdir: cannot create directory 
':
 File exists
  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed

  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
Warning: Transient problem: HTTP error Will retry in 1 seconds. 3 retries 
Warning: left.

  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
Warning: Transient problem: HTTP error Will retry in 2 seconds. 2 retries 
Warning: left.

  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
Warning: Transient problem: HTTP error Will retry in 4 seconds. 1 retries 
Warning: left.

  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
curl: (22) The requested URL returned error: 503 Service Temporarily Unavailable
Could not retrieve available patches from JIRA
Build step 'Execute shell' marked build as failure

Build failed in Jenkins: oozie-trunk-find-patches-available #100302

2015-06-05 Thread Apache Jenkins Server

See 

--
[...truncated 35 lines...]
  0 00 00 0  0  0 --:--:--  0:00:16 --:--:-- 0
  0 00 00 0  0  0 --:--:--  0:00:17 --:--:-- 0
  0 00 00 0  0  0 --:--:--  0:00:18 --:--:-- 0
  0 00 00 0  0  0 --:--:--  0:00:19 --:--:-- 0
  0 00 00 0  0  0 --:--:--  0:00:20 --:--:-- 0
  0 00 00 0  0  0 --:--:--  0:00:21 --:--:-- 0
  0 00 00 0  0  0 --:--:--  0:00:22 --:--:-- 0
  0 00 00 0  0  0 --:--:--  0:00:23 --:--:-- 0
  0 00 00 0  0  0 --:--:--  0:00:24 --:--:-- 0
  0 00 00 0  0  0 --:--:--  0:00:25 --:--:-- 0
  0 00 00 0  0  0 --:--:--  0:00:26 --:--:-- 0
  0 00 00 0  0  0 --:--:--  0:00:27 --:--:-- 0
  0 00 00 0  0  0 --:--:--  0:00:28 --:--:-- 0
  0 00 00 0  0  0 --:--:--  0:00:29 --:--:-- 0
  0 00 00 0  0  0 --:--:--  0:00:30 --:--:-- 0
  0 00 00 0  0  0 --:--:--  0:00:31 --:--:-- 0
  0 00 00 0  0  0 --:--:--  0:00:32 --:--:-- 0
  0 00 00 0  0  0 --:--:--  0:00:33 --:--:-- 0
  0 00 00 0  0  0 --:--:--  0:00:34 --:--:-- 0
  0 00 00 0  0  0 --:--:--  0:00:35 --:--:-- 0
  0 00 00 0  0  0 --:--:--  0:00:36 --:--:-- 0
  0 00 00 0  0  0 --:--:--  0:00:37 --:--:-- 0
  0 00 00 0  0  0 --:--:--  0:00:38 --:--:-- 0
  0 00 00 0  0  0 --:--:--  0:00:39 --:--:-- 0
  0 00 00 0  0  0 --:--:--  0:00:40 --:--:-- 0
  0 00 00 0  0  0 --:--:--  0:00:41 --:--:-- 0
  0 00 00 0  0  0 --:--:--  0:00:42 --:--:-- 0
  0 00 00 0  0  0 --:--:--  0:00:43 --:--:-- 0
  0 00 00 0  0  0 --:--:--  0:00:44 --:--:-- 0
  0 00 00 0  0  0 --:--:--  0:00:45 --:--:-- 0
  0 00 00 0  0  0 --:--:--  0:00:46 --:--:-- 0
  0 00 00 0  0  0 --:--:--  0:00:47 --:--:-- 0
  0 00 00 0  0  0 --:--:--  0:00:48 --:--:-- 0
  0 00 00 0  0  0 --:--:--  0:00:49 --:--:-- 0
  0 00 00 0  0  0 --:--:--  0:00:50 --:--:-- 0
  0 00 00 0  0  0 --:--:--  0:00:51 --:--:-- 0
  0 00 00 0  0  0 --:--:--  0:00:52 --:--:-- 0
  0 00 00 0  0  0 --:--:--  0:00:53 --:--:-- 0
  0 00 00 0  0  0 --:--:--  0:00:54 --:--:-- 0
  0 00 00 0  0  0 --:--:--  0:00:55 --:--:-- 0
  0 00 00 0  0  0 --:--:--  0:00:56 --:--:-- 0
  0 00 00 0  0  0 --:--:--  0:00:57 --:--:-- 0
  0 00 00 0  0  0 --:--:--  0:00:58 --:--:-- 0
  0 00 00 0  0  0 --:--:--  0:00:59 --:--:-- 0
  0 00 00 0  0  0 --:--:--  0:01:00 --:--:-- 0
  0 00 00 0  0  0 --:--:--  0:01:01 --:--:-- 0
  0 00 00 0  0  0 --:--:--  0:01:02 --:--:-- 0
  0 00 00 0  0  0 --:--:--  0:01:03 --:--:-- 0
  0 00 00 0  0  0 --:--:--  0:01:03 --:--:-- 0
Warning: Transient problem: HTTP error Will retry in 1 seconds. 3 retries 
Warning: left.

  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
  0 00 00 0  0  0 --:--:--  0:00:01 --:--:-- 0
  0 00 00 0  0  0 --:--:--  0:00:02 --:--:-- 0
  0 00 00 0  0  0 --:--:--  0:00:03 --:--:-- 0
  0 00 00 0  0  0 --:--:--  0:00:04 --:--:-- 0
  0 00 00 0  0  0 --:--:--  0:00:05 --:--:-- 0
  0 00 00 0  0  0 --:--:--  0:00:06 --:--:-- 0
  0 00 00 0  0  0 --:--:--  0:00:07 --:--:-- 0
  0 00 00 0  0  0 --:--:--  0:00:08 --:--:-- 0
  0 00 00 0  0  0 --:--:--  0:00:09 --:--:-- 0
  0 00 00 0  0  0 --:--:--  0:00:10 --:--:-- 0
  0 00

[jira] Subscription: Oozie Patch Available

Re: Review Request 33400: OOZIE-2215 Support glob in FS EL function

[jira] [Commented] (OOZIE-2195) Expose Pig read/write bytes details as counters in Launcher

[jira] [Commented] (OOZIE-2251) Expose instrumental matrices in Realtime Graphing tool

Build failed in Jenkins: oozie-trunk-precommit-build #2392

Re: Review Request 34854: OOZIE-2251 : Expose instrumental matrices in Realtime Graphing tool

Re: Review Request 34854: OOZIE-2251 : Expose instrumental matrices in Realtime Graphing tool

[jira] [Updated] (OOZIE-2251) Expose instrumental matrices in Realtime Graphing tool

[jira] [Commented] (OOZIE-2195) Expose Pig read/write bytes details as counters in Launcher

[jira] [Commented] (OOZIE-2259) Create a callback action

[jira] [Commented] (OOZIE-2195) Expose Pig read/write bytes details as counters in Launcher

[jira] [Updated] (OOZIE-2195) Expose Pig read/write bytes details as counters in Launcher

Jenkins build is back to normal : oozie-trunk-find-patches-available #100306

[jira] Subscription: Oozie Patch Available

Build failed in Jenkins: oozie-trunk-find-patches-available #100305

Build failed in Jenkins: oozie-trunk-find-patches-available #100304

Build failed in Jenkins: oozie-trunk-find-patches-available #100302

17 matches

Site Navigation

Mail list logo

Footer information