[jira] [Updated] (OOZIE-1936) Queuedump command should display queue information for all server.

2014-09-11 Thread Azrael (JIRA) 

 [ 
https://issues.apache.org/jira/browse/OOZIE-1936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Azrael updated OOZIE-1936:
--
Attachment: OOZIE-1936.1.patch

> Queuedump command should display queue information for all server.
> --
>
> Key: OOZIE-1936
> URL: https://issues.apache.org/jira/browse/OOZIE-1936
> Project: Oozie
>  Issue Type: Bug
>  Components: HA
>Reporter: Purshotam Shah
> Attachments: OOZIE-1936.1.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 24165: OOZIE-1954: Add a way for the MapReduce action to be configured by Java code

2014-09-11 Thread shwetha . gs 


> On Sept. 9, 2014, 9:24 a.m., shwethags wrote:
> > docs/src/site/twiki/WorkflowFunctionalSpec.twiki, line 605
> > 
> >
> > Instead of depending on oozie artifact for the interface(which is 
> > difficult to get given that oozie artifacts are not available in any mvn 
> > repo), can the interface be org.apache.hadoop.util.Tool. The user provided 
> > run() can update the Conf. 
> > 
> > run() doesn't sound like setup, but serves the purpose with conf.
> 
> Robert Kanter wrote:
> Using Tool gives us less control if we want to make any changes (e.g. add 
> new methods, etc) in the future, so I'd prefer to use an Oozie-specific 
> interface.  It's also more clear what this is for.
> 
> I think the real solution for that difficulting is to publish artifacts, 
> which many users have been asking for, and everyone seems to agree that we 
> should do.  I'm hoping that we'll publish starting with the 4.1 release.
> 
> Robert Kanter wrote:
> I've talked with Bowen offline and he's going to publish the 4.1  
> (OOZIE-1842).  And by the time we actually release OOZIE-1954, publishing to 
> maven should be part of the release process.

I am fine with it if the oozie artifacts are available. 

If we move the interface to oozie-client, then the launcher job will need 
oozie-client jar as well?


- shwethags


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24165/#review52696
---


On July 31, 2014, 10:33 p.m., Robert Kanter wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/24165/
> ---
> 
> (Updated July 31, 2014, 10:33 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1954
> https://issues.apache.org/jira/browse/OOZIE-1954
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> See https://issues.apache.org/jira/browse/OOZIE-1954
> 
> 
> Diffs
> -
> 
>   client/src/main/resources/oozie-workflow-0.5.xsd 6620a4e 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 
> 94b55cf 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 
> 7fe1f42 
>   docs/src/site/twiki/WorkflowFunctionalSpec.twiki 3319bcc 
>   examples/src/main/apps/map-reduce/job-with-config-class.properties 
> PRE-CREATION 
>   examples/src/main/apps/map-reduce/job.properties 7b7a24c 
>   examples/src/main/apps/map-reduce/workflow-with-config-class.xml 
> PRE-CREATION 
>   
> examples/src/main/java/org/apache/oozie/example/SampleOozieActionConfigurator.java
>  PRE-CREATION 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 
> 2cb690f 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java
>  322f498 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java
>  f417a78 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/OozieActionConfigurator.java
>  PRE-CREATION 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/OozieActionConfiguratorException.java
>  PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PipesMain.java 
> 67a2a43 
>   
> sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/OozieActionConfiguratorForTest.java
>  PRE-CREATION 
>   
> sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java
>  e78c98a 
>   
> sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutorUberJar.java
>  bcc22f2 
> 
> Diff: https://reviews.apache.org/r/24165/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Robert Kanter
> 
>

[jira] [Updated] (OOZIE-1980) Sql error should not fail coord job

2014-09-11 Thread Shwetha G S (JIRA) 

 [ 
https://issues.apache.org/jira/browse/OOZIE-1980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shwetha G S updated OOZIE-1980:
---
Assignee: Purshotam Shah

> Sql error should not fail coord job
> ---
>
> Key: OOZIE-1980
> URL: https://issues.apache.org/jira/browse/OOZIE-1980
> Project: Oozie
>  Issue Type: Bug
>Reporter: Purshotam Shah
>Assignee: Purshotam Shah
> Attachments: OOZIE-1980-V1.patch, OOZIE-1980-V2.patch
>
>
> We know that sql error is transient, it should not fail the job.
> We should ignore the SQL error, so that next materialization cycle can pick 
> the job.
> {code}
> protected void materialize() throws CommandException {
> Instrumentation.Cron cron = new Instrumentation.Cron();
> cron.start();
> try {
> materializeActions(false);
> updateJobMaterializeInfo(coordJob);
> }
> catch (CommandException ex) {
> LOG.warn("Exception occurred:" + ex.getMessage() + " Making the 
> job failed ", ex);
> coordJob.setStatus(Job.Status.FAILED);
> coordJob.resetPending();
> // remove any materialized actions and slaEvents
> insertList.clear();
> }
> catch (Exception e) {
> LOG.error("Exception occurred:" + e.getMessage() + " Making the 
> job failed ", e);
> coordJob.setStatus(Job.Status.FAILED);
> try {
> 
> CoordJobQueryExecutor.getInstance().executeUpdate(CoordJobQuery.UPDATE_COORD_JOB_MATERIALIZE,
>  coordJob);
> }
> catch (JPAExecutorException jex) {
> throw new CommandException(ErrorCode.E1011, jex);
> }
> throw new CommandException(ErrorCode.E1012, e.getMessage(), e);
> } finally {
> cron.stop();
> instrumentation.addCron(INSTRUMENTATION_GROUP, getName() + 
> ".materialize", cron);
> }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-1922) MemoryLocksService fails if lock is acquired multiple times in same thread and released

2014-09-11 Thread Shwetha G S (JIRA) 

 [ 
https://issues.apache.org/jira/browse/OOZIE-1922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shwetha G S updated OOZIE-1922:
---
Assignee: Azrael

> MemoryLocksService fails if lock is acquired multiple times in same thread 
> and released
> ---
>
> Key: OOZIE-1922
> URL: https://issues.apache.org/jira/browse/OOZIE-1922
> Project: Oozie
>  Issue Type: Bug
>Reporter: Purshotam Shah
>Assignee: Azrael
> Attachments: OOZIE-1922.1.patch
>
>
> ReentrantLock can be acquired multiple times in same thread. For multiple 
> acquire call, ReentrantLock hold count is incremented by one.
> So if we acquire lock multiple time from same thread, all will be successful 
> and  hold count is increased for every call.
> But if we release lock, MemoryLocksService ignore the count and deletes the 
> lock. Even if it's held by some command.
> Simple step can reproduce it.
> {code}
> service.getWriteLock("test", 5000); //writeHoldCount = 1
> MemoryLockToken lock = (MemoryLockToken)service.getWriteLock("test", 
> 5000); //writeHoldCount = 2
> lock.release(); //writeHoldCount = 1
> lock = (MemoryLockToken)service.getWriteLock("test", 5000); 
> //writeHoldCount = 1, it should be 2.
> {code}
> {code}
> @Override
> public void release() {
> int val = rwLock.getQueueLength();
> if (val == 0) {
> synchronized (locks) {
> locks.remove(resource);
> }
> }
> lock.unlock();
> }
> }
> {code}
> MemoryLocks should check hold count before removing lock.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-1638) Action retry does not use default retry max count.

2014-09-11 Thread Shwetha G S (JIRA) 

 [ 
https://issues.apache.org/jira/browse/OOZIE-1638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shwetha G S updated OOZIE-1638:
---
Assignee: Azrael

> Action retry does not use default retry max count.
> --
>
> Key: OOZIE-1638
> URL: https://issues.apache.org/jira/browse/OOZIE-1638
> Project: Oozie
>  Issue Type: Bug
>  Components: action
>Affects Versions: 4.0.0
>Reporter: Azrael
>Assignee: Azrael
>Priority: Minor
> Attachments: OOZIE-1638.2.patch, OOZIE-1638.3.patch, 
> OOZIE-1638.4.patch, OOZIE-1638.5.patch, OOZIE-1638.6.patch, OOZIE-1638.patch
>
>
> the oozie-default
> {code:xml}
> 
> oozie.service.LiteWorkflowStoreService.user.retry.max
> 3
> 
> Automatic retry max count for workflow action is 3 in default.
> 
> 
> {code}
> but the action failed with error, is not retried. 
> It should retry $\{user.retry.max\} times.
> If the patch applied, some other tests in TestActionError failed because the 
> action is not terminated for retrying.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Review Request 25531: OOZIE-1936 Queuedump command should display queue information for all server.

2014-09-11 Thread Azrael Park 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25531/
---

Review request for oozie.


Bugs: OOZIE-1936
https://issues.apache.org/jira/browse/OOZIE-1936


Repository: oozie-git


Description
---

Queuedump command should display queue information for all server.

In AdminServlet, the arguemnt of getQueueDump() method is changed from 
JSONObject to JSONArray. 
Hence, V1AdminServlet#getQueueDump() is revised.
I am not sure we should keep the implementation of 
V1AdminServlet#getQueueDump(). 


Diffs
-

  client/src/main/java/org/apache/oozie/client/OozieClient.java d6ff2d0 
  client/src/main/java/org/apache/oozie/client/rest/JsonTags.java 8a86bf1 
  core/src/main/java/org/apache/oozie/servlet/BaseAdminServlet.java 64d3f1f 
  core/src/main/java/org/apache/oozie/servlet/V0AdminServlet.java fd573d5 
  core/src/main/java/org/apache/oozie/servlet/V1AdminServlet.java 2383433 
  core/src/main/java/org/apache/oozie/servlet/V2AdminServlet.java adaab76 
  core/src/test/java/org/apache/oozie/service/DummyV2AdminServlet.java 0e466fc 
  core/src/test/java/org/apache/oozie/service/TestHAQueueDump.java PRE-CREATION 
  core/src/test/java/org/apache/oozie/servlet/TestV1AdminServlet.java 35568e1 
  core/src/test/java/org/apache/oozie/servlet/TestV2AdminServlet.java 
PRE-CREATION 

Diff: https://reviews.apache.org/r/25531/diff/


Testing
---

Unit test done.
Manual test done.


Thanks,

Azrael Park

[jira] [Commented] (OOZIE-1936) Queuedump command should display queue information for all server.

2014-09-11 Thread Azrael (JIRA) 

[ 
https://issues.apache.org/jira/browse/OOZIE-1936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14129752#comment-14129752
 ] 

Azrael commented on OOZIE-1936:
---

The queuedump result is expected as following:

{noformat}
=
[Server Host]: http://az0:11000/oozie
[Server Queue Dump]:
[action.notification_c37c80ff-9bb2-4bb2-bfed-0af2f912693c] priority=0 
delay=56818
**
[Server Uniqueness Map Dump]:
action.notification_94e54f0f-cc03-4fbd-993d-caefcc42a785=Fri Sep 05 13:34:15 
KST 2014
=
[Server Host]: http://az4:11000/oozie
[Server Queue Dump]:
[action.notification_42a69b63-119a-4406-a83e-59cf060af870] priority=0 
delay=55079
**
[Server Uniqueness Map Dump]:
action.notification_52200866-7c5e-444d-ae91-194072246229=Fri Sep 05 13:34:12 
KST 2014
=
[Server Host]: http://az3:11000/oozie
[Server Queue Dump]:
[action.notification_37e810a3-1d6d-4852-995a-52dd4877e2f5] priority=0 
delay=52984
**
[Server Uniqueness Map Dump]:
action.notification_37e810a3-1d6d-4852-995a-52dd4877e2f5=Fri Sep 05 13:36:06 
KST 2014
{noformat}

I linked reviewboard.


> Queuedump command should display queue information for all server.
> --
>
> Key: OOZIE-1936
> URL: https://issues.apache.org/jira/browse/OOZIE-1936
> Project: Oozie
>  Issue Type: Bug
>  Components: HA
>Reporter: Purshotam Shah
> Attachments: OOZIE-1936.1.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (OOZIE-1936) Queuedump command should display queue information for all server.

2014-09-11 Thread Azrael (JIRA) 

 [ 
https://issues.apache.org/jira/browse/OOZIE-1936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Azrael reassigned OOZIE-1936:
-

Assignee: Azrael

> Queuedump command should display queue information for all server.
> --
>
> Key: OOZIE-1936
> URL: https://issues.apache.org/jira/browse/OOZIE-1936
> Project: Oozie
>  Issue Type: Bug
>  Components: HA
>Reporter: Purshotam Shah
>Assignee: Azrael
> Attachments: OOZIE-1936.1.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Build failed in Jenkins: oozie-trunk-precommit-build #1974

2014-09-11 Thread Apache Jenkins Server 
See 

Changes:

[purushah] OOZIE-1813 Add service to report/kill rogue bundles and coordinator 
jobs

[shwetha.gs] OOZIE-1950 Coordinator job info should support timestamp (nominal 
time) (shwethags)

--
[...truncated 9352 lines...]
[INFO] share/lib already added, skipping
[INFO] Building tar : 

[INFO]  already added, skipping
[INFO] share already added, skipping
[INFO] share/lib already added, skipping
[INFO]  already added, skipping
[INFO] share already added, skipping
[INFO] share/lib already added, skipping
[INFO]  already added, skipping
[INFO] share already added, skipping
[INFO] share/lib already added, skipping
[INFO]  already added, skipping
[INFO] share already added, skipping
[INFO] share/lib already added, skipping
[INFO]  already added, skipping
[INFO] share already added, skipping
[INFO] share/lib already added, skipping
[INFO]  already added, skipping
[INFO] share already added, skipping
[INFO] share/lib already added, skipping
[INFO] 
[INFO] 
[INFO] Building Apache Oozie Tools 4.2.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
oozie-tools ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ oozie-tools 
---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
oozie-tools ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 2 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
oozie-tools ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ oozie-tools ---
[INFO] Tests are skipped.
[INFO] 
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @ oozie-tools ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-assembly-plugin:2.2.1:single (default-cli) @ oozie-tools ---
[INFO] Reading assembly descriptor: ../src/main/assemblies/tools.xml
[WARNING] The following patterns were never triggered in this artifact 
exclusion filter:
o  '*:*:pom:*'

[INFO] Copying files to 

[WARNING] Assembly file: 

 is not a regular file (it may be a directory). It cannot be attached to the 
project build for installation or deployment.
[INFO] 
[INFO] 
[INFO] Building Apache Oozie MiniOozie 4.2.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
oozie-mini ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ oozie-mini 
---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
oozie-mini ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
oozie-mini ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ oozie-mini ---
[INFO] Tests are skipped.
[INFO] 
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @ oozie-mini ---
[WARNING] JAR will be empty - no content was marked for inclusion!
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-assembly-plugin:2.2.1:single (default-cli) @ oozie-mini ---
[INFO] Reading assembly descriptor: src/main/assemblies/empty.xml
[INFO]

[jira] [Commented] (OOZIE-1936) Queuedump command should display queue information for all server.

2014-09-11 Thread Hadoop QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/OOZIE-1936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14129772#comment-14129772
 ] 

Hadoop QA commented on OOZIE-1936:
--

Testing JIRA OOZIE-1936

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:green}+1{color} the patch does adds/modifies 4 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
{color:red}-1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:red}-1{color} the patch seems to introduce 9 new javac warning(s)
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color} - patch does not compile, cannot run testcases
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}


The full output of the test-patch run is available at

.   https://builds.apache.org/job/oozie-trunk-precommit-build/1974/

> Queuedump command should display queue information for all server.
> --
>
> Key: OOZIE-1936
> URL: https://issues.apache.org/jira/browse/OOZIE-1936
> Project: Oozie
>  Issue Type: Bug
>  Components: HA
>Reporter: Purshotam Shah
>Assignee: Azrael
> Attachments: OOZIE-1936.1.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-1536) Coordinator action reruns start a new workflow

2014-09-11 Thread Rohini Palaniswamy (JIRA) 

[ 
https://issues.apache.org/jira/browse/OOZIE-1536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14130257#comment-14130257
 ] 

Rohini Palaniswamy commented on OOZIE-1536:
---

[~shwethags],
   We would still prefer having one new workflow for coord action reruns for 
maintaining history/change log which is very useful and important as well for 
us and our users. Even yesterday there was a user reported problem and we had 
to look at the previous coord action reruns to determine the issue. This has 
happened a lot of times for us and we did add the "Coord Action Reruns" tab 
specifically for this purpose to easily get the list of reruns. In fact, we 
have asks to show the older runs in the workflow page itself in addition to the 
"Coord Action Reruns" page as that will be more easy. With the current 
behavior, if needed we can easily add checks to not allow rerun of the workflow 
if another workflow from that coord action is running to avoid the issues you 
are seeing. We can also totally not allow rerunning the workflow directly at 
all if coord is a parent, but if they want to quickly rerun and do skip-nodes 
to meet SLA, that will prevent users from doing it. 

  If putting safe guards in the existing behavior does not work for you, then 
we can explore possibilities of having both behaviors available and make that 
configurable as a server side option. 

> Coordinator action reruns start a new workflow
> --
>
> Key: OOZIE-1536
> URL: https://issues.apache.org/jira/browse/OOZIE-1536
> Project: Oozie
>  Issue Type: Improvement
>Reporter: Srikanth Sundarrajan
>
> Coordinator action reruns start a new workflow and if existing workflow for 
> the action is in running state, the same is not checked. Coord rerun can 
> possibly do a workflow re-run to prevent this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Reopened] (OOZIE-1950) Coordinator job info should support timestamp (nominal time)

2014-09-11 Thread Bowen Zhang (JIRA) 

 [ 
https://issues.apache.org/jira/browse/OOZIE-1950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bowen Zhang reopened OOZIE-1950:


This is not checked into 4.1 yet.

> Coordinator job info should support timestamp (nominal time)
> 
>
> Key: OOZIE-1950
> URL: https://issues.apache.org/jira/browse/OOZIE-1950
> Project: Oozie
>  Issue Type: Bug
>Reporter: Purshotam Shah
>Assignee: Shwetha G S
> Fix For: 4.1.0
>
> Attachments: OOZIE-1950-v2.patch, OOZIE-1950-v3.patch, 
> OOZIE-1950-v4.patch, OOZIE-1950-v4.patch, OOZIE-1950-v6.patch, 
> OOZIE-1950-v7.patch, OOZIE-1950-v7.patch, OOZIE-1950.patch
>
>
> It will be useful for user to list all actions (with length and offset) 
> after/before nominal time.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-1997) Oozie cannot materialize coordinator job with sqlserver in windows

2014-09-11 Thread Bowen Zhang (JIRA) 

[ 
https://issues.apache.org/jira/browse/OOZIE-1997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14130295#comment-14130295
 ] 

Bowen Zhang commented on OOZIE-1997:


Committed to trunk and 4.1 branch. Thanks for reviewing.

> Oozie cannot materialize coordinator job with sqlserver in windows
> --
>
> Key: OOZIE-1997
> URL: https://issues.apache.org/jira/browse/OOZIE-1997
> Project: Oozie
>  Issue Type: Bug
>Reporter: Bowen Zhang
>Assignee: Bowen Zhang
>Priority: Blocker
> Fix For: 4.1.0
>
> Attachments: oozie-1997.patch
>
>
> The error of " An aggregate may not appear in the WHERE clause unless it is 
> in a subquery contained in a HAVING clause or a select list, and the column 
> being aggregated is an outer reference" is specific to sqlserver since it's 
> stricter on subquery syntax. This is introduced by 
> https://issues.apache.org/jira/browse/OOZIE-1527. As a result of this change, 
> coordinator job will not work in windows environment.
> {code}
> 2014-09-06 00:06:06,144 ERROR 
> CoordMaterializeTriggerService$CoordMaterializeTriggerRunnable:540 - 
> SERVER[ONPREM-HCAT1] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] 
> Exception while attempting to materialize coordinator jobs, An aggregate may 
> not appear in the WHERE clause unless it is in a subquery contained in a 
> HAVING clause or a select list, and the column being aggregated is an outer 
> reference. {prepstmnt 486918974 SELECT TOP 50 t0.id, t0.last_modified_time 
> FROM COORD_JOBS t0 WHERE (t0.start_time <= ? AND (t0.status = ? OR t0.status 
> = ? OR t0.status = ?) AND (t0.next_matd_time < ? OR t0.next_matd_time IS 
> NULL) AND (t0.next_matd_time IS NULL OR t0.end_time > t0.next_matd_time AND 
> (t0.pause_time IS NULL OR t0.pause_time > t0.next_matd_time)) AND 
> t0.mat_throttling > (SELECT COUNT(t0.id) FROM COORD_ACTIONS t1 WHERE 
> (t1.job_id = t0.id AND t1.status = ?))) ORDER BY t0.last_modified_time ASC} 
> [code=147, state=S0001]
>  
> org.apache.openjpa.persistence.PersistenceException: An aggregate may not 
> appear in the WHERE clause unless it is in a subquery contained in a HAVING 
> clause or a select list, and the column being aggregated is an outer 
> reference. {prepstmnt 486918974 SELECT TOP 50 t0.id, t0.last_modified_time 
> FROM COORD_JOBS t0 WHERE (t0.start_time <= ? AND (t0.status = ? OR t0.status 
> = ? OR t0.status = ?) AND (t0.next_matd_time < ? OR t0.next_matd_time IS 
> NULL) AND (t0.next_matd_time IS NULL OR t0.end_time > t0.next_matd_time AND 
> (t0.pause_time IS NULL OR t0.pause_time > t0.next_matd_time)) AND 
> t0.mat_throttling > (SELECT COUNT(t0.id) FROM COORD_ACTIONS t1 WHERE 
> (t1.job_id = t0.id AND t1.status = ?))) ORDER BY t0.last_modified_time ASC} 
> [code=147, state=S0001]
> FailedObject: select w.id from CoordinatorJobBean w where w.startTimestamp <= 
> :matTime AND (w.statusStr = 'PREP' OR w.statusStr = 'RUNNING' or w.statusStr 
> = 'RUNNINGWITHERROR') AND (w.nextMaterializedTimestamp < :matTime OR 
> w.nextMaterializedTimestamp IS NULL) AND (w.nextMaterializedTimestamp IS NULL 
> OR (w.endTimestamp > w.nextMaterializedTimestamp AND (w.pauseTimestamp IS 
> NULL OR w.pauseTimestamp > w.nextMaterializedTimestamp))) and w.matThrottling 
> > ( select count(w.id) from CoordinatorActionBean a where a.jobId = w.id and 
> a.statusStr = 'WAITING') order by w.lastModifiedTimestamp [java.lang.String]
>   at 
> org.apache.openjpa.jdbc.sql.DBDictionary.narrow(DBDictionary.java:4962)
>   at 
> org.apache.openjpa.jdbc.sql.DBDictionary.newStoreException(DBDictionary.java:4922)
>   at 
> org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:136)
>   at 
> org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:118)
>   at 
> org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:70)
>   at 
> org.apache.openjpa.jdbc.kernel.SelectResultObjectProvider.handleCheckedException(SelectResultObjectProvider.java:155)
>   at 
> org.apache.openjpa.kernel.QueryImpl$PackingResultObjectProvider.handleCheckedException(QueryImpl.java:2111)
>   at 
> org.apache.openjpa.lib.rop.EagerResultList.(EagerResultList.java:40)
>   at org.apache.openjpa.kernel.QueryImpl.toResult(QueryImpl.java:1251)
>   at org.apache.openjpa.kernel.QueryImpl.execute(QueryImpl.java:1007)
>   at org.apache.openjpa.kernel.QueryImpl.execute(QueryImpl.java:863)
>   at org.apache.openjpa.kernel.QueryImpl.execute(QueryImpl.java:794)
>   at 
> org.apache.openjpa.kernel.DelegatingQuery.execute(DelegatingQuery.java:542)
>   at org.apache.openjpa.persistence.QueryImpl.execute(QueryImpl.java:286)
>   at 
> org.apache.openjpa.persistence.QueryImpl.getResultList(QueryImpl.java:302)
>   at 
> org.apache.oozie.se

oozie on oracle issue

2014-09-11 Thread bowen zhang 
Hi guys,
I just want to ask if any of you encountered this problem with oozie on oracle:
under the same oracle SID, you have multiple db_owners with respective schema. 
ooziedb.sh create -run creates tables in another db_owner's schema even though 
your username in oozie-site.xml is correct.
Bowen

[jira] [Commented] (OOZIE-1950) Coordinator job info should support timestamp (nominal time)

2014-09-11 Thread Rohini Palaniswamy (JIRA) 

[ 
https://issues.apache.org/jira/browse/OOZIE-1950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14130529#comment-14130529
 ] 

Rohini Palaniswamy commented on OOZIE-1950:
---

[~bowenzhangusa]
   This is a new feature and not bug fix. Why does it have to go into 4.1?

> Coordinator job info should support timestamp (nominal time)
> 
>
> Key: OOZIE-1950
> URL: https://issues.apache.org/jira/browse/OOZIE-1950
> Project: Oozie
>  Issue Type: Bug
>Reporter: Purshotam Shah
>Assignee: Shwetha G S
> Fix For: 4.1.0
>
> Attachments: OOZIE-1950-v2.patch, OOZIE-1950-v3.patch, 
> OOZIE-1950-v4.patch, OOZIE-1950-v4.patch, OOZIE-1950-v6.patch, 
> OOZIE-1950-v7.patch, OOZIE-1950-v7.patch, OOZIE-1950.patch
>
>
> It will be useful for user to list all actions (with length and offset) 
> after/before nominal time.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-1950) Coordinator job info should support timestamp (nominal time)

2014-09-11 Thread Bowen Zhang (JIRA) 

[ 
https://issues.apache.org/jira/browse/OOZIE-1950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14130634#comment-14130634
 ] 

Bowen Zhang commented on OOZIE-1950:


The creator of the jira marked it as fixed version 4.1. 

> Coordinator job info should support timestamp (nominal time)
> 
>
> Key: OOZIE-1950
> URL: https://issues.apache.org/jira/browse/OOZIE-1950
> Project: Oozie
>  Issue Type: Bug
>Reporter: Purshotam Shah
>Assignee: Shwetha G S
> Fix For: 4.1.0
>
> Attachments: OOZIE-1950-v2.patch, OOZIE-1950-v3.patch, 
> OOZIE-1950-v4.patch, OOZIE-1950-v4.patch, OOZIE-1950-v6.patch, 
> OOZIE-1950-v7.patch, OOZIE-1950-v7.patch, OOZIE-1950.patch
>
>
> It will be useful for user to list all actions (with length and offset) 
> after/before nominal time.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 24165: OOZIE-1954: Add a way for the MapReduce action to be configured by Java code

2014-09-11 Thread Robert Kanter 


> On Sept. 9, 2014, 9:24 a.m., shwethags wrote:
> > docs/src/site/twiki/WorkflowFunctionalSpec.twiki, line 605
> > 
> >
> > Instead of depending on oozie artifact for the interface(which is 
> > difficult to get given that oozie artifacts are not available in any mvn 
> > repo), can the interface be org.apache.hadoop.util.Tool. The user provided 
> > run() can update the Conf. 
> > 
> > run() doesn't sound like setup, but serves the purpose with conf.
> 
> Robert Kanter wrote:
> Using Tool gives us less control if we want to make any changes (e.g. add 
> new methods, etc) in the future, so I'd prefer to use an Oozie-specific 
> interface.  It's also more clear what this is for.
> 
> I think the real solution for that difficulting is to publish artifacts, 
> which many users have been asking for, and everyone seems to agree that we 
> should do.  I'm hoping that we'll publish starting with the 4.1 release.
> 
> Robert Kanter wrote:
> I've talked with Bowen offline and he's going to publish the 4.1  
> (OOZIE-1842).  And by the time we actually release OOZIE-1954, publishing to 
> maven should be part of the release process.
> 
> shwethags wrote:
> I am fine with it if the oozie artifacts are available. 
> 
> If we move the interface to oozie-client, then the launcher job will need 
> oozie-client jar as well?

Correct.  I think putting it in the oozie-sharelib (or perhaps a new artifact?) 
makes the most sense.


- Robert


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24165/#review52696
---


On July 31, 2014, 10:33 p.m., Robert Kanter wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/24165/
> ---
> 
> (Updated July 31, 2014, 10:33 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1954
> https://issues.apache.org/jira/browse/OOZIE-1954
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> See https://issues.apache.org/jira/browse/OOZIE-1954
> 
> 
> Diffs
> -
> 
>   client/src/main/resources/oozie-workflow-0.5.xsd 6620a4e 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 
> 94b55cf 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 
> 7fe1f42 
>   docs/src/site/twiki/WorkflowFunctionalSpec.twiki 3319bcc 
>   examples/src/main/apps/map-reduce/job-with-config-class.properties 
> PRE-CREATION 
>   examples/src/main/apps/map-reduce/job.properties 7b7a24c 
>   examples/src/main/apps/map-reduce/workflow-with-config-class.xml 
> PRE-CREATION 
>   
> examples/src/main/java/org/apache/oozie/example/SampleOozieActionConfigurator.java
>  PRE-CREATION 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 
> 2cb690f 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java
>  322f498 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java
>  f417a78 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/OozieActionConfigurator.java
>  PRE-CREATION 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/OozieActionConfiguratorException.java
>  PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PipesMain.java 
> 67a2a43 
>   
> sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/OozieActionConfiguratorForTest.java
>  PRE-CREATION 
>   
> sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java
>  e78c98a 
>   
> sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutorUberJar.java
>  bcc22f2 
> 
> Diff: https://reviews.apache.org/r/24165/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Robert Kanter
> 
>

[jira] [Commented] (OOZIE-1959) TestZKUtilsWithSecurity fails

2014-09-11 Thread Robert Kanter (JIRA) 

[ 
https://issues.apache.org/jira/browse/OOZIE-1959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14130959#comment-14130959
 ] 

Robert Kanter commented on OOZIE-1959:
--

While we're at it, if anyone has any good ideas on how to make this test less 
hacky...

> TestZKUtilsWithSecurity fails
> -
>
> Key: OOZIE-1959
> URL: https://issues.apache.org/jira/browse/OOZIE-1959
> Project: Oozie
>  Issue Type: Bug
>Affects Versions: trunk
>Reporter: Robert Kanter
>Assignee: Robert Kanter
>Priority: Blocker
> Fix For: trunk
>
>
> TestZKUtilsWithSecurity fails:
> {noformat}
> ---
>  T E S T S
> ---
> Picked up _JAVA_OPTIONS: -Djava.awt.headless=true
> Concurrency config is parallel='classes', perCoreThreadCount=false, 
> threadCount=1, useUnlimitedThreads=false
> 2014-07-31 16:44:33.786 java[38553:da03] Unable to load realm info from 
> SCDynamicStore
> Running org.apache.oozie.util.TestZKUtilsWithSecurity
> Tests run: 2, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 71.418 sec 
> <<< FAILURE!
> Results :
> Tests in error: 
>   testCheckAndSetACLs(org.apache.oozie.util.TestZKUtilsWithSecurity): E1700: 
> Issue communicating with ZooKeeper: KeeperErrorCode = AuthFailed for /services
>   testNewUsingACLs(org.apache.oozie.util.TestZKUtilsWithSecurity): 
> KeeperErrorCode = AuthFailed for /oozie
> Tests run: 2, Failures: 0, Errors: 2, Skipped: 0
> {noformat}
> It would also be good to get test-patch to run this (it currently does not 
> because it needs to be run in a separate JVM as it changes JVM security 
> settings that can't be unchanged); I think we can actually just create a new 
> module for ZK Security tests (and move this test there) because each module 
> uses a new JVM.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-1950) Coordinator job info should support timestamp (nominal time)

2014-09-11 Thread Shwetha G S (JIRA) 

[ 
https://issues.apache.org/jira/browse/OOZIE-1950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14131108#comment-14131108
 ] 

Shwetha G S commented on OOZIE-1950:


I marked it as 4.1. We have usecases of getting coord actions whose nominal 
time is between some start and end time. We used to do this by setting offset = 
start action number.(yes, offset is for pagination, but served the purpose). 
But in 4.1, there is purging of coord actions for active coordinators. So, 
using offset is not an option anymore. So, we need this feature.

> Coordinator job info should support timestamp (nominal time)
> 
>
> Key: OOZIE-1950
> URL: https://issues.apache.org/jira/browse/OOZIE-1950
> Project: Oozie
>  Issue Type: Bug
>Reporter: Purshotam Shah
>Assignee: Shwetha G S
> Fix For: 4.1.0
>
> Attachments: OOZIE-1950-v2.patch, OOZIE-1950-v3.patch, 
> OOZIE-1950-v4.patch, OOZIE-1950-v4.patch, OOZIE-1950-v6.patch, 
> OOZIE-1950-v7.patch, OOZIE-1950-v7.patch, OOZIE-1950.patch
>
>
> It will be useful for user to list all actions (with length and offset) 
> after/before nominal time.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-1950) Coordinator job info should support timestamp (nominal time)

2014-09-11 Thread Shwetha G S (JIRA) 

[ 
https://issues.apache.org/jira/browse/OOZIE-1950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14131109#comment-14131109
 ] 

Shwetha G S commented on OOZIE-1950:


Committed to branch-4.1

> Coordinator job info should support timestamp (nominal time)
> 
>
> Key: OOZIE-1950
> URL: https://issues.apache.org/jira/browse/OOZIE-1950
> Project: Oozie
>  Issue Type: Bug
>Reporter: Purshotam Shah
>Assignee: Shwetha G S
> Fix For: 4.1.0
>
> Attachments: OOZIE-1950-v2.patch, OOZIE-1950-v3.patch, 
> OOZIE-1950-v4.patch, OOZIE-1950-v4.patch, OOZIE-1950-v6.patch, 
> OOZIE-1950-v7.patch, OOZIE-1950-v7.patch, OOZIE-1950.patch
>
>
> It will be useful for user to list all actions (with length and offset) 
> after/before nominal time.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (OOZIE-1950) Coordinator job info should support timestamp (nominal time)

2014-09-11 Thread Shwetha G S (JIRA) 

 [ 
https://issues.apache.org/jira/browse/OOZIE-1950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shwetha G S resolved OOZIE-1950.

Resolution: Fixed

> Coordinator job info should support timestamp (nominal time)
> 
>
> Key: OOZIE-1950
> URL: https://issues.apache.org/jira/browse/OOZIE-1950
> Project: Oozie
>  Issue Type: Bug
>Reporter: Purshotam Shah
>Assignee: Shwetha G S
> Fix For: 4.1.0
>
> Attachments: OOZIE-1950-v2.patch, OOZIE-1950-v3.patch, 
> OOZIE-1950-v4.patch, OOZIE-1950-v4.patch, OOZIE-1950-v6.patch, 
> OOZIE-1950-v7.patch, OOZIE-1950-v7.patch, OOZIE-1950.patch
>
>
> It will be useful for user to list all actions (with length and offset) 
> after/before nominal time.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)