Hi all,
Suppose I submit the attached Coordinator job. The important parts to note
are that the start time is 09/03/2014 at 9:00am, the URI template ends with
/${YEAR}/${MONTH}/${DAY}/${HOUR}, and the input-event is using
${coord:current(0)}.
When you submit this, current(0) seems to be incorrect:
ID : 0000000-140923162340061-oozie-rkan-C@1
----------------------------------------------------------------------------------------------------------
Action Number : 1
Console URL : -
Error Code : -
Error Message : -
External ID : -
External Status : -
Job ID : 0000000-140923162340061-oozie-rkan-C
Tracker URI : -
Created : 2014-09-23 23:23 GMT
Nominal Time : 2014-09-03 *09*:00 GMT
Status : WAITING
Last Modified : 2014-09-23 23:23 GMT
First Missing Dependency :
hdfs://localhost:8020/user/rkanter/path_coord/input-data/2014/09/03/*08*
/_SUCCESS
----------------------------------------------------------------------------------------------------------
*Notice that the first missing dependency ends with 08 instead of 09, even
though the nominal time is 9am!*
I was poking through the code and it looks like the problem may be in
CoordELFunctions.getCurrentInstance(...) which subtracts once frequency
amount from the computed current instance. If I remove that code, it
appears to behave correctly:
diff --git
a/core/src/main/java/org/apache/oozie/coord/CoordELFunctions.java
b/core/src/main/java/org/apache/oozie/coord/CoordELFunctions.java
index 3bb191e..02f4166 100644
--- a/core/src/main/java/org/apache/oozie/coord/CoordELFunctions.java
+++ b/core/src/main/java/org/apache/oozie/coord/CoordELFunctions.java
@@ -1343,8 +1343,6 @@ public class CoordELFunctions {
current.add(dsTimeUnit.getCalendarUnit(), dsFreq);
instanceCount[0]++;
}
- current.add(dsTimeUnit.getCalendarUnit(), -dsFreq);
- instanceCount[0]--;
return current;
}
When I run the original code through a debugger, current is set to 8am; my
modified code has 9am.
Is there something I'm missing? I feel like we would have noticed this by
now because current is pretty common...
This code was greatly changed a few months ago by OOZIE-1709, but the old
code did the same thing were it would subtract at the end and had 8am
instead of 9am.
thanks
- Robert
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<coordinator-app name="path_coord" frequency="${coord:hours(1)}" start="2014-09-03T09:00Z" end="2113-01-01T00:00Z" timezone="UTC"
xmlns="uri:oozie:coordinator:0.2">
<controls>
<timeout>60</timeout>
<concurrency>1</concurrency>
</controls>
<datasets>
<dataset name="testDataset" frequency="${coord:hours(1)}" initial-instance="2013-01-01T00:01Z" timezone="UTC">
<uri-template>${nameNode}/user/${coord:user()}/path_coord/input-data/${YEAR}/${MONTH}/${DAY}/${HOUR}</uri-template>
</dataset>
</datasets>
<input-events>
<data-in name="test-event" dataset="testDataset">
<instance>${coord:current(0)}</instance>
</data-in>
</input-events>
<action>
<workflow>
<app-path>${nameNode}/user/${coord:user()}/${examplesRoot}/apps/map-reduce</app-path>
<configuration>
<property>
<name>jobTracker</name>
<value>${jobTracker}</value>
</property>
<property>
<name>nameNode</name>
<value>${nameNode}</value>
</property>
<property>
<name>queueName</name>
<value>${queueName}</value>
</property>
<property>
<name>examplesRoot</name>
<value>${examplesRoot}</value>
</property>
</configuration>
</workflow>
</action>
</coordinator-app>
