Oh, I see now. Wow, some of these things can get quite tricky and subtle.
I'm glad we didn't have such a scary flaw here :)
thanks
- Robert
On Tue, Sep 23, 2014 at 10:11 PM, Shwetha GS <shwetha...@inmobi.com> wrote:
> The ds initial instance is "2013-01-01T00:01Z"(Note 1st minute of hour).
> So, the instance before nominal time 2014-09-03T*09*:00Z is 2014/09/03/*08.
> If you change the initial instance to *"2013-01-01T00:00Z", the resolved
> path will be for 2014/09/03/*09*
>
> *-Shwetha*
>
> On Wed, Sep 24, 2014 at 5:11 AM, Robert Kanter <rkan...@cloudera.com>
> wrote:
>
>> Hi all,
>>
>> Suppose I submit the attached Coordinator job. The important parts to
>> note are that the start time is 09/03/2014 at 9:00am, the URI template ends
>> with /${YEAR}/${MONTH}/${DAY}/${HOUR}, and the input-event is using
>> ${coord:current(0)}.
>>
>> When you submit this, current(0) seems to be incorrect:
>> ID : 0000000-140923162340061-oozie-rkan-C@1
>>
>> ----------------------------------------------------------------------------------------------------------
>> Action Number : 1
>> Console URL : -
>> Error Code : -
>> Error Message : -
>> External ID : -
>> External Status : -
>> Job ID : 0000000-140923162340061-oozie-rkan-C
>> Tracker URI : -
>> Created : 2014-09-23 23:23 GMT
>> Nominal Time : 2014-09-03 *09*:00 GMT
>> Status : WAITING
>> Last Modified : 2014-09-23 23:23 GMT
>> First Missing Dependency :
>> hdfs://localhost:8020/user/rkanter/path_coord/input-data/2014/09/03/*08*
>> /_SUCCESS
>>
>> ----------------------------------------------------------------------------------------------------------
>> *Notice that the first missing dependency ends with 08 instead of 09,
>> even though the nominal time is 9am!*
>>
>> I was poking through the code and it looks like the problem may be in
>> CoordELFunctions.getCurrentInstance(...) which subtracts once frequency
>> amount from the computed current instance. If I remove that code, it
>> appears to behave correctly:
>> diff --git
>> a/core/src/main/java/org/apache/oozie/coord/CoordELFunctions.java
>> b/core/src/main/java/org/apache/oozie/coord/CoordELFunctions.java
>> index 3bb191e..02f4166 100644
>> --- a/core/src/main/java/org/apache/oozie/coord/CoordELFunctions.java
>> +++ b/core/src/main/java/org/apache/oozie/coord/CoordELFunctions.java
>> @@ -1343,8 +1343,6 @@ public class CoordELFunctions {
>> current.add(dsTimeUnit.getCalendarUnit(), dsFreq);
>> instanceCount[0]++;
>> }
>> - current.add(dsTimeUnit.getCalendarUnit(), -dsFreq);
>> - instanceCount[0]--;
>> return current;
>> }
>> When I run the original code through a debugger, current is set to 8am;
>> my modified code has 9am.
>>
>> Is there something I'm missing? I feel like we would have noticed this
>> by now because current is pretty common...
>>
>> This code was greatly changed a few months ago by OOZIE-1709, but the old
>> code did the same thing were it would subtract at the end and had 8am
>> instead of 9am.
>>
>>
>> thanks
>> - Robert
>>
>
>
> _____________________________________________________________
> The information contained in this communication is intended solely for the
> use of the individual or entity to whom it is addressed and others
> authorized to receive it. It may contain confidential or legally privileged
> information. If you are not the intended recipient you are hereby notified
> that any disclosure, copying, distribution or taking any action in reliance
> on the contents of this information is strictly prohibited and may be
> unlawful. If you have received this communication in error, please notify
> us immediately by responding to this email and then delete it from your
> system. The firm is neither liable for the proper and complete transmission
> of the information contained in this communication nor for any delay in its
> receipt.
