Hi,Alejandro,
because the patch has some JPA changes, this caused "-1" in the test-patch
result , and if we submit this patch into trunk, it means we should bump up
trunk to v4, and most likely create a new dev branch-3. When oozie users
use v4, to be compatible with old workflow , they maybe need to upgrade
database manually. To avoid upgrading database manually,we can replace
this code "
props.setProperty("openjpa.jdbc.SynchronizeMappings","buildSchema(ForeignKeys=true)");"
into "props.setProperty("openjpa.jdbc.SynchronizeMappings",
"buildSchema(ForeignKeys=true,SchemaAction=refresh)");
"SchemaAction=refresh" means JPA will update database according to our
entity automaticlly .
Thx!
2012/12/20 Alejandro Abdelnur <[email protected]>:
> Thanks for the clarification, still I don't understand why you want to
> modify JPA settings for this. This should be done as part of a command
> queued everything the WF state changes.
>
> Cheers
>
>
> On Thu, Dec 20, 2012 at 2:48 AM, jinwei zhu <[email protected]> wrote:
>
>> Hi,Alejandro,
>> 1. the patch computes total Actions num that will run in one workflow
>> first, and then use "lastActionNumber * 1.0f / totalActionNumber" to
>> compute the progress , the "lastActionNumber" is the number of actions
>> that have done. "totalActionNumber" will be persisted into db.
>> 2. the patch does not compute the progress by getting job progress
>> reporting from the cluster.
>>
>> for example,
>> if the workflow has 4 actions, then the progress will be shown like "
>> 0, 25, 50, 75, 100%";
>> if the workflow has only 1 action, then the progress will be shown
>> like " 0, 100%";
>>
>> Thx
>>
>>
>>
>> 2012/12/20 Alejandro Abdelnur <[email protected]>:
>> > Jinwei Zhu,
>> >
>> > I think you are confused, job progress reporting from the cluster has
>> > nothing to do with JPA.
>> >
>> > Thx
>> >
>> > On Wed, Dec 19, 2012 at 8:48 AM, jinwei zhu <[email protected]>
>> wrote:
>> >
>> >> hi, Alejandro, I think we can change this code in JPAService.java:
>> >> props.setProperty("openjpa.jdbc.SynchronizeMappings",
>> >> "buildSchema(ForeignKeys=true)");
>> >> into
>> >> props.setProperty("openjpa.jdbc.SynchronizeMappings",
>> >> "buildSchema(ForeignKeys=true,SchemaAction=refresh)");
>> >>
>> >> after changing, oozie can update db automaticlly if our entity
changed.
>> >>
>> >>
>> >>
>> >> 2012/12/18 Alejandro Abdelnur (JIRA) <[email protected]>:
>> >> >
>> >> > [
>> >>
>>
https://issues.apache.org/jira/browse/OOZIE-547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13534877#comment-13534877
>> ]
>> >> >
>> >> > Alejandro Abdelnur commented on OOZIE-547:
>> >> > ------------------------------------------
>> >> >
>> >> > If we commit this patch to trunk, it means we should bump up trunk
to
>> >> v4, and most likely create a new dev branch-3 to introduce new
features
>> in
>> >> v3.4 and onwards. Would you mind starting the discussion in
>> [email protected]?
>> >> >
>> >> >> build workflow progress information in Oozie
>> >> >> --------------------------------------------
>> >> >>
>> >> >> Key: OOZIE-547
>> >> >> URL:
https://issues.apache.org/jira/browse/OOZIE-547
>> >> >> Project: Oozie
>> >> >> Issue Type: New Feature
>> >> >> Reporter: Hadoop QA
>> >> >> Assignee: zhu jin wei
>> >> >> Attachments: oozie-547.patch
>> >> >>
>> >> >>
>> >> >> For a user, knowing progress of her workflow is always desirable.
>> This
>> >> ticket is to introduce this support to Oozie.
>> >> >> I know it's a hard problem. For my initial effort, I plan to start
>> with
>> >> simple workflows that do not contain decision nodes or fork/join
nodes,
>> >> i.e., chain type workflows. I plan to use percentage of finished
>> actions as
>> >> the overall wf progress estimate.
>> >> >> Going forward we can improve the estimation by:
>> >> >> 1) handle general workflows that contain decision, fork/join nodes;
>> >> >> 2) incorporate the action level progress into wf level progress
>> >> estimation to make the estimate better. To be more specific:
>> >> >> In the case of "opaque" actions like pig/hive/jaql where the status
>> can
>> >> only be 0% or 100% (or failure) we plug that value into the overall
DAG
>> >> status of 0-100%. If a DAG had say 4 opaque actions, the progress
would
>> >> move in discrete steps 0, 25, 50, 75, 100%. For the m/r actions where
>> the
>> >> JobTracker
>> >> >> gives values between 0-100% for an action then the overall progress
>> >> will be smoother. We can do same thing for pig/hive/jaql actions as
>> well if
>> >> they expose their own progress info.
>> >> >
>> >> > --
>> >> > This message is automatically generated by JIRA.
>> >> > If you think it was sent incorrectly, please contact your JIRA
>> >> administrators
>> >> > For more information on JIRA, see:
>> >> http://www.atlassian.com/software/jira
>> >>
>>
>
>
>
> --
> Alejandro
