----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/20006/#review40421 -----------------------------------------------------------
Ship it! Ship It! - Rohini Palaniswamy On April 14, 2014, 9:26 p.m., Purshotam Shah wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/20006/ > ----------------------------------------------------------- > > (Updated April 14, 2014, 9:26 p.m.) > > > Review request for oozie, Ryota Egashira and Rohini Palaniswamy. > > > Bugs: OOZIE-1769 > https://issues.apache.org/jira/browse/OOZIE-1769 > > > Repository: oozie-git > > > Description > ------- > > To change the coord definition user can update coord definition in hdfs and > issue a update command. Existing coordinator definition will be replaced by > new definition. The refreshed coordinator > would keep the same coordinator ID, state, and coordinator actions. > User can also provide -dryrun to validate changes. > All created coord action(including in waiting) will use old configuration. > User can rerun actions with -refresh option, -refresh option will use new > configuration to rerun coord action. > Ex. > 1. $ oozie job -update <coordID> -config > examples/apps/aggregator/job.properties > Will update coord definition and action. > 2. $ oozie job -update <coordID> > Will fetch coord definition path from bundle and update coord definition > > > Diffs > ----- > > client/src/main/java/org/apache/oozie/cli/OozieCLI.java e1f551d > client/src/main/java/org/apache/oozie/client/OozieClient.java 46c4288 > client/src/main/java/org/apache/oozie/client/rest/JsonTags.java ae391c7 > client/src/main/java/org/apache/oozie/client/rest/RestConstants.java > a7fe06e > core/pom.xml f0547a3 > core/src/main/java/org/apache/oozie/CoordinatorEngine.java 3f10024 > core/src/main/java/org/apache/oozie/CoordinatorJobBean.java 5eb134b > core/src/main/java/org/apache/oozie/ErrorCode.java f69d7a2 > core/src/main/java/org/apache/oozie/command/coord/CoordSubmitXCommand.java > 9e6a3d5 > core/src/main/java/org/apache/oozie/command/coord/CoordUpdateXCommand.java > e69de29 > core/src/main/java/org/apache/oozie/executor/jpa/CoordJobQueryExecutor.java > 5f7744c > core/src/main/java/org/apache/oozie/servlet/BaseJobServlet.java aa43e68 > core/src/main/java/org/apache/oozie/servlet/V0JobServlet.java 443ab6d > core/src/main/java/org/apache/oozie/servlet/V1JobServlet.java ac399e9 > core/src/main/java/org/apache/oozie/servlet/V2JobServlet.java 2d26599 > > core/src/test/java/org/apache/oozie/command/coord/TestCoordUpdateXCommand.java > e69de29 > core/src/test/resources/coord-update-test.xml e69de29 > docs/src/site/twiki/DG_CommandLineTool.twiki 351f0f2 > docs/src/site/twiki/WebServicesAPI.twiki 37c0bc0 > pom.xml cb10007 > > Diff: https://reviews.apache.org/r/20006/diff/ > > > Testing > ------- > > Invalid xml. > purushah$ > /Users/purushah/views/coor_swap/oozie/distro/target/oozie-4.1.0-SNAPSHOT-distro/oozie-4.1.0-SNAPSHOT/bin/oozie > job -oozie http://localhost:11000/oozie -config temp/job.properties -update > 0000005-140402104721140-oozie-puru-C -dryrun > Picked up JAVA_TOOL_OPTIONS: -Djava.awt.headless=true > Error: E1023 : E1023: Coord Job update Error: [E0701: XML schema error, > cvc-complex-type.2.4.a: Invalid content was found starting with element > 'datasets'. One of '{"uri:oozie:coordinator:0.2":execution, > "uri:oozie:coordinator:0.2":throttle}' is expected.] > purushah$ > > > valid XML. > $ oozie job -oozie http://localhost:11000/oozie -config temp/job.properties > -update 0000005-140402104721140-oozie-puru-C -dryrun > **********Job definition changes********** > @@ -3,8 +3,8 @@ > <concurrency>1</concurrency> > </controls> > <input-events> > - <data-in name="input" dataset="raw-logs"> > - <dataset name="raw-logs" frequency="20" > initial-instance="2010-01-01T00:00Z" timezone="UTC" freq_timeunit="MINUTE" > end_of_duration="NONE"> > + <data-in name="input" dataset="raw-logs-rename"> > + <dataset name="raw-logs-rename" frequency="20" > initial-instance="2010-01-01T00:00Z" timezone="UTC" freq_timeunit="MINUTE" > end_of_duration="NONE"> > > <uri-template>hdfs://localhost:9000/user/purushah/examples/input-data/rawLogs/</uri-template> > <done-flag /> > </dataset> > ********************************** > **********Job conf changes********** > @@ -8,10 +8,6 @@ > > <value>hdfs://localhost:9000/user/purushah/examples/apps/aggregator/coordinator.xml</value> > </property> > <property> > - <name>aaaaaa</name> > - <value>b</value> > - </property> > - <property> > <name>user.name</name> > <value>purushah</value> > </property> > @@ -28,6 +24,10 @@ > <value>hdfs://localhost:9000</value> > </property> > <property> > + <name>adding</name> > + <value>new</value> > + </property> > + <property> > <name>jobTracker</name> > <value>localhost:9001</value> > </property> > ********************************** > $ > > $ > /Users/purushah/views/coor_swap/oozie/distro/target/oozie-4.1.0-SNAPSHOT-distro/oozie-4.1.0-SNAPSHOT/bin/oozie > job -oozie http://localhost:11000/oozie -config temp/job.properties -update > 0000000-140408140947673-oozie-puru-C > Picked up JAVA_TOOL_OPTIONS: -Djava.awt.headless=true > Error: E0803 : E0803: IO error, E1023: Coord Job update Error: [Frequency > can't be changes. Old frqenecy 60 new frequency 180] > $ > > Tested Doc......... > DG_CommandLineTool.html#Updating_coordinator_definition_and_properties > > Updating coordinator definition and properties > > Existing coordinator definition will be replaced by new definition. The > refreshed coordinator would keep the same coordinator ID, state, and > coordinator actions. All created coord action(including in WAITING) will use > old configuration. One can rerun actions with -refresh option, -refresh > option will use new configuration to rerun coord action > > Update command also verifies coordinator definition like submit command, if > there is any issue with definition, update will fail. Update command with > -dryrun will show coordinator definition and properties differences. Config > option is optional, if not specified existing coordinator property is used to > find coordinator path. > > Update command doesn't update frequency, end time, start time, timzone, name, > execution order. Update command will fail if any of this element has changed > in job definition. > > $ oozie job -oozie http://localhost:11000/oozie -config job.properties > -update 0000005-140402104721140-oozie-puru-C -dryrun**********Job definition > changes********** > @@ -3,8 +3,8 @@ > <concurrency>1</concurrency> > </controls> > <input-events> > - <data-in name="input" dataset="raw-logs"> > - <dataset name="raw-logs" frequency="20" > initial-instance="2010-01-01T00:00Z" timezone="UTC" freq_timeunit="MINUTE" > end_of_duration="NONE"> > + <data-in name="input" dataset="raw-logs-rename"> > + <dataset name="raw-logs-rename" frequency="20" > initial-instance="2010-01-01T00:00Z" timezone="UTC" freq_timeunit="MINUTE" > end_of_duration="NONE"> > > <uri-template>hdfs://localhost:9000/user/purushah/examples/input-data/rawLogs/</uri-template> > <done-flag /> > </dataset> > ********************************** > **********Job conf changes********** > @@ -8,10 +8,6 @@ > > <value>hdfs://localhost:9000/user/purushah/examples/apps/aggregator/coordinator.xml</value> > </property> > <property> > - <name>old</name> > - <value>test</value> > - </property> > - <property> > <name>user.name</name> > <value>purushah</value> > </property> > @@ -28,6 +24,10 @@ > <value>hdfs://localhost:9000</value> > </property> > <property> > + <name>adding</name> > + <value>new</value> > + </property> > + <property> > <name>jobTracker</name> > <value>localhost:9001</value> > </property> > ********************************** > > > WebServicesAPI.html#Updating_coordinator_definition_and_properties > > Existing coordinator definition and properties will be replaced by new > definition and properties. Refer Updating coordinator definition and > properties > > PUT oozie/v2/job/0000000-140414102048137-oozie-puru-C?action=update > Response: > > HTTP/1.1 200 OK > Content-Type: application/json;charset=UTF-8 > {"update": > {"diff":"**********Job definition > changes**********\n******************************************\n**********Job > conf changes****************\n@@ -8,16 +8,12 @@\n > > <value>hdfs:\/\/localhost:9000\/user\/purushah\/examples\/apps\/aggregator\/coordinator.xml<\/value>\r\n > <\/property>\r\n <property>\r\n > - <name>user.name<\/name>\r\n > - <value>purushah<\/value>\r\n > - <\/property>\r\n > - <property>\r\n <name>start<\/name>\r\n > <value>2010-01-01T01:00Z<\/value>\r\n <\/property>\r\n > <property>\r\n > - <name>newproperty<\/name>\r\n > - <value>new<\/value>\r\n > + <name>user.name<\/name>\r\n > + <value>purushah<\/value>\r\n <\/property>\r\n > <property>\r\n > > <name>queueName<\/name>\r\n******************************************\n" > } > } > > > Thanks, > > Purshotam Shah > >
