[
https://issues.apache.org/jira/browse/FALCON-1565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15065304#comment-15065304
]
Praveen Adlakha commented on FALCON-1565:
-----------------------------------------
Hi All,
I have submitted the patch for this one and have done the required testing also.
In distributed mode:
{code}
dataqa@blr-test-129:/usr/local/falcon/falcon-distributed-0.9-SNAPSHOT$
bin/falcon instance -type feed -name feed1 -start 2026-10-09T18:00Z -listing
ERROR: Bad Request;local/Specified End date 2015-12-19T07:21Z is before the
entity was scheduled 2026-10-09T18:00Z
feed definition:
dataqa@blr-test-129:/usr/local/falcon/falcon-distributed-0.9-SNAPSHOT$
bin/falcon entity -type feed -name feed1 -definition
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<feed name="feed1" description="Input File" xmlns="uri:falcon:feed:0.1">
<frequency>minutes(5)</frequency>
<timezone>UTC</timezone>
<late-arrival cut-off="minutes(2)"/>
<clusters>
<cluster name="local" type="source">
<validity start="2015-10-09T14:00Z" end="2025-10-09T18:00Z"/>
<retention limit="days(1000000)" action="delete"/>
</cluster>
</clusters>
<locations>
<location type="data"
path="/user/dataqa/testEffective/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}"/>
</locations>
<ACL owner="dataqa" group="dataqa" permission="*"/>
<schema location="none" provider="none"/>
<properties>
<property name="field1" value="value1"/>
<property name="field2" value="value2"/>
</properties>
</feed>
Process definition
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<process name="DWH-CI-daily-supply" xmlns="uri:falcon:process:0.1">
<clusters>
<cluster name="local">
<validity start="2015-12-20T01:00Z" end="2015-12-21T00:00Z"/>
</cluster>
</clusters>
<parallel>1</parallel>
<order>FIFO</order>
<frequency>days(1)</frequency>
<timezone>UTC</timezone>
<inputs>
<input name="feed1" feed="feed1" start="today(-1,0)"
end="today(-1,0)"/>
</inputs>
<properties>
<property name="queueName" value="supply"/>
<property name="jobPriority" value="LOW"/>
<property name="date" value="${formatTime(dateOffset(instanceTime(),
-1, 'DAY'), 'yyyy-MM-dd')}"/>
<property name="dbUploadHostName" value="glgw4006.grid.uh1.inmobi.com"/>
<property name="dbuploadscript"
value="/home/supply/tar//daily/daily-tar-job.sh"/>
<property name="hdfspath" value="/projects/supply/data"/>
<property name="tarpath" value="opt/mkhoj/var/sibyl/supply"/>
</properties>
<workflow path="/projects/ivory/working/workflow.xml"
lib="/projects/ivory/working/lib/"/>
<retry policy="periodic" delay="minutes(2)" attempts="3"/>
</process>
dataqa@blr-test-129:/usr/local/falcon/falcon-distributed-0.9-SNAPSHOT$ date
Sat Dec 19 13:11:02 IST 2015
dataqa@blr-test-129:/usr/local/falcon/falcon-distributed-0.9-SNAPSHOT$
bin/falcon entity -type process -name DWH-CI-daily-supply -status
local/RUNNING
{code}
I am not sure what should be the output in case of process please verify and
confirm for the same.
In Embedded mode:
{code}
dataqa@blr-test-129:/usr/local/falcon/falcon-0.9-SNAPSHOT$ bin/falcon entity
-type feed -name feed1 -definition
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<feed name="feed1" description="Input File" xmlns="uri:falcon:feed:0.1">
<frequency>minutes(5)</frequency>
<timezone>UTC</timezone>
<late-arrival cut-off="minutes(2)"/>
<clusters>
<cluster name="local" type="source">
<validity start="2015-10-09T14:00Z" end="2025-10-09T18:00Z"/>
<retention limit="days(1000000)" action="delete"/>
</cluster>
</clusters>
<locations>
<location type="data"
path="/user/dataqa/testEffective/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}"/>
</locations>
<ACL owner="dataqa" group="dataqa" permission="*"/>
<schema location="none" provider="none"/>
<properties>
<property name="field1" value="value1"/>
<property name="field2" value="value2"/>
</properties>
</feed>
Process definition
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<process name="DWH-CI-daily-supply" xmlns="uri:falcon:process:0.1">
<clusters>
<cluster name="local">
<validity start="2015-12-20T01:00Z" end="2015-12-21T00:00Z"/>
</cluster>
</clusters>
<parallel>1</parallel>
<order>FIFO</order>
<frequency>days(1)</frequency>
<timezone>UTC</timezone>
<inputs>
<input name="feed1" feed="feed1" start="today(-1,0)"
end="today(-1,0)"/>
</inputs>
<properties>
<property name="queueName" value="supply"/>
<property name="jobPriority" value="LOW"/>
<property name="date" value="${formatTime(dateOffset(instanceTime(),
-1, 'DAY'), 'yyyy-MM-dd')}"/>
<property name="dbUploadHostName" value="glgw4006.grid.uh1.inmobi.com"/>
<property name="dbuploadscript"
value="/home/supply/tar//daily/daily-tar-job.sh"/>
<property name="hdfspath" value="/projects/supply/data"/>
<property name="tarpath" value="opt/mkhoj/var/sibyl/supply"/>
</properties>
<workflow path="/projects/ivory/working/workflow.xml"
lib="/projects/ivory/working/lib/"/>
<retry policy="periodic" delay="minutes(2)" attempts="3"/>
</process>
dataqa@blr-test-129:/usr/local/falcon/falcon-0.9-SNAPSHOT$ date
Sat Dec 19 14:27:57 IST 2015
dataqa@blr-test-129:/usr/local/falcon/falcon-0.9-SNAPSHOT$ bin/falcon entity
-type process -name DWH-CI-daily-supply -status
default/RUNNING
{code}
> Listing API non-intuitive response if time > endTime
> ----------------------------------------------------
>
> Key: FALCON-1565
> URL: https://issues.apache.org/jira/browse/FALCON-1565
> Project: Falcon
> Issue Type: Bug
> Affects Versions: 0.8
> Environment: QA
> Reporter: Pragya Mittal
> Assignee: Praveen Adlakha
> Labels: newbie
> Fix For: 0.9
>
> Attachments: FALCON-1565.patch
>
>
> While listing for an entity with time > endTime, response shows stack trace.
> Instead error mesage should be shown as response.
> Feed definition is :
> {noformat}
> <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
> <feed name="sla-feed" description="clicks log" xmlns="uri:falcon:feed:0.1">
> <partitions>
> <partition name="country"/>
> <partition name="colo"/>
> </partitions>
> <frequency>minutes(2)</frequency>
> <sla slaLow="minutes(1)" slaHigh="minutes(3)"/>
> <timezone>UTC</timezone>
> <late-arrival cut-off="hours(6)"/>
> <clusters>
> <cluster name="A9e7e0672-e02bcc65">
> <validity start="2015-10-28T11:58Z" end="2015-10-30T11:17Z"/>
> <retention limit="days(1000000)" action="delete"/>
> </cluster>
> </clusters>
> <locations>
> <location type="data"
> path="/tmp/falcon-regression/FeedSlaMonitoring/input/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}"/>
> <location type="stats" path="/projects/falcon/clicksStats"/>
> <location type="meta" path="/projects/falcon/clicksMetaData"/>
> </locations>
> <ACL owner="pragya" group="dataqa" permission="*"/>
> <schema location="/schema/clicks" provider="protobuf"/>
> <properties>
> <property name="field1" value="value1"/>
> </properties>
> </feed>
> {noformat}
> Listing API response
> {noformat}
> dataqa@lda01:/mnt/users/pragya/defn/sla$ falcon instance -type feed -name
> sla-feed -start 2015-10-30T11:20Z -listing
> ERROR: Bad Request;<?xml version="1.0" encoding="UTF-8"
> standalone="yes"?><feedInstanceResult><status>FAILED</status><message>ua1/org.apache.falcon.FalconException::javax.ws.rs.WebApplicationException:
> javax.xml.bind.UnmarshalException: unexpected element (uri:"",
> local:"instancesResult"). Expected elements are
> <{}feedInstanceResult>,<{}instance>,<{}result>
> {noformat}
> Stack trace is :
> {noformat}
> 2015-10-28 12:15:54,414 ERROR - [1963200284@qtp-2030538903-5 -
> 0f02eeb7-1f02-4bea-bbb7-85d5e61b568f:dataqa:GET//instance/listing/feed/sla-feed]
> ~ Failed to get instances listing (AbstractInstanceManager:528)
> org.apache.falcon.FalconException: Specified End date 2015-10-28T12:15Z is
> before the entity was scheduled 2015-10-30T11:20Z
> at
> org.apache.falcon.resource.AbstractInstanceManager.getStartAndEndDate(AbstractInstanceManager.java:853)
> at
> org.apache.falcon.resource.AbstractInstanceManager.getStartAndEndDate(AbstractInstanceManager.java:842)
> at
> org.apache.falcon.resource.AbstractInstanceManager.getListing(AbstractInstanceManager.java:524)
> at
> org.apache.falcon.resource.InstanceManager.getListing(InstanceManager.java:141)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
> at
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
> at
> com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
