[
https://issues.apache.org/jira/browse/FALCON-2030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15337964#comment-15337964
]
Ajay Yadava commented on FALCON-2030:
-------------------------------------
Hello [~me.venkatr],
Feed doesn't have any restriction that there should be a consumer. What if
there are no consuming processes? May be the user just uses falcon for periodic
cleaning of a directory on Hadoop. In other scenario the user may maintain only
one copy of the metadata and uses it as a feed for ad-hoc analysis. May be the
users have aperiodic data and maintains versions of instances but not in
YYYY-MM-DD format.
The larger point is that retention today is possible for any particular type of
data - periodic/aperiodic, versioned/non-versioned and users don't need to
model their data locations in a particular pattern in order to onboard falcon.
This is a great advantage. Forcing a particular pattern in data location is
restrictive IMO and the lack of this validation shouldn't cause any issues.
Hope it helps.
> Enforce time partition pattern in the data location path in feed definition
> ----------------------------------------------------------------------------
>
> Key: FALCON-2030
> URL: https://issues.apache.org/jira/browse/FALCON-2030
> Project: Falcon
> Issue Type: Improvement
> Components: feed
> Reporter: Venkatesan Ramachandran
> Assignee: Venkatesan Ramachandran
>
> In feed definition, data location can be specified without time series
> pattern like below:
> <locations>
> <location type="data"
> path="/tmp/falcon-regression/RetentionTest/testFolders/"/>
> <location type="stats" path="/projects/falcon/clicksStats"/>
> <location type="meta" path="/projects/falcon/clicksMetaData"/>
> </locations>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
