[
https://issues.apache.org/jira/browse/PIG-3453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13777343#comment-13777343
]
Brian ONeill commented on PIG-3453:
-----------------------------------
First question, for DISTINCT within Storm, do you believe we should have a
sliding time window within which we perform the distinct? There is mention of
the fact that it will be stateful (since we need to keep a set in memory with
which to de-dupe). Do we intend to leverage the concept of Trident State for
this? (which may make sense, implement State then on each commit/flush perform
the de-duping)
thoughts?
> Implement a Storm backend to Pig
> --------------------------------
>
> Key: PIG-3453
> URL: https://issues.apache.org/jira/browse/PIG-3453
> Project: Pig
> Issue Type: New Feature
> Reporter: Pradeep Gollakota
> Labels: storm
>
> There is a lot of interest around implementing a Storm backend to Pig for
> streaming processing. The proposal and initial discussions can be found at
> https://cwiki.apache.org/confluence/display/PIG/Pig+on+Storm+Proposal
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
