[ https://issues.apache.org/jira/browse/TEZ-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14146322#comment-14146322 ]
Krisztian Horvath commented on TEZ-1608: ---------------------------------------- In the second version every Sum processor maintains a local top K and only writes these values when the task finishes, so the writer only has to select the top K from the local top Ks. The implementation is a bit more complex than the previous one. > TopK example > ------------ > > Key: TEZ-1608 > URL: https://issues.apache.org/jira/browse/TEZ-1608 > Project: Apache Tez > Issue Type: Sub-task > Affects Versions: 0.5.0 > Reporter: Janos Matyas > Attachments: TEZ-1608-1.patch, TEZ-1608-2.patch > > > The goal of this sample is to find the topK elements of a dataset, while > guiding through the basics of Tez (DAG creation, tokenizers, custom > comparators and parallelism). > An example use case for top K: > Given a large data set in CSV format of user comments on a site listed as: > userid,postid,commentid,comment,timestamp and we are looking for the top K > commenter or the posts with the most comment. -- This message was sent by Atlassian JIRA (v6.3.4#6332)