[ https://issues.apache.org/jira/browse/PIG-953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Pradeep Kamath updated PIG-953: ------------------------------- Attachment: PIG-953-8.patch Fixed commit() code in PigOutputCommitter in multi store case to correctly set up the StoreConfig and StoreFunc in the Conf before calling commit() on the storefunc. > Enable merge join in pig to work with loaders and store functions which can > internally index sorted data > --------------------------------------------------------------------------------------------------------- > > Key: PIG-953 > URL: https://issues.apache.org/jira/browse/PIG-953 > Project: Pig > Issue Type: Improvement > Affects Versions: 0.3.0 > Reporter: Pradeep Kamath > Assignee: Pradeep Kamath > Attachments: PIG-953-2.patch, PIG-953-3.patch, PIG-953-4.patch, > PIG-953-5.patch, PIG-953-6.patch, PIG-953-7.patch, PIG-953-8.patch, > PIG-953.patch > > > Currently merge join implementation in pig includes construction of an index > on sorted data and use of that index to seek into the "right input" to > efficiently perform the join operation. Some loaders (notably the zebra > loader) internally implement an index on sorted data and can perform this > seek efficiently using their index. So the use of the index needs to be > abstracted in such a way that when the loader supports indexing, pig uses it > (indirectly through the loader) and does not construct an index. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.