[
https://issues.apache.org/jira/browse/PIG-934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated PIG-934:
---------------------------------
Attachment: pig-934.patch
I think instead of creating a new method, better is to overload
FileLocalizer.open() to have a consistent api.
Currently, there is
{code}
InputStream open(String fileSpec, PigContext pigContext)
{code}
Add an Overload method like this and within that fallback to previous method,
if there is no need to seek.
{code}
InputStream open(String fileSpec, long offset, PigContext pigContext){
if (offset == 0)
return open(String fileSpec, PigContext pigContext);
else
// first open, then seek and then return
}
{code}
This ensures minimum impact to clients of this api.
Attaching a patch which does this.
> Merge join implementation currently does not seek to right point on the right
> side input based on the offset provided by the index
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: PIG-934
> URL: https://issues.apache.org/jira/browse/PIG-934
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.3.1
> Reporter: Pradeep Kamath
> Attachments: pig-934.patch
>
>
> We use POLoad to seek into right file which has the following code:
> {noformat}
> public void setUp() throws IOException{
> String filename = lFile.getFileName();
> loader =
> (LoadFunc)PigContext.instantiateFuncFromSpec(lFile.getFuncSpec());
> is = FileLocalizer.open(filename, pc);
> loader.bindTo(filename , new BufferedPositionedInputStream(is),
> this.offset, Long.MAX_VALUE);
> }
> {noformat}
> Between opening the stream and bindTo we do not seek to the right offset.
> bindTo itself does not perform any seek.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.