[ https://issues.apache.org/jira/browse/PIG-3204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rohini Palaniswamy updated PIG-3204: ------------------------------------ Attachment: PIG-3204-2.patch [~cheolsoo] You are right. validateQuery is not required. Attaching PIG-3204-2.patch with suggested changes. > Reduce the number of getSchema calls during script parsing > ---------------------------------------------------------- > > Key: PIG-3204 > URL: https://issues.apache.org/jira/browse/PIG-3204 > Project: Pig > Issue Type: Improvement > Affects Versions: 0.10.1 > Reporter: Rohini Palaniswamy > Assignee: Rohini Palaniswamy > Fix For: 0.12 > > Attachments: PIG-3204-1.patch, PIG-3204-2.patch > > > Currently there are a lot of NN calls made to determine if there is a > schema file for a path in a LOAD statement. When there is a slow NN(caused by > whole bunch of other issues), it takes a lot of time for this and we found > the scripts spending anywhere from 5 mins to 40 mins depending upon the > script. It seems to be a good place for optimization. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira