[ https://issues.apache.org/jira/browse/PIG-4913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15848538#comment-15848538 ]
Rohini Palaniswamy commented on PIG-4913: ----------------------------------------- My concern was same script was being recompiled multiple times. Reading from the file multiple times is just a side effect of that. The fix should be that we don't recompile again. > Reduce jython function initiation during compilation > ---------------------------------------------------- > > Key: PIG-4913 > URL: https://issues.apache.org/jira/browse/PIG-4913 > Project: Pig > Issue Type: Improvement > Reporter: Rohini Palaniswamy > Assignee: Adam Szita > Attachments: PIG-4913.patch > > > While investigating PIG-4908, saw that ScriptEngine.getScriptAsStream was > invoked way too many times during compilation phase for a simple script. > {code:title=sleep.py} > #!/usr/bin/python > import time; > @outputSchema("sltime:int") > def sleep(num): > if num == 1: > print "Sleeping for %d minutes" % num; > time.sleep(num * 60); > return num; > {code} > {code:title=sleep.pig} > register 'sleep.py' using jython; > A = LOAD '/tmp/sleepdata' as (f1:int); > B = FOREACH A generate $0, sleep($0); > STORE B into '/tmp/tezout'; > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)