[
https://issues.apache.org/jira/browse/PIG-4700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16037262#comment-16037262
]
Rohini Palaniswamy commented on PIG-4700:
-----------------------------------------
bq. Could you make it 1000? i.e schedule it every second
Actually it can be optimized better. Progress information is sent as part of
task heartbeat. In setups like ours heartbeat interval is set to 3 seconds so
that the AM can handle 10s of thousands of tasks. So we should set it to a
value slightly lower than heartbeat interval.
{code}
progressHelper.scheduleProgressTaskService(100, Math.max(1000,
(conf.getInt(TezConfiguration.TEZ_TASK_AM_HEARTBEAT_INTERVAL_MS,
TezConfiguration.TEZ_TASK_AM_HEARTBEAT_INTERVAL_MS_DEFAULT) - 50)));
{code}
> Enable progress reporting for Tasks in Tez
> ------------------------------------------
>
> Key: PIG-4700
> URL: https://issues.apache.org/jira/browse/PIG-4700
> Project: Pig
> Issue Type: Bug
> Reporter: Rohini Palaniswamy
> Assignee: Satish Subhashrao Saley
> Fix For: 0.18.0
>
> Attachments: PIG-4700-1.patch, PIG-4700-2.patch, PIG-4700-3.patch,
> PIG-4700-4.patch, PIG-4700-5.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
