[
https://issues.apache.org/jira/browse/TEZ-3391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16496968#comment-16496968
]
Rohini Palaniswamy commented on TEZ-3391:
-----------------------------------------
bq. 1) Moving the validation checks to AM
Yo can skip this. Looking at the code, it is not a easy thing as the AM will
have to deconstruct it to MRInput and then perform the check. It would be
wasteful to do that just for this purpose. The validation check is better done
by clients like Pig which create the file. They can do it even before
submitting the DAG which is even more better.
> MR split file validation should be done in the AM
> -------------------------------------------------
>
> Key: TEZ-3391
> URL: https://issues.apache.org/jira/browse/TEZ-3391
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Rohini Palaniswamy
> Assignee: Nishant Dash
> Priority: Major
>
> We had a case where Split metadata size exceeded 10000000. Instead of job
> failing from validation during initialization in AM like mapreduce, each of
> the tasks failed doing that validation during initialization.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
