[ https://issues.apache.org/jira/browse/PIG-3047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13509411#comment-13509411 ]
Jonathan Coveney commented on PIG-3047: --------------------------------------- Prashant: that sounds good to me. Just make it configurable (as you proposed) and have those configuration keys in the PigConfiguration class and it should be good. > Check the size of a relation before adding it to distributed cache in > Replicated join > ------------------------------------------------------------------------------------- > > Key: PIG-3047 > URL: https://issues.apache.org/jira/browse/PIG-3047 > Project: Pig > Issue Type: Improvement > Reporter: Julien Le Dem > > Right now if someone makes a mistake and put the large relation last, Pig > will copy a huge file into distributed cache and it will take a long time > before the job eventually fails. It would be better to check before copying > the relation that it is of reasonable size. > <1 GB ? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira