[ https://issues.apache.org/jira/browse/PIG-5442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rohini Palaniswamy updated PIG-5442: ------------------------------------ Attachment: PIG-5442-1.patch > Add only credentials from setStoreLocation to the Job Conf > ---------------------------------------------------------- > > Key: PIG-5442 > URL: https://issues.apache.org/jira/browse/PIG-5442 > Project: Pig > Issue Type: Bug > Reporter: Alagappan Maruthappan > Assignee: Alagappan Maruthappan > Priority: Major > Attachments: PIG-5442-1.patch > > > While testing HCatStorer with Iceberg realized Pig calls setStoreLocation on > all Stores with the same Job object - > [https://github.com/apache/pig/blob/b050a33c66fc22d648370b5c6bda04e0e51d3aa3/src/org/apache/pig/backend/hadoop/executionengine/tez/TezDagBuilder.java#L1081] > Setting populated by one store is affecting the other stores. In my case the > "mapred.output.committer.class" is set as HiveIcebergCommitter by PigStore > that is used by the Iceberg table and the other stores which inserts data to > a non-iceberg tables also use that setting and trying to use > HiveIcebergCommitter. > > On checking with [~rohini] , it is called to get the credentials from all > stores since addCredentials API was added later and not all stores have > implemented it and some still set configuration in setLocation method (i.e, > HCatStorer). > > Fixed it by passing a separate copy of Job object to each store's setLocation > method and adding only the credential object from the call. -- This message was sent by Atlassian Jira (v8.20.10#820010)