Alagappan Maruthappan created PIG-5442: ------------------------------------------
Summary: Add only credentials from setStoreLocation to the Job Conf Key: PIG-5442 URL: https://issues.apache.org/jira/browse/PIG-5442 Project: Pig Issue Type: Bug Reporter: Alagappan Maruthappan While testing HCatStorer with Iceberg realized Pig calls setStoreLocation on all Stores with the same Job object - [https://github.com/apache/pig/blob/b050a33c66fc22d648370b5c6bda04e0e51d3aa3/src/org/apache/pig/backend/hadoop/executionengine/tez/TezDagBuilder.java#L1081] Setting populated by one store is affecting the other stores. In my case the "mapred.output.committer.class" is set as HiveIcebergCommitter by PigStore that is used by the Iceberg table and the other stores which inserts data to a non-iceberg tables also use that setting and trying to use HiveIcebergCommitter. On checking with [~rohini] , it is called to get the credentials from all stores since addCredentials API was added later and not all stores have implemented it and some still set configuration in setLocation method (i.e, HCatStorer). Fixed it by passing a separate copy of Job object to each store's setLocation method and adding only the credential object from the call. -- This message was sent by Atlassian Jira (v8.20.10#820010)