[ https://issues.apache.org/jira/browse/PIG-4814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Work on PIG-4814 started by Daniel Dai. --------------------------------------- > AvroStorage does not take namenode HA as part of schema file url > ---------------------------------------------------------------- > > Key: PIG-4814 > URL: https://issues.apache.org/jira/browse/PIG-4814 > Project: Pig > Issue Type: Bug > Reporter: Daniel Dai > Assignee: Daniel Dai > Fix For: 0.16.0 > > Attachments: PIG-4814-1.patch > > > The following script fail: > {code} > in = LOAD 'recordsOfStringArrays.avro' USING AvroStorage('', '-d -f > hdfs://ml-ha/user/aeckstein/notower-rw/streamRunwaysInputSchema.avsc'); > {code} > Where ml-ha is the configured HA name for namenode. > The reason is AvroStorage creates a new Configuration object in the backend, > which missing the namenode HA configuration in hdfs-site.xml: > {code} > <property> > <name>dfs.client.failover.proxy.provider.ml-ha</name> > > <value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value> > </property> > {code} > It should use the configuration passed from frontend rather than construct a > new one. -- This message was sent by Atlassian JIRA (v6.3.4#6332)