[
https://issues.apache.org/jira/browse/PIG-4814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Dai updated PIG-4814:
----------------------------
Attachment: PIG-4814-1.patch
It might be hard to come out a test cuz the reproduction involve namenode HA
setting.
> AvroStorage does not take namenode HA as part of schema file url
> ----------------------------------------------------------------
>
> Key: PIG-4814
> URL: https://issues.apache.org/jira/browse/PIG-4814
> Project: Pig
> Issue Type: Bug
> Reporter: Daniel Dai
> Assignee: Daniel Dai
> Fix For: 0.16.0
>
> Attachments: PIG-4814-1.patch
>
>
> The following script fail:
> {code}
> in = LOAD 'recordsOfStringArrays.avro' USING AvroStorage('', '-d -f
> hdfs://ml-ha/user/aeckstein/notower-rw/streamRunwaysInputSchema.avsc');
> {code}
> Where ml-ha is the configured HA name for namenode.
> The reason is AvroStorage creates a new Configuration object in the backend,
> which missing the namenode HA configuration in hdfs-site.xml:
> {code}
> <property>
> <name>dfs.client.failover.proxy.provider.ml-ha</name>
>
> <value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value>
> </property>
> {code}
> It should use the configuration passed from frontend rather than construct a
> new one.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
