Thanks again for the prompt response Charles. Following is the cluster stack:
1. Drill - v1.17 2. Hadoop - v3.1.2 3. Hive - v3.1.2 Please let me know if you need any further information. Regards, Ravi On Tue, Jul 14, 2020 at 1:44 PM Charles Givre <[email protected]> wrote: > Hi Ravi, > Gotcha! I just saw the error and immediately thought it was a host > resolution error. Let me look into that. Can you share what version(s) of > Drill, Hive and Hadoop you are using? > THanks, > -- C > > > > > On Jul 14, 2020, at 4:42 PM, ravi kanth <[email protected]> wrote: > > > > Hi Charles, > > > > Thanks for responding. > > > > *hadoop-qa* is the nameservice our HA enabled Hadoop cluster uses and > it's > > not technically a server to have an IP rather a logical name. So, to your > > question, unfortunately, I cannot change that to an IP. Although, if I > try > > to change that, Hive mappings in the metastore database for Hadoop paths > > are all mapped with Hadoop Nameservice and not IP. > > > > From Hadoop docs, > > > > - > > > > *dfs.nameservices* - the logical name for this new nameservice > > > > Choose a logical name for this nameservice, for example “mycluster”, > and > > use this logical name for the value of this config option. The name you > > choose is arbitrary. It will be used both for configuration and as the > > authority component of absolute HDFS paths in the cluster. > > > > *Note:* If you are also using HDFS Federation, this configuration > > setting should also include the list of other nameservices, HA or > > otherwise, as a comma-separated list. > > > > <property> > > <name>dfs.nameservices</name> > > <value>mycluster</value> > > </property> > > > > > > > > Is this something Drill support or tested? > > > > Thanks, > > Ravi > > > > > > On Tue, Jul 14, 2020 at 1:24 PM Charles Givre <[email protected]> wrote: > > > >> Could you try with the IP address of hadoop-qa and see if that works? > >> -- C > >> > >>> On Jul 14, 2020, at 4:22 PM, ravi kanth <[email protected]> wrote: > >>> > >>> Hi Team, > >>> > >>> I have a Drill cluster with 4 nodes. I wanted to make use of the Drill > >>> cluster to query a Hive Metastore Service which is built on top of HA > >>> enabled HDFS cluster. > >>> > >>> As the Hadoop cluster is HA enabled the paths for the Hive tables > >>> partitions are identified by a nameservice instead of a URL. > >>> > >>> In our case, HDFS Nameservice is hadoop-qa and so, are the HDFS > partition > >>> paths in Hive like "hdfs://hadoop-qa/dwh/data/tables/base_table/" > >>> > >>> Following is the storage-plugins-override.conf: > >>> > >>> "storage":{ > >>> hive: { > >>> type: "hive", > >>> configProps: { > >>> "hive.metastore.uris": "thrift://<host_name>:9083", > >>> "hive.metastore.warehouse.dir": > >>> "hdfs://hadoop-qa:8020/user/hive/warehouse", > >>> "fs.default.name": "hdfs://hadoop-qa:8020", > >>> "hive.metastore.sasl.enabled": "false", > >>> }, > >>> enabled: true > >>> } > >>> } > >>> > >>> When I try to query Hive using Drill, it fails to understand the > >>> nameservice *hadoop-qa* and throws the following exception: > >>> > >>> > >>> Error: SYSTEM ERROR: UnknownHostException: hadoop-qa > >>> > >>> Can someone suggest if the configurations are wrong? I followed the > ones > >> in > >>> the documentation. Any help will be greatly appreciated. > >>> > >>> *We are trying to evaluate Drill for our Production facing analytical > use > >>> cases and this is a blocker for us in order to proceed with Drill.* > >>> > >>> Thanks, > >>> Ravi > >> > >> > >
