I see your point Min. Yes this is possible however only the metadata calls to the HiveServer should emanate from Hive.java. An execute should not call an execute etc... but it is possible in deployments where the HiveServer is its own clients and if I am correct thrift stubs should be able to deal with this and not make a network connection.
However, the ideal deployment would have a HiveServer in front of a metastore server (which is different). A cluster of HiveServers front a single metastore server. We would want the clustering on HiveServers as they serve out the data to clients talking jdbc. So in that deployment (which we think would make sense if a deployment is large scale and hit by many users), the metadata calls made though Hive.java (in turn called though the execute call in HiveServer) would just go to a different metastore server. Ashish ________________________________________ From: Min Zhou [[email protected]] Sent: Wednesday, May 13, 2009 8:21 PM To: [email protected] Subject: Re: Is it possible hiveserver both be a server and a client of itself? HiveServer does call methods in Hive.java when compiling and planing a query command. HiveServer talk to itself through Hive.java and a socket connection. This is the logic of the current hive trunk code. On Thu, May 14, 2009 at 11:16 AM, Raghu Murthy <[email protected]<mailto:[email protected]>> wrote: Hive.java does not call methods in HiveServer. Can you explain what you are trying to achieve? You should just be able to start the HiveServer and then create a HiveClient to invoke methods of both HiveServer itself and HiveMetaStore. On 5/13/09 8:07 PM, "Min Zhou" <[email protected]<mailto:[email protected]>> wrote: > Hey Raghu, > > Agreed with you. My question is , Hive.java code called by HiveServer, and > Hive.java would talked to HiveServer through a socket connection. Am I right? > > Min > On Thu, May 14, 2009 at 11:02 AM, Raghu Murthy > <[email protected]<mailto:[email protected]>> wrote: >> We have used thrift's inheritance feature where one service can extend >> another service, i.e., HiveServer extends HiveMetaStore. This means that >> 1. HiveServer exposes HiveMetaStore methods >> 2. HiveClient can be used to invoke methods exposed by both HiveServer and >> HiveMetaStore. >> >> Does that make sense? >> >> >> On 5/13/09 7:52 PM, "Min Zhou" >> <[email protected]<mailto:[email protected]>> wrote: >> >>>> Sorry, I didn't precisely express my meaning. I didn't talk about a >>>> supposition, but talked about a possible phenomenon may happen at hive. >>>> The hiveserver may both be a server and a client of itself, you can look >>>> >> at >>>> the definition of HiveServer >>>> >>>> public static class HiveServerHandler extends HiveMetaStore.HMSHandler ... >>>> { >>>> ... >>>> } >>>> >>>> >>>> >>>> It extends HiveMetaStore.HMSHandler, which provide the same function of >>>> metastore . >>>> >>>> public class HiveMetaStore extends ThriftHiveMetastore { >>>> >>>> public static class HMSHandler extends FacebookBase implements >>>> ThriftHiveMetastore.Iface { >>>> // omit >>>> } >>>> >>>> // main entry of the MetaStore but didn't be called at this case >>>> public static void main(String[] args) { >>>> ... >>>> } >>>> } >>>> >>>> there is no another standalone metastore server if you have up a >>> hiveserver, >>>> which is also a metaserver itself. >>>> >>>> Min >>>> On Thu, May 14, 2009 at 10:35 AM, Amr Awadallah >>>> <[email protected]<mailto:[email protected]>> wrote: >>>>>> >>>>>> >>>>>> I don't think that would work, where would you store the meta-data for >>>>>> >>> the >>>>>> meta-data? >>>>>> >>>>>> >>>>>> Min Zhou wrote: >>>>>>>> Hi Ashish, >>>>>>>> Thank you for your swift reply. I guess it's HiveServer code inherit >>>>>>>> the >>>>>>>> metastore thrift api. HiveServer provide metadata service by itself, >>>>>>>> not >>>>>>>> call another metastore server, and Hive.class talks to the HiveServer >>>>>>>> to >>>>>>>> get >>>>>>>> the metadata. >>>>>>>> Regards, >>>>>>>> Min >>>>>>>> >>>>>>>> >>>>>>>> On Thu, May 14, 2009 at 9:19 AM, Ashish Thusoo >>>>>>>> <[email protected]<mailto:[email protected]>> >>>>> wrote: >>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Apologies for the delay Min.. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> The HiveServer code calls the metastore thrift api to get the >>>>>> metadata. The >>>>>>>>>> Hive.class in ql also talks to the metastore api to get the metadata. >>>>>>>>>> Not >>>>>>>>>> sure if I follow your question though? >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Ashish >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> From: Min Zhou >>>>>>>>>> [mailto:[email protected]<mailto:[email protected]>] >>>>>>>>>> Sent: Wednesday, May 13, 2009 6:07 PM >>>>>>>>>> To: hive-user >>>>>>>>>> Subject: Re: Is it possible hiveserver both be a server and a client >>>>>>>>>> of >>>>>>>>>> itself? >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Is there anyone can answer my question? >>>>>>>>>> Thanks in advanced! >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Sun, May 10, 2009 at 12:00 AM, Min Zhou >>>>>>>>>> <[email protected]<mailto:[email protected]>> >>>>>>>>>> >>>>> wrote: >>>>>>>>>> >>>>>>>>>>>> Hi guys, >>>>>>>>>>>> >>>>>>>>>>>> I found that HiveServer can fetch metadata thourgh Hive class after >>>>>>>>>>>> generated a task like DDLTask, and meanwhile Hive class is a client >>>>>>>>>>>> who ask HiveServer for metadata. So I guess it may be possible >>>>>>>>>>>> hiveserver both be a server and a client of itself, am I right? >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Min >>>>>>>>>>>> -- >>>>>>>>>>>> My research interests are distributed systems, parallel computing >>>>>>>>>>>> and >>>>>>>>>>>> bytecode based virtual machine. >>>>>>>>>>>> >>>>>>>>>>>> My profile: >>>>>>>>>>>> http://www.linkedin.com/in/coderplay >>>>>>>>>>>> My blog: >>>>>>>>>>>> http://coderplay.javaeye.com >>>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> My research interests are distributed systems, parallel computing and >>>>>>>>>> bytecode based virtual machine. >>>>>>>>>> >>>>>>>>>> My profile: >>>>>>>>>> http://www.linkedin.com/in/coderplay >>>>>>>>>> My blog: >>>>>>>>>> http://coderplay.javaeye.com >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> My research interests are distributed systems, parallel computing and >>>>>>>> bytecode based virtual machine. >>>>>>>> >>>>>>>> My profile: >>>>>>>> http://www.linkedin.com/in/coderplay >>>>>>>> My blog: >>>>>>>> http://coderplay.javaeye.com >>>> >>>> >> > > -- My research interests are distributed systems, parallel computing and bytecode based virtual machine. My profile: http://www.linkedin.com/in/coderplay My blog: http://coderplay.javaeye.com
