Re: Thrift Server
Mark, JDBC does not talk to hive via thrift server. I shut down the thrift server, JDBC still works. However, you might be right in the case of non-local connection, or as they call "standalone mode". Van On Thu, Jul 5, 2012 at 8:47 PM, Mark Grover wrote: > Hi Ransom, > JDBC talks to the Hive driver via the Hive Thrift server. Since the thrift > server is limited to single connection at a time, out of curiosity, how > much benefit do concurrent connections through JDBC give you, given that > they will also be handled serially when being sent to the Thrift server? > > Thanks in advance! > Mark > > - Original Message - > From: "Hezhiqiang (Ransom)" > To: user@hive.apache.org > Cc: "Zhaojun (Terry)" > Sent: Friday, June 29, 2012 9:31:01 PM > Subject: RE: Thrift Server > > Hi Van > In my test, JDBC supported about 25 connections in the same time without > delay. > If more connections , it will line up and timeout exception will happen. > > > Regards > Ransom > > > > -Original Message- > From: VanHuy Pham [mailto:huy.pham...@gmail.com] > Sent: Saturday, June 30, 2012 8:59 AM > To: user@hive.apache.org > Subject: Re: Thrift Server > > Thanks for the response. > I see. Would JDBC then be a better option for concurrent connections? > I am not aware of the implementation of hive-JDBC so wonder if it > support multiple connections? > Any idea? > > On 6/29/12, Hezhiqiang (Ransom) wrote: > > Hi Van > > Hive doesn’t supported multiple connections now. > > it's impossible for HiveServer to support concurrent connections > > using the current Thrift API > > you can see: > > https://issues.apache.org/jira/browse/HIVE-2935 > > > https://cwiki.apache.org/confluence/download/attachments/27362054/HiveServer2HadoopSummit2012BoF.pdf?version=1&modificationDate=1339790767000 > > > > > > From: VanHuy Pham [mailto:huy.pham...@gmail.com] > > Sent: Saturday, June 30, 2012 4:33 AM > > To: user@hive.apache.org > > Subject: Thrift Server > > > > Hi hive folks, > > Does hive thrift server support multiple requests from clients at the > > same time? > > It looks like the server serves the request sequentially, which > means it > > processes each request one by one. Am I wrong here? > > I make two clients, which make two requests (select data) to two > > different in hive; judging by the terminal screen of the hive server, it > > processes one request, finishes it, and process the other. > > > > Van > > >
Re: Thrift Server
Hi Ransom, JDBC talks to the Hive driver via the Hive Thrift server. Since the thrift server is limited to single connection at a time, out of curiosity, how much benefit do concurrent connections through JDBC give you, given that they will also be handled serially when being sent to the Thrift server? Thanks in advance! Mark - Original Message - From: "Hezhiqiang (Ransom)" To: user@hive.apache.org Cc: "Zhaojun (Terry)" Sent: Friday, June 29, 2012 9:31:01 PM Subject: RE: Thrift Server Hi Van In my test, JDBC supported about 25 connections in the same time without delay. If more connections , it will line up and timeout exception will happen. Regards Ransom -Original Message- From: VanHuy Pham [mailto:huy.pham...@gmail.com] Sent: Saturday, June 30, 2012 8:59 AM To: user@hive.apache.org Subject: Re: Thrift Server Thanks for the response. I see. Would JDBC then be a better option for concurrent connections? I am not aware of the implementation of hive-JDBC so wonder if it support multiple connections? Any idea? On 6/29/12, Hezhiqiang (Ransom) wrote: > Hi Van > Hive doesn’t supported multiple connections now. > it's impossible for HiveServer to support concurrent connections > using the current Thrift API > you can see: > https://issues.apache.org/jira/browse/HIVE-2935 > https://cwiki.apache.org/confluence/download/attachments/27362054/HiveServer2HadoopSummit2012BoF.pdf?version=1&modificationDate=1339790767000 > > > From: VanHuy Pham [mailto:huy.pham...@gmail.com] > Sent: Saturday, June 30, 2012 4:33 AM > To: user@hive.apache.org > Subject: Thrift Server > > Hi hive folks, > Does hive thrift server support multiple requests from clients at the > same time? > It looks like the server serves the request sequentially, which means it > processes each request one by one. Am I wrong here? > I make two clients, which make two requests (select data) to two > different in hive; judging by the terminal screen of the hive server, it > processes one request, finishes it, and process the other. > > Van >
RE: Thrift Server
Hi Van In my test, JDBC supported about 25 connections in the same time without delay. If more connections , it will line up and timeout exception will happen. Regards Ransom -Original Message- From: VanHuy Pham [mailto:huy.pham...@gmail.com] Sent: Saturday, June 30, 2012 8:59 AM To: user@hive.apache.org Subject: Re: Thrift Server Thanks for the response. I see. Would JDBC then be a better option for concurrent connections? I am not aware of the implementation of hive-JDBC so wonder if it support multiple connections? Any idea? On 6/29/12, Hezhiqiang (Ransom) wrote: > Hi Van > Hive doesn’t supported multiple connections now. > it's impossible for HiveServer to support concurrent connections > using the current Thrift API > you can see: > https://issues.apache.org/jira/browse/HIVE-2935 > https://cwiki.apache.org/confluence/download/attachments/27362054/HiveServer2HadoopSummit2012BoF.pdf?version=1&modificationDate=1339790767000 > > > From: VanHuy Pham [mailto:huy.pham...@gmail.com] > Sent: Saturday, June 30, 2012 4:33 AM > To: user@hive.apache.org > Subject: Thrift Server > > Hi hive folks, > Does hive thrift server support multiple requests from clients at the > same time? > It looks like the server serves the request sequentially, which means it > processes each request one by one. Am I wrong here? > I make two clients, which make two requests (select data) to two > different in hive; judging by the terminal screen of the hive server, it > processes one request, finishes it, and process the other. > > Van >
Re: Thrift Server
Thanks for the response. I see. Would JDBC then be a better option for concurrent connections? I am not aware of the implementation of hive-JDBC so wonder if it support multiple connections? Any idea? On 6/29/12, Hezhiqiang (Ransom) wrote: > Hi Van > Hive doesn’t supported multiple connections now. > it's impossible for HiveServer to support concurrent connections > using the current Thrift API > you can see: > https://issues.apache.org/jira/browse/HIVE-2935 > https://cwiki.apache.org/confluence/download/attachments/27362054/HiveServer2HadoopSummit2012BoF.pdf?version=1&modificationDate=1339790767000 > > > From: VanHuy Pham [mailto:huy.pham...@gmail.com] > Sent: Saturday, June 30, 2012 4:33 AM > To: user@hive.apache.org > Subject: Thrift Server > > Hi hive folks, > Does hive thrift server support multiple requests from clients at the > same time? > It looks like the server serves the request sequentially, which means it > processes each request one by one. Am I wrong here? > I make two clients, which make two requests (select data) to two > different in hive; judging by the terminal screen of the hive server, it > processes one request, finishes it, and process the other. > > Van >
RE: Thrift Server
Hi Van Hive doesn’t supported multiple connections now. it's impossible for HiveServer to support concurrent connections using the current Thrift API you can see: https://issues.apache.org/jira/browse/HIVE-2935 https://cwiki.apache.org/confluence/download/attachments/27362054/HiveServer2HadoopSummit2012BoF.pdf?version=1&modificationDate=1339790767000 From: VanHuy Pham [mailto:huy.pham...@gmail.com] Sent: Saturday, June 30, 2012 4:33 AM To: user@hive.apache.org Subject: Thrift Server Hi hive folks, Does hive thrift server support multiple requests from clients at the same time? It looks like the server serves the request sequentially, which means it processes each request one by one. Am I wrong here? I make two clients, which make two requests (select data) to two different in hive; judging by the terminal screen of the hive server, it processes one request, finishes it, and process the other. Van
Re: Thrift Server not starting with Hive 0.8.1 and hadoop-1.0.1
Hello Shashwat Thanks for your reply Infact yes , its started :) I thought i would say some message in console Hi , Thrift has been started :) Anyways my bad didnt not check it . Thanks Edward and to you also Shashwat :) - Original Message - From: shashwat shriparv Sent: 04/05/12 10:24 PM To: user@hive.apache.org Subject: Re: Thrift Server not starting with Hive 0.8.1 and hadoop-1.0.1 It does not hangs rather it is started, just open another shell and type netstat -nl | grep 1 if something shows up means the server is running, and also in new terminal windows type bin/hive and the start createing your creating tables. On Thu, Apr 5, 2012 at 10:18 PM, Shin Chan < had...@gmx.com > wrote: Hello Edward Thank You for your reply I have dowloaded hadoop 0.20 version. Setup hadoop for pesudo mode exported HADOOP_HOME , HIVE_HOME variables Now when in start thrift server it still hangs $hive --service hiveserver -verbose Starting Hive Thrift Server Starting hive server on port 1 with 100 min worker threads and 2147483647 max worker threads Do I need to do any setting on HIVE side? Thanks once again. - Original Message - From: Edward Capriolo Sent: 04/05/12 09:34 PM To: user@hive.apache.org Subject: Re: Thrift Server not starting with Hive 0.8.1 and hadoop-1.0.1 In a nutshell the hive community is trying to catch up with all the versions of hadoop. 0.20.205, 1.0, 0.23.1, I have lost track myself. Every time I consider a major like 0.20.205 I find it does not work with hive. Then I watch issues open up to get hive to work with 0.20.205, before those issues close you get word that the new gold standard is 1.0 or 0.23.1 , or whatever. I still run hadoop 0.20.2. Works great! Less filling! On Thu, Apr 5, 2012 at 11:44 AM, Shin Chan < had...@gmx.com > wrote: > Hello All > > I am trying to start Thrift server on HIVE > > My configuration for Hadoop is Pseudo mode > > When i say > > hive --service hiveserver > > It just prints > > Starting Hive Thrift Server > WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please > use org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties > files. > > And then nothing comes > > Can you guide me steps for configuring HIVE Thrift service with Hadoop > > Thanks in advance :) > > Thanks and Regards , -- ∞ Shashwat Shriparv Thanks and Regards ,
Re: Thrift Server not starting with Hive 0.8.1 and hadoop-1.0.1
Hi Shin, Not sure what do you mean by hangs here. I think this is about what the server would output on proper startup. (it is at least for me. :)) Did you try invoking hive sever using hive jdbc client [1] or equivalent? If not can you try and see whether it works. Regards Buddhika [1] https://cwiki.apache.org/Hive/hiveclient.html On Thu, Apr 5, 2012 at 10:48 PM, Shin Chan wrote: > Hello Edward > > Thank You for your reply > > I have dowloaded hadoop 0.20 version. > > Setup hadoop for pesudo mode > exported HADOOP_HOME , HIVE_HOME variables > > Now when in start thrift server it still hangs > > $hive --service hiveserver -verbose > Starting Hive Thrift Server > Starting hive server on port 1 with 100 min worker threads and > 2147483647 max worker threads > > Do I need to do any setting on HIVE side? > > Thanks once again. > > > > - Original Message - > > From: Edward Capriolo > > Sent: 04/05/12 09:34 PM > > To: user@hive.apache.org > > Subject: Re: Thrift Server not starting with Hive 0.8.1 and hadoop-1.0.1 > > > In a nutshell the hive community is trying to catch up with all the > versions of hadoop. 0.20.205, 1.0, 0.23.1, I have lost track myself. > Every time I consider a major like 0.20.205 I find it does not work > with hive. Then I watch issues open up to get hive to work with > 0.20.205, before those issues close you get word that the new gold > standard is 1.0 or 0.23.1 , or whatever. > > I still run hadoop 0.20.2. Works great! Less filling! > > On Thu, Apr 5, 2012 at 11:44 AM, Shin Chan wrote: > > Hello All > > > > I am trying to start Thrift server on HIVE > > > > My configuration for Hadoop is Pseudo mode > > > > When i say > > > > hive --service hiveserver > > > > It just prints > > > > Starting Hive Thrift Server > > WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please > > use org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties > > files. > > > > And then nothing comes > > > > Can you guide me steps for configuring HIVE Thrift service with Hadoop > > > > Thanks in advance :) > > > > > > > > > > Thanks and Regards ,
Re: Thrift Server not starting with Hive 0.8.1 and hadoop-1.0.1
It does not hangs rather it is started, just open another shell and type netstat -nl | grep 1 if something shows up means the server is running, and also in new terminal windows type bin/hive and the start createing your creating tables. On Thu, Apr 5, 2012 at 10:18 PM, Shin Chan wrote: > Hello Edward > > Thank You for your reply > > I have dowloaded hadoop 0.20 version. > > Setup hadoop for pesudo mode > exported HADOOP_HOME , HIVE_HOME variables > > Now when in start thrift server it still hangs > > $hive --service hiveserver -verbose > Starting Hive Thrift Server > Starting hive server on port 1 with 100 min worker threads and > 2147483647 max worker threads > > Do I need to do any setting on HIVE side? > > Thanks once again. > > > > - Original Message - > > From: Edward Capriolo > > Sent: 04/05/12 09:34 PM > > To: user@hive.apache.org > > Subject: Re: Thrift Server not starting with Hive 0.8.1 and hadoop-1.0.1 > > > In a nutshell the hive community is trying to catch up with all the > versions of hadoop. 0.20.205, 1.0, 0.23.1, I have lost track myself. > Every time I consider a major like 0.20.205 I find it does not work > with hive. Then I watch issues open up to get hive to work with > 0.20.205, before those issues close you get word that the new gold > standard is 1.0 or 0.23.1 , or whatever. > > I still run hadoop 0.20.2. Works great! Less filling! > > On Thu, Apr 5, 2012 at 11:44 AM, Shin Chan wrote: > > Hello All > > > > I am trying to start Thrift server on HIVE > > > > My configuration for Hadoop is Pseudo mode > > > > When i say > > > > hive --service hiveserver > > > > It just prints > > > > Starting Hive Thrift Server > > WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please > > use org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties > > files. > > > > And then nothing comes > > > > Can you guide me steps for configuring HIVE Thrift service with Hadoop > > > > Thanks in advance :) > > > > > > > > > > Thanks and Regards , -- ∞ Shashwat Shriparv
Re: Thrift Server not starting with Hive 0.8.1 and hadoop-1.0.1
Hello Edward Thank You for your reply I have dowloaded hadoop 0.20 version. Setup hadoop for pesudo mode exported HADOOP_HOME , HIVE_HOME variables Now when in start thrift server it still hangs $hive --service hiveserver -verbose Starting Hive Thrift Server Starting hive server on port 1 with 100 min worker threads and 2147483647 max worker threads Do I need to do any setting on HIVE side? Thanks once again. - Original Message - From: Edward Capriolo Sent: 04/05/12 09:34 PM To: user@hive.apache.org Subject: Re: Thrift Server not starting with Hive 0.8.1 and hadoop-1.0.1 In a nutshell the hive community is trying to catch up with all the versions of hadoop. 0.20.205, 1.0, 0.23.1, I have lost track myself. Every time I consider a major like 0.20.205 I find it does not work with hive. Then I watch issues open up to get hive to work with 0.20.205, before those issues close you get word that the new gold standard is 1.0 or 0.23.1 , or whatever. I still run hadoop 0.20.2. Works great! Less filling! On Thu, Apr 5, 2012 at 11:44 AM, Shin Chan wrote: > Hello All > > I am trying to start Thrift server on HIVE > > My configuration for Hadoop is Pseudo mode > > When i say > > hive --service hiveserver > > It just prints > > Starting Hive Thrift Server > WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please > use org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties > files. > > And then nothing comes > > Can you guide me steps for configuring HIVE Thrift service with Hadoop > > Thanks in advance :) > > Thanks and Regards ,
Re: Thrift Server not starting with Hive 0.8.1 and hadoop-1.0.1
In a nutshell the hive community is trying to catch up with all the versions of hadoop. 0.20.205, 1.0, 0.23.1, I have lost track myself. Every time I consider a major like 0.20.205 I find it does not work with hive. Then I watch issues open up to get hive to work with 0.20.205, before those issues close you get word that the new gold standard is 1.0 or 0.23.1 , or whatever. I still run hadoop 0.20.2. Works great! Less filling! On Thu, Apr 5, 2012 at 11:44 AM, Shin Chan wrote: > Hello All > > I am trying to start Thrift server on HIVE > > My configuration for Hadoop is Pseudo mode > > When i say > > hive --service hiveserver > > It just prints > > Starting Hive Thrift Server > WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please > use org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties > files. > > And then nothing comes > > Can you guide me steps for configuring HIVE Thrift service with Hadoop > > Thanks in advance :) > >
