AppId is missing in the latest hive version (3.1.2)
Hi Folks, I just found that the yarn appId is missing in the beeline log when using hive version (3.1.2) This is very inconvenience for problem shooting. what is the consideration to remove that ? Could you add that back to the beeline log ? -- Best Regards Jeff Zhang
no mr-job execution info in beeline
In beeline, I could not see the job execution info (like job progress), I have already set the following properties in hive-site.xml. Could anyone help to figure out how to diagnose such issues ? How can I check whether hive server2 take the correct configuration ? hive.server2.logging.operation.level VERBOSE I only see the following log in beeline WARNING: Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases. -- Best Regards Jeff Zhang
Does hive metastore service support proxy user access ?
I try to access hive metastore service using proxy user, but didn't succeed. I just wonder whether hive metastore support this kind of access ? 16/03/15 08:57:57 DEBUG security.UserGroupInformation: PrivilegedAction as:jeff (auth:PROXY) via l...@example.com (auth:KERBEROS) from:org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.open(TUGIAssumingTransport.java:49) 16/03/15 08:57:57 ERROR transport.TSaslTransport: SASL negotiation failure javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)] at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:212) at org.apache.thrift.transport.TSaslClientTransport.handleSaslStartMessage(TSaslClientTransport.java:94) at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:271) at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37) at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52) at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.open(TUGIAssumingTransport.java:49) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:420) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:236) at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.(SessionHiveMetaStoreClient.java:74) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:526) at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.(RetryingMetaStoreClient.java:86) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104) at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005) at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024) at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234) at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174) at org.apache.hadoop.hive.ql.metadata.Hive.(Hive.java:166) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) -- Best Regards Jeff Zhang
Re: [ANNOUNCE] New Hive Committer - Wei Zheng
Congratulations, Wei ! On Thu, Mar 10, 2016 at 3:27 PM, Lefty Leverenz wrote: > Congratulations! > > -- Lefty > > On Wed, Mar 9, 2016 at 10:30 PM, Dmitry Tolpeko > wrote: > >> Congratulations, Wei! >> >> On Thu, Mar 10, 2016 at 5:48 AM, Chao Sun wrote: >> >>> Congratulations! >>> >>> On Wed, Mar 9, 2016 at 6:44 PM, Prasanth Jayachandran < >>> pjayachand...@hortonworks.com> wrote: >>> >>>> Congratulations Wei! >>>> >>>> On Mar 9, 2016, at 8:43 PM, Sergey Shelukhin >>> <mailto:ser...@hortonworks.com>> wrote: >>>> >>>> Congrats! >>>> >>>> From: Szehon Ho mailto:sze...@cloudera.com>> >>>> Reply-To: "user@hive.apache.org<mailto:user@hive.apache.org>" < >>>> user@hive.apache.org<mailto:user@hive.apache.org>> >>>> Date: Wednesday, March 9, 2016 at 17:40 >>>> To: "user@hive.apache.org<mailto:user@hive.apache.org>" < >>>> user@hive.apache.org<mailto:user@hive.apache.org>> >>>> Cc: "d...@hive.apache.org<mailto:d...@hive.apache.org>" < >>>> d...@hive.apache.org<mailto:d...@hive.apache.org>>, "w...@apache.org >>>> <mailto:w...@apache.org>" mailto:w...@apache.org>> >>>> Subject: Re: [ANNOUNCE] New Hive Committer - Wei Zheng >>>> >>>> Congratulations Wei! >>>> >>>> On Wed, Mar 9, 2016 at 5:26 PM, Vikram Dixit K >>> <mailto:vik...@apache.org>> wrote: >>>> The Apache Hive PMC has voted to make Wei Zheng a committer on the >>>> Apache Hive Project. Please join me in congratulating Wei. >>>> >>>> Thanks >>>> Vikram. >>>> >>>> >>>> >>> >> > -- Best Regards Jeff Zhang
Fwd: Wrong column is picked in HIVE 2.0.0 + TEZ 0.8.2 left join
+ hive mail list, more likely hive issue. -- Forwarded message -- From: GAO Chi Date: Tue, Mar 1, 2016 at 12:24 PM Subject: Wrong column is picked in HIVE 2.0.0 + TEZ 0.8.2 left join To: u...@tez.apache.org Hi all, We encountered a strange behavior after upgrading to HIVE 2.0.0 + TEZ 0.8.2. I simplified our query to this: SELECT a.key, a.a_one, b.b_one, a.a_zero, b.b_zero FROM ( SELECT 11 key, 0 confuse_you, 1 a_one, 0 a_zero ) a LEFT JOIN ( SELECT 11 key, 0 confuse_you, 1 b_one, 0 b_zero ) b ON a.key = b.key ; Above query generates this unexpected result: INFO : Status: Running (Executing on YARN cluster with App id application_1456723490535_3653) INFO : Map 1: 0/1 Map 2: 0/1 INFO : Map 1: 0/1 Map 2: 0(+1)/1 INFO : Map 1: 0(+1)/1 Map 2: 0(+1)/1 INFO : Map 1: 0(+1)/1 Map 2: 1/1 INFO : Map 1: 1/1 Map 2: 1/1 INFO : Completed executing command(queryId=hive_20160301115630_0a0dbee5-ba4b-45e7-b027-085f655640fd); Time taken: 10.225 seconds INFO : OK ++--+--+---+---+--+ | a.key | a.a_one | b.b_one | a.a_zero | b.b_zero | ++--+--+---+---+--+ | 11 | 1| 0| 0 | 1 | ++--+--+---+---+--+ If you change the constant value of subquery-b’s confuse_you column from 0 to 2, the problem disappears. The plan returned from EXPLAIN shows the incorrect one is picking _col1 and _col2, while the correct one is picking _col2 and _col3 form sub query b. Seems it cannot distinguish 2 columns with same constant value? Anyone encountered similar problem? Thanks! Chi -- Best Regards Jeff Zhang
Re: Getting dot files for DAGs
Hi James, It is under the working directory of the yarn container (should be the first container which is AM) Best Regard, Jeff Zhang From: James Pirz mailto:james.p...@gmail.com>> Reply-To: "u...@tez.apache.org<mailto:u...@tez.apache.org>" mailto:u...@tez.apache.org>> Date: Thursday, October 1, 2015 at 8:29 AM To: "u...@tez.apache.org<mailto:u...@tez.apache.org>" mailto:u...@tez.apache.org>> Cc: "user@hive.apache.org<mailto:user@hive.apache.org>" mailto:user@hive.apache.org>> Subject: Getting dot files for DAGs I am using Tez 0.7.0 on Hadopp 2.6 to run Hive queries. I am interested in checking DAGs for my queries visually, and I realized that I can do that by graphviz once I can get "dot" files of my DAGs. My issue is I can not find those files, they are not in the log directory of Yarn or Hadoop or under /tmp . Any hint as where I can find those files would be great. Do I need to add any settings to my tez-site.xml in-order to enable generating them ? Thanks.
Bucketing is not leveraged in filter push down ?
I have one table which is bucketed on column name. Then I have the following sql: - select count(1) from student_bucketed_2 where name = 'calvin nixon'; Ideally I think it should only scan one bucket. But from the log I still see it will scan all the bucket files. Why bucketing is not leveraged in filter push down ? Here's the log: 2015-09-24 14:59:22,282 INFO [TezChild] io.HiveContextAwareRecordReader: Processing file hdfs:// 0.0.0.0:9000/user/hive/warehouse/student_bucketed_2/age=26/01_0 2015-09-24 14:59:22,282 INFO [TezChild] exec.Utilities: PLAN PATH = hdfs:// 0.0.0.0:9000/tmp/hive/jzhang/1ddc9a11-28f8-45e5-aa12-aa457fc9530d/hive_2015-09-24_14-59-04_435_7446484965928171550-1/-mr-10004/b2224650-d31a-4599-8606-57681287bd41/map.xml 2015-09-24 14:59:22,289 INFO [TezChild] io.HiveContextAwareRecordReader: Processing file hdfs:// 0.0.0.0:9000/user/hive/warehouse/student_bucketed_2/age=26/02_0 2015-09-24 14:59:22,289 INFO [TezChild] exec.Utilities: PLAN PATH = hdfs:// 0.0.0.0:9000/tmp/hive/jzhang/1ddc9a11-28f8-45e5-aa12-aa457fc9530d/hive_2015-09-24_14-59-04_435_7446484965928171550-1/-mr-10004/b2224650-d31a-4599-8606-57681287bd41/map.xml 2015-09-24 14:59:22,296 INFO [TezChild] io.HiveContextAwareRecordReader: Processing file hdfs:// 0.0.0.0:9000/user/hive/warehouse/student_bucketed_2/age=26/03_0 2015-09-24 14:59:22,296 INFO [TezChild] exec.Utilities: PLAN PATH = hdfs:// 0.0.0.0:9000/tmp/hive/jzhang/1ddc9a11-28f8-45e5-aa12-aa457fc9530d/hive_2015-09-24_14-59-04_435_7446484965928171550-1/-mr-10004/b2224650-d31a-4599-8606-57681287bd41/map.xml 2015-09-24 14:59:22,304 INFO [TezChild] io.HiveContextAwareRecordReader: Processing file hdfs:// 0.0.0.0:9000/user/hive/warehouse/student_bucketed_2/age=26/04_0 2015-09-24 14:59:22,304 INFO [TezChild] exec.Utilities: PLAN PATH = hdfs:// 0.0.0.0:9000/tmp/hive/jzhang/1ddc9a11-28f8-45e5-aa12-aa457fc9530d/hive_2015-09-24_14-59-04_435_7446484965928171550-1/-mr-10004/b2224650-d31a-4599-8606-57681287bd41/map.xml 2015-09-24 14:59:22,311 INFO [TezChild] io.HiveContextAwareRecordReader: Processing file hdfs:// 0.0.0.0:9000/user/hive/warehouse/student_bucketed_2/age=26/05_0 -- Best Regards Jeff Zhang
Re: Hive Compile mode
Use explain https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Explain On Thu, Sep 10, 2015 at 11:07 AM, Raajay wrote: > Is it possible to use Hive only in compile mode ( and not execute the > queries) ? > > The output here would be a DAG say to be executed on TEZ later. > > Thanks > Raajay > -- Best Regards Jeff Zhang
Re: Getting error while performing Insert query
What is your hive sql ? And please check the hive.log which may have more info.
(By default hive.log is located in /tmp/${user}/hive.log )
Best Regard,
Jeff Zhang
From: Sateesh Karuturi
mailto:sateesh.karutu...@gmail.com>>
Reply-To: "u...@tez.apache.org<mailto:u...@tez.apache.org>"
mailto:u...@tez.apache.org>>
Date: Wednesday, September 9, 2015 at 1:39 PM
To: "user@hive.apache.org<mailto:user@hive.apache.org>"
mailto:user@hive.apache.org>>,
"u...@tez.apache.org<mailto:u...@tez.apache.org>"
mailto:u...@tez.apache.org>>
Subject: Getting error while performing Insert query
hello...,
iam using hive 1.1 and tez 0.7...
Whenever iam trying to INSERT data into hive table using tez via java iam
getting following error:
Exception in thread "main" org.apache.hive.service.cli.HiveSQLException: Error
while compiling statement: FAILED: SemanticException [Error 10293]: Unable to
create temp file for insert values Expression of type TOK_TABLE_OR_COL not
supported in insert/values
Re: Is YSmart integrated into Hive on tez ?
+ dev mail list
The original correlation optimization might be designed for mr engine. But
similar optimization could be applied for tez too. Is there any existing
jira to track that ?
On Tue, Sep 1, 2015 at 1:58 PM, Jeff Zhang wrote:
> Hi Pengcheng,
>
> Is there reason why the correlation optimization disabled in tez ?
>
> And even when I change the code to enable the correlation optimization in
> tez. I still get the same query plan.
>
> >>> Vertex dependency in root stage
> >>> Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
> >>> Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
>
> On Tue, Sep 1, 2015 at 1:14 AM, Pengcheng Xiong wrote:
>
>> Hi Jeff,
>>
>> From code base point of view, YSmart is integrated into Hive on Tez
>> because it is one of the optimization of the current Hive. However, from
>> the execution point of view, it is now disabled when Hive is running on
>> Tez. You may take look at the source code of Hive
>>
>> Optimizer.java, L175-180:
>> {code}
>>
>> if(HiveConf.getBoolVar(hiveConf, HiveConf.ConfVars.HIVEOPTCORRELATION) &&
>>
>> !HiveConf.getBoolVar(hiveConf, HiveConf.ConfVars.HIVEGROUPBYSKEW)
>> &&
>>
>> !HiveConf.getBoolVar(hiveConf, HiveConf.ConfVars.
>> HIVE_OPTIMIZE_SKEWJOIN_COMPILETIME) &&
>>
>> !isTezExecEngine) {
>>
>> transformations.add(new CorrelationOptimizer());
>>
>> }
>> {code}
>>
>> Hope it helps.
>>
>> Best
>> Pengcheng Xiong
>>
>>
>> On Mon, Aug 31, 2015 at 12:56 AM, Jeff Zhang wrote:
>>
>>> The reason why I ask this question is that when I execute the following
>>> sql, it will generated a query plan with 4 vertices. But as my
>>> understanding if YSmart is integrated into hive, it should only take 3
>>> vertices since the join key and group by key are the same. Anybody know
>>> this ? Thanks
>>>
>>>
>>> >> insert overwrite directory '/tmp/jzhang/1' select o.o_orderkey as
>>> orderkey,count(1) from lineitem l >> join orders o on
>>> l.l_orderkey=o.o_orderkey group by o.o_orderkey;
>>>
>>> *YSmart Hive Jira*
>>>
>>> https://issues.apache.org/jira/browse/HIVE-2206
>>>
>>>
>>>
>>>
>>> --
>>> Best Regards
>>>
>>> Jeff Zhang
>>>
>>
>>
>
>
> --
> Best Regards
>
> Jeff Zhang
>
--
Best Regards
Jeff Zhang
Re: Is YSmart integrated into Hive on tez ?
Hi Pengcheng,
Is there reason why the correlation optimization disabled in tez ?
And even when I change the code to enable the correlation optimization in
tez. I still get the same query plan.
>>> Vertex dependency in root stage
>>> Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
>>> Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
On Tue, Sep 1, 2015 at 1:14 AM, Pengcheng Xiong wrote:
> Hi Jeff,
>
> From code base point of view, YSmart is integrated into Hive on Tez
> because it is one of the optimization of the current Hive. However, from
> the execution point of view, it is now disabled when Hive is running on
> Tez. You may take look at the source code of Hive
>
> Optimizer.java, L175-180:
> {code}
>
> if(HiveConf.getBoolVar(hiveConf, HiveConf.ConfVars.HIVEOPTCORRELATION) &&
>
> !HiveConf.getBoolVar(hiveConf, HiveConf.ConfVars.HIVEGROUPBYSKEW)
> &&
>
> !HiveConf.getBoolVar(hiveConf, HiveConf.ConfVars.
> HIVE_OPTIMIZE_SKEWJOIN_COMPILETIME) &&
>
> !isTezExecEngine) {
>
> transformations.add(new CorrelationOptimizer());
>
> }
> {code}
>
> Hope it helps.
>
> Best
> Pengcheng Xiong
>
>
> On Mon, Aug 31, 2015 at 12:56 AM, Jeff Zhang wrote:
>
>> The reason why I ask this question is that when I execute the following
>> sql, it will generated a query plan with 4 vertices. But as my
>> understanding if YSmart is integrated into hive, it should only take 3
>> vertices since the join key and group by key are the same. Anybody know
>> this ? Thanks
>>
>>
>> >> insert overwrite directory '/tmp/jzhang/1' select o.o_orderkey as
>> orderkey,count(1) from lineitem l >> join orders o on
>> l.l_orderkey=o.o_orderkey group by o.o_orderkey;
>>
>> *YSmart Hive Jira*
>>
>> https://issues.apache.org/jira/browse/HIVE-2206
>>
>>
>>
>>
>> --
>> Best Regards
>>
>> Jeff Zhang
>>
>
>
--
Best Regards
Jeff Zhang
Re: Hive on tez error
Please refer this wiki page https://cwiki.apache.org/confluence/display/TEZ/How+to+Diagnose+Tez+App Best Regard, Jeff Zhang From: Sateesh Karuturi mailto:sateesh.karutu...@gmail.com>> Reply-To: "u...@tez.apache.org<mailto:u...@tez.apache.org>" mailto:u...@tez.apache.org>> Date: Friday, August 28, 2015 at 12:33 AM To: "user@hive.apache.org<mailto:user@hive.apache.org>" mailto:user@hive.apache.org>>, "u...@tez.apache.org<mailto:u...@tez.apache.org>" mailto:u...@tez.apache.org>> Subject: Hive on tez error I am trying to connect hive database(execution.engine value changed to tez) using Java code... In case of select query its working But in the case of INSERT getting an error: The error looks like. Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask Please help me out
Re: tez error
+ hive mail list Might be a hive bug. Best Regard, Jeff Zhang From: "hanked...@emar.com<mailto:hanked...@emar.com>" mailto:hanked...@emar.com>> Reply-To: user mailto:u...@tez.apache.org>> Date: Thursday, August 27, 2015 at 5:51 PM To: user mailto:u...@tez.apache.org>> Subject: tez error Hi ~all I build tez-0.7.0 with 2.6.0-cdh5.4.0, deploy and run demo wordcount success. but, hive on tez problems. error infomation: Status: Running (Executing on YARN cluster with App id application_1440664563598_0005) VERTICES STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED Map 1 FAILED -1 00 -1 0 0 Reducer 2 KILLED 1 001 0 0 VERTICES: 00/02 [>>--] 0%ELAPSED TIME: 0.24 s Status: Failed Vertex failed, vertexName=Map 1, vertexId=vertex_1440664563598_0005_1_00, diagnostics=[Vertex vertex_1440664563598_0005_1_00 [Map 1] killed/failed due to:ROOT_INPUT_INIT_FAILURE, Vertex Input: test initializer failed, vertex=vertex_1440664563598_0005_1_00 [Map 1], java.lang.IllegalArgumentException: Illegal Capacity: -12185 at java.util.ArrayList.(ArrayList.java:156) at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:330) at org.apache.hadoop.hive.ql.io.HiveInputFormat.addSplitsForGroup(HiveInputFormat.java:306) at org.apache.hadoop.hive.ql.io.HiveInputFormat.getSplits(HiveInputFormat.java:408) at org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.initialize(HiveSplitGenerator.java:129) at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:245) at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:239) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671) at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:239) at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:226) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) ] Vertex killed, vertexName=Reducer 2, vertexId=vertex_1440664563598_0005_1_01, diagnostics=[Vertex received Kill in INITED state., Vertex vertex_1440664563598_0005_1_01 [Reducer 2] killed/failed due to:null] DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:1 FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask hanked...@emar.com<mailto:hanked...@emar.com>
Re: execution error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.teztask error on hive on tez
Please check the hive log for more useful info which is located in
/tmp/${user}/hive.log by default.
Best Regard,
Jeff Zhang
From: Sateesh Karuturi
mailto:sateesh.karutu...@gmail.com>>
Reply-To: "u...@tez.apache.org<mailto:u...@tez.apache.org>"
mailto:u...@tez.apache.org>>
Date: Wednesday, July 29, 2015 at 5:54 PM
To: "user@hive.apache.org<mailto:user@hive.apache.org>"
mailto:user@hive.apache.org>>,
"u...@tez.apache.org<mailto:u...@tez.apache.org>"
mailto:u...@tez.apache.org>>
Subject: execution error, return code 1 from
org.apache.hadoop.hive.ql.exec.tez.teztask error on hive on tez
iam using hive 1.0 and tez 0.7 whenever iam performing insert query its returns
following error:
execution error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.teztask
Re: multiple users for hive access
Have you tried to start hive cli using these 2 users ? What issue did you see ? On Wed, Jul 8, 2015 at 11:50 AM, Jack Yang wrote: > Thanks, mate. I have mysql run as my metadata store. > > > > What is the next step? when I start hive (0.13 version), I just type in > hive in my command line. > > > > Now, that is say I have two users: A and B. I would like A and B access > hive tables using hive-cli. > > > > How can I do that? > > > > > > *From:* Jeff Zhang [mailto:zjf...@gmail.com] > *Sent:* Tuesday, 7 July 2015 4:22 PM > *To:* user@hive.apache.org > *Subject:* Re: multiple users for hive access > > > > Hive support multiple user scenario as long as the metadata store support > multiple user access. By default hive use derby embedded mode which don't > support multiple user access. You can configure it to server mode or use > other metadata store like mysql etc. Here's the tutorial for how to > configure derby server mode > > > > https://cwiki.apache.org/confluence/display/Hive/HiveDerbyServerMode > > > > > > > > On Tue, Jul 7, 2015 at 1:50 PM, Jack Yang wrote: > > Hi all, > > > > I would like to have multiple users to access hive. > > Does anyone try that before? > > Is there any tutorial or link I can study from? > > > > Best regards, > > Jack > > > > > > > > -- > > Best Regards > > Jeff Zhang > -- Best Regards Jeff Zhang
Re: multiple users for hive access
Hive support multiple user scenario as long as the metadata store support multiple user access. By default hive use derby embedded mode which don't support multiple user access. You can configure it to server mode or use other metadata store like mysql etc. Here's the tutorial for how to configure derby server mode https://cwiki.apache.org/confluence/display/Hive/HiveDerbyServerMode On Tue, Jul 7, 2015 at 1:50 PM, Jack Yang wrote: > Hi all, > > > > I would like to have multiple users to access hive. > > Does anyone try that before? > > Is there any tutorial or link I can study from? > > > > Best regards, > > Jack > > > -- Best Regards Jeff Zhang
Re: Hive With tez
Regarding the mapper task number, Hive on tez is very similar with Hive on MapReduce. One difference is that hive on tez can group split together which may use less tasks than mapreduce. What issues did you see when you use hive on tez ? On Sun, Jul 5, 2015 at 10:39 PM, saurabh wrote: > Hi, > > We are in process of exploring TEZ for Hive 0.14. > Needed some pointers to start on Hive with Tez. > E.g. in Hive HDFS Block size plays a vital role in getting the number of > Mappers and later independent execution of mappers can accelerate > processing substantially. > > I understand this is a very vast topic and cannot be described, however > some quick pointers will be helpful. > > I am currently working on: > Query vectorization and COB with ORC tables. > > Thanks, > Saurabh > -- Best Regards Jeff Zhang
Re: hive -e run tez query error
> But keeping that client cache disabled when running against trunk generally kills queries all the time with occasional errors like these. I think tez staging directory is supposed to be deleted when JVM exit (deleteOnExit), why killing query will cause the path deleted ? On Sat, Jun 27, 2015 at 11:48 AM, Gopal Vijayaraghavan wrote: > > > perhaps deleteOnExit() is set somewhere > > fs.cache.disable settings from hdfs-site.xml are usually to blame for that. > > Till hive-1.0, HiveServer2 used to leak filesystem objects, so the cache > was disabled. > > > 2015-06-25 15:54:33,673 INFO FSNamesystem.audit: allowed=true > >ugi=lujian (auth:SIMPLE)ip=/10.17.28.11 <http://10.17.28.11/> > >cmd=delete > >src=/user/lujian/lujian/_tez_session_dir/abb91da9-ac07-4024-a09f-8622ee1ca > >edf dst=nullperm=null > > > But keeping that client cache disabled when running against trunk > generally kills queries all the time with occasional errors like these. > > Cheers, > Gopal > > > -- Best Regards Jeff Zhang
Is it expected behavior ?
I use the following command to try to check the command options of beeline. beeline --help It does display the help info, but after that it continue enter beeline. I think it should only display the help info and should not go into beeline. Is it expected behavior ? --truncateTable=[true/false]truncate table column when it exceeds length --delimiterForDSV=DELIMITER specify the delimiter for delimiter-separated values output format (default: |) --isolation=LEVEL set the transaction isolation level --nullemptystring=[true/false] set to true to get historic behavior of printing null as empty string --addlocaldriverjar=DRIVERJARNAME Add driver jar file in the beeline client side --addlocaldrivername=DRIVERNAME Add drvier name needs to be supported in the beeline client side --help display this message Beeline version 2.0.0-SNAPSHOT by Apache Hive beeline> -- Best Regards Jeff Zhang
Re: hive -e run tez query error
ez/application_1433219182593_252390/tez-conf.pb > ]], TaskAttempt 3 failed, info=[Container > container_1433219182593_252390_01_06 finished with diagnostics set to > [Container failed. File does not exist: hdfs:// > yhd-jqhadoop2.int.yihaodian.com:8020/user/lujian/lujian/_tez_session_dir/63de23a2-1cff-4434-96ad-1304089fb489/.tez/application_1433219182593_252390/ > tez-conf.pb > ]]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex > vertex_1433219182593_252390_2_00 [Map 1] killed/failed due to:null] > DAG failed due to vertex failure. failedVertices:1 killedVertices:0 > FAILED: Execution Error, return code 2 from > org.apache.hadoop.hive.ql.exec.tez.TezTask > > *use hivecli to execute same query ,no exception throw.* > *use hive -e to execute same query on mr ,no exeception throw * > > > > -- > r7raul1...@163.com > > -- Best Regards Jeff Zhang
Re: Hive - Tez error with big join - Container expired.
Tez will hold the idle containers for a while, but it would also expire the container if it reach some threshold. Have you set property tez.am.container.idle.release-timeout-max.millis in tez-site.xml ? And can you attach the yarn app log ? Best Regard, Jeff Zhang From: Daniel Klinger mailto:d...@web-computing.de>> Reply-To: "user@hive.apache.org<mailto:user@hive.apache.org>" mailto:user@hive.apache.org>> Date: Thursday, June 18, 2015 at 5:35 AM To: "user@hive.apache.org<mailto:user@hive.apache.org>" mailto:user@hive.apache.org>> Subject: Hive - Tez error with big join - Container expired. Hi all, I have a pretty big Hive Query. I'm joining over 3 Hive-Tables which have thousands of lines each. I'm grouping this join by several columns. In the Hive-Shell this query only reach about 80%. After about 1400 seconds its canceling with the following error: Status: Failed Vertex failed, vertexName=Map 2, vertexId=vertex_1434357133795_0008_1_01, diagnostics=[Task failed, taskId=task_1434357133795_0008_1_01_33, diagnostics=[TaskAttempt 0 failed, info=[Containercontainer_1434357133795_0008_01_39 finished while trying to launch. Diagnostics: [Container failed. Container expired since it was unused]], TaskAttempt 1 failed, info=[Containercontainer_1434357133795_0008_01_55 finished while trying to launch. Diagnostics: [Container failed. Container expired since it was unused]], TaskAttempt 2 failed, info=[Containercontainer_1434357133795_0008_01_72 finished while trying to launch. Diagnostics: [Container failed. Container expired since it was unused]], TaskAttempt 3 failed, info=[Containercontainer_1434357133795_0008_01_000101 finished while trying to launch. Diagnostics: [Container failed. Container expired since it was unused]]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1434357133795_0008_1_01 [Map 2] killed/failed due to:null] DAG failed due to vertex failure. failedVertices:1 killedVertices:0 FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask My yarn resource manager is at 100% during the whole execution (using all of the 300 GB memory). I tried to extend the live time of my containers with the following setting in the yarn-site.xml but no success: yarn.resourcemanager.rm.container-allocation.expiry-interval-ms = 120 After this change my query stays at 0% over thousands of seconds. The query itself is working (tested with less data). How can I solve this problem. Thanks for your help. Greetz DK
Re: error on hive insert query
You can refer to the following link to figure out whether this is tez problem https://cwiki.apache.org/confluence/display/TEZ/How+to+Diagnose+Tez+App Best Regard, Jeff Zhang From: Sateesh Karuturi mailto:sateesh.karutu...@gmail.com>> Reply-To: "u...@tez.apache.org<mailto:u...@tez.apache.org>" mailto:u...@tez.apache.org>> Date: Wednesday, June 17, 2015 at 1:19 AM To: "user@hive.apache.org<mailto:user@hive.apache.org>" mailto:user@hive.apache.org>>, "u...@tez.apache.org<mailto:u...@tez.apache.org>" mailto:u...@tez.apache.org>> Subject: error on hive insert query iam using hive 1.0.0 and tez 0.5.2. when i set hive.execution.engine value in hive-site.xml to tezselect query works well... but in case of insert getting error. the query is : insert into table tablename values(intvalue,'string value'); and the error is : FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.Tez Task
Re: hive tez error
Sorry, to make it more clear. The applicationId is the placeholder of your applicationId of your hive session. You can look for it in the RM web UI Best Regard, Jeff Zhang From: Sateesh Karuturi mailto:sateesh.karutu...@gmail.com>> Reply-To: "user@hive.apache.org<mailto:user@hive.apache.org>" mailto:user@hive.apache.org>> Date: Monday, June 8, 2015 at 3:52 PM To: "user@hive.apache.org<mailto:user@hive.apache.org>" mailto:user@hive.apache.org>> Subject: Re: hive tez error when i try to check getting this hadoop@localhost:~$ yarn logs -applicationId options parsing failed: Missing argument for option: applicationId Retrieve logs for completed YARN applications. usage: yarn logs -applicationId [OPTIONS] general options are: -appOwnerAppOwner (assumed to be current user if not specified) -containerId ContainerId (must be specified if node address is specified) -nodeAddress NodeAddress in the format nodename:port (must be specified if container id is specified) On Mon, Jun 8, 2015 at 1:20 PM, Jianfeng (Jeff) Zhang mailto:jzh...@hortonworks.com>> wrote: Could you check the yarn app logs ? By invoking command : "yarn logs -applicationId" Best Regard, Jeff Zhang From: Sateesh Karuturi mailto:sateesh.karutu...@gmail.com>> Reply-To: "user@hive.apache.org<mailto:user@hive.apache.org>" mailto:user@hive.apache.org>> Date: Monday, June 8, 2015 at 3:45 PM To: "user@hive.apache.org<mailto:user@hive.apache.org>" mailto:user@hive.apache.org>> Subject: hive tez error getting FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask error... when iam trying to perform insert operation on hive(set hive.execution.engine=tez). whenever hive.execution.engine value is set to mr its works fine please help me out
Re: hive tez error
Could you check the yarn app logs ? By invoking command : "yarn logs -applicationId" Best Regard, Jeff Zhang From: Sateesh Karuturi mailto:sateesh.karutu...@gmail.com>> Reply-To: "user@hive.apache.org<mailto:user@hive.apache.org>" mailto:user@hive.apache.org>> Date: Monday, June 8, 2015 at 3:45 PM To: "user@hive.apache.org<mailto:user@hive.apache.org>" mailto:user@hive.apache.org>> Subject: hive tez error getting FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask error... when iam trying to perform insert operation on hive(set hive.execution.engine=tez). whenever hive.execution.engine value is set to mr its works fine please help me out
Re: drop table command hang
I think I got the error. When I started hive, it will throw mysql key too long exception (only in log file but not in client log, this is very unfriendly IMO ) 2015-03-14 14:18:49,588 ERROR [main]: DataNucleus.Datastore (Log4JLogger.java:error(115)) - An exception was thrown while adding/validating class(es) : Specified key was too long; max key length is 767 bytes On Sat, Mar 14, 2015 at 1:44 PM, Jeff Zhang wrote: > It doesn't matter whether I truncate the table, it always hangs there. > Very werid. > > > > On Wed, Mar 11, 2015 at 3:06 PM, Mich Talebzadeh > wrote: > >> Have you truncated the table before dropping it? I >> >> >> >> Truncate table >> >> Drop table >> >> >> >> Mich Talebzadeh >> >> >> >> http://talebzadehmich.wordpress.com >> >> >> >> *Publications due shortly:* >> >> *Creating in-memory Data Grid for Trading Systems with Oracle TimesTen >> and Coherence Cache* >> >> >> >> NOTE: The information in this email is proprietary and confidential. This >> message is for the designated recipient only, if you are not the intended >> recipient, you should destroy it immediately. Any information in this >> message shall not be understood as given or endorsed by Peridale Ltd, its >> subsidiaries or their employees, unless expressly so stated. It is the >> responsibility of the recipient to ensure that this email is virus free, >> therefore neither Peridale Ltd, its subsidiaries nor their employees accept >> any responsibility. >> >> >> >> *From:* Jeff Zhang [mailto:zjf...@gmail.com] >> *Sent:* 11 March 2015 06:56 >> *To:* user@hive.apache.org >> *Subject:* drop table command hang >> >> >> >> I invoke a drop table command and it hangs there. Here's the log. I am >> using mysql and I can invoke describe command and create table through >> mysql console, so I assume mysql works properly. Can anyone help this ? >> Thanks >> >> >> >> >> >> >> >> 2015-03-11 14:48:09,441 INFO [main]: ql.Driver >> (Driver.java:checkConcurrency(161)) - Concurrency mode is disabled, not >> creating a lock manager >> >> 2015-03-11 14:48:09,441 INFO [main]: log.PerfLogger >> (PerfLogger.java:PerfLogBegin(121)) - > from=org.apache.hadoop.hive.ql.Driver> >> >> 2015-03-11 14:48:09,441 INFO [main]: ql.Driver >> (Driver.java:execute(1321)) - Starting command: drop table >> student_bucketed_s1 >> >> 2015-03-11 14:48:09,441 INFO [main]: log.PerfLogger >> (PerfLogger.java:PerfLogEnd(148)) - > start=1426056489421 end=1426056489441 duration=20 >> from=org.apache.hadoop.hive.ql.Driver> >> >> 2015-03-11 14:48:09,442 INFO [main]: log.PerfLogger >> (PerfLogger.java:PerfLogBegin(121)) - > from=org.apache.hadoop.hive.ql.Driver> >> >> 2015-03-11 14:48:09,442 INFO [main]: log.PerfLogger >> (PerfLogger.java:PerfLogBegin(121)) - > from=org.apache.hadoop.hive.ql.Driver> >> >> 2015-03-11 14:48:09,442 INFO [main]: ql.Driver >> (Driver.java:launchTask(1640)) - Starting task [Stage-0:DDL] in serial mode >> >> 2015-03-11 14:48:09,442 INFO [main]: metastore.HiveMetaStore >> (HiveMetaStore.java:logInfo(743)) - 0: get_table : db=default >> tbl=student_bucketed_s1 >> >> 2015-03-11 14:48:09,443 INFO [main]: HiveMetaStore.audit >> (HiveMetaStore.java:logAuditEvent(368)) - ugi=jzhang ip=unknown-ip-addr >> cmd=get_table : db=default tbl=student_bucketed_s1 >> >> 2015-03-11 14:48:09,458 INFO [main]: metastore.HiveMetaStore >> (HiveMetaStore.java:logInfo(743)) - 0: get_table : db=default >> tbl=student_bucketed_s1 >> >> 2015-03-11 14:48:09,458 INFO [main]: HiveMetaStore.audit >> (HiveMetaStore.java:logAuditEvent(368)) - ugi=jzhang ip=unknown-ip-addr >> cmd=get_table : db=default tbl=student_bucketed_s1 >> >> 2015-03-11 14:48:09,474 INFO [main]: metastore.HiveMetaStore >> (HiveMetaStore.java:logInfo(743)) - 0: drop_table : db=default >> tbl=student_bucketed_s1 >> >> 2015-03-11 14:48:09,474 INFO [main]: HiveMetaStore.audit >> (HiveMetaStore.java:logAuditEvent(368)) - ugi=jzhang ip=unknown-ip-addr >> cmd=drop_table : db=default tbl=student_bucketed_s1 >> >> >> >> >> >> >> -- >> >> Best Regards >> >> Jeff Zhang >> > > > > -- > Best Regards > > Jeff Zhang > -- Best Regards Jeff Zhang
Re: drop table command hang
It doesn't matter whether I truncate the table, it always hangs there. Very werid. On Wed, Mar 11, 2015 at 3:06 PM, Mich Talebzadeh wrote: > Have you truncated the table before dropping it? I > > > > Truncate table > > Drop table > > > > Mich Talebzadeh > > > > http://talebzadehmich.wordpress.com > > > > *Publications due shortly:* > > *Creating in-memory Data Grid for Trading Systems with Oracle TimesTen and > Coherence Cache* > > > > NOTE: The information in this email is proprietary and confidential. This > message is for the designated recipient only, if you are not the intended > recipient, you should destroy it immediately. Any information in this > message shall not be understood as given or endorsed by Peridale Ltd, its > subsidiaries or their employees, unless expressly so stated. It is the > responsibility of the recipient to ensure that this email is virus free, > therefore neither Peridale Ltd, its subsidiaries nor their employees accept > any responsibility. > > > > *From:* Jeff Zhang [mailto:zjf...@gmail.com] > *Sent:* 11 March 2015 06:56 > *To:* user@hive.apache.org > *Subject:* drop table command hang > > > > I invoke a drop table command and it hangs there. Here's the log. I am > using mysql and I can invoke describe command and create table through > mysql console, so I assume mysql works properly. Can anyone help this ? > Thanks > > > > > > > > 2015-03-11 14:48:09,441 INFO [main]: ql.Driver > (Driver.java:checkConcurrency(161)) - Concurrency mode is disabled, not > creating a lock manager > > 2015-03-11 14:48:09,441 INFO [main]: log.PerfLogger > (PerfLogger.java:PerfLogBegin(121)) - from=org.apache.hadoop.hive.ql.Driver> > > 2015-03-11 14:48:09,441 INFO [main]: ql.Driver > (Driver.java:execute(1321)) - Starting command: drop table > student_bucketed_s1 > > 2015-03-11 14:48:09,441 INFO [main]: log.PerfLogger > (PerfLogger.java:PerfLogEnd(148)) - start=1426056489421 end=1426056489441 duration=20 > from=org.apache.hadoop.hive.ql.Driver> > > 2015-03-11 14:48:09,442 INFO [main]: log.PerfLogger > (PerfLogger.java:PerfLogBegin(121)) - from=org.apache.hadoop.hive.ql.Driver> > > 2015-03-11 14:48:09,442 INFO [main]: log.PerfLogger > (PerfLogger.java:PerfLogBegin(121)) - from=org.apache.hadoop.hive.ql.Driver> > > 2015-03-11 14:48:09,442 INFO [main]: ql.Driver > (Driver.java:launchTask(1640)) - Starting task [Stage-0:DDL] in serial mode > > 2015-03-11 14:48:09,442 INFO [main]: metastore.HiveMetaStore > (HiveMetaStore.java:logInfo(743)) - 0: get_table : db=default > tbl=student_bucketed_s1 > > 2015-03-11 14:48:09,443 INFO [main]: HiveMetaStore.audit > (HiveMetaStore.java:logAuditEvent(368)) - ugi=jzhang ip=unknown-ip-addr > cmd=get_table : db=default tbl=student_bucketed_s1 > > 2015-03-11 14:48:09,458 INFO [main]: metastore.HiveMetaStore > (HiveMetaStore.java:logInfo(743)) - 0: get_table : db=default > tbl=student_bucketed_s1 > > 2015-03-11 14:48:09,458 INFO [main]: HiveMetaStore.audit > (HiveMetaStore.java:logAuditEvent(368)) - ugi=jzhang ip=unknown-ip-addr > cmd=get_table : db=default tbl=student_bucketed_s1 > > 2015-03-11 14:48:09,474 INFO [main]: metastore.HiveMetaStore > (HiveMetaStore.java:logInfo(743)) - 0: drop_table : db=default > tbl=student_bucketed_s1 > > 2015-03-11 14:48:09,474 INFO [main]: HiveMetaStore.audit > (HiveMetaStore.java:logAuditEvent(368)) - ugi=jzhang ip=unknown-ip-addr > cmd=drop_table : db=default tbl=student_bucketed_s1 > > > > > > > -- > > Best Regards > > Jeff Zhang > -- Best Regards Jeff Zhang
Re: when start hive could not generate log file
By default, hive.log is located in /tmp/${user}/hive.log
Best Regard,
Jeff Zhang
From: zhangjp mailto:smart...@hotmail.com>>
Reply-To: "user@hive.apache.org<mailto:user@hive.apache.org>"
mailto:user@hive.apache.org>>
Date: Wednesday, March 11, 2015 at 7:12 PM
To: "user@hive.apache.org<mailto:user@hive.apache.org>"
mailto:user@hive.apache.org>>
Subject: when start hive could not generate log file
when i run command " hive" , the message as follows
[@xxx/]# hive
log4j:WARN No appenders could be found for logger
(org.apache.hadoop.hive.common.LogUtils).
log4j:WARN Please initialize the log4j system properly.
Logging initialized using configuration in
file:/search/apache-hive-0.13.1-bin/conf/hive-log4j.properties?
My hive-log4j.properties use the default template. but when I run "find -name
hive.log" couldn't find any file.?
drop table command hang
I invoke a drop table command and it hangs there. Here's the log. I am using mysql and I can invoke describe command and create table through mysql console, so I assume mysql works properly. Can anyone help this ? Thanks 2015-03-11 14:48:09,441 INFO [main]: ql.Driver (Driver.java:checkConcurrency(161)) - Concurrency mode is disabled, not creating a lock manager 2015-03-11 14:48:09,441 INFO [main]: log.PerfLogger (PerfLogger.java:PerfLogBegin(121)) - 2015-03-11 14:48:09,441 INFO [main]: ql.Driver (Driver.java:execute(1321)) - Starting command: drop table student_bucketed_s1 2015-03-11 14:48:09,441 INFO [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(148)) - 2015-03-11 14:48:09,442 INFO [main]: log.PerfLogger (PerfLogger.java:PerfLogBegin(121)) - 2015-03-11 14:48:09,442 INFO [main]: log.PerfLogger (PerfLogger.java:PerfLogBegin(121)) - 2015-03-11 14:48:09,442 INFO [main]: ql.Driver (Driver.java:launchTask(1640)) - Starting task [Stage-0:DDL] in serial mode 2015-03-11 14:48:09,442 INFO [main]: metastore.HiveMetaStore (HiveMetaStore.java:logInfo(743)) - 0: get_table : db=default tbl=student_bucketed_s1 2015-03-11 14:48:09,443 INFO [main]: HiveMetaStore.audit (HiveMetaStore.java:logAuditEvent(368)) - ugi=jzhang ip=unknown-ip-addr cmd=get_table : db=default tbl=student_bucketed_s1 2015-03-11 14:48:09,458 INFO [main]: metastore.HiveMetaStore (HiveMetaStore.java:logInfo(743)) - 0: get_table : db=default tbl=student_bucketed_s1 2015-03-11 14:48:09,458 INFO [main]: HiveMetaStore.audit (HiveMetaStore.java:logAuditEvent(368)) - ugi=jzhang ip=unknown-ip-addr cmd=get_table : db=default tbl=student_bucketed_s1 2015-03-11 14:48:09,474 INFO [main]: metastore.HiveMetaStore (HiveMetaStore.java:logInfo(743)) - 0: drop_table : db=default tbl=student_bucketed_s1 2015-03-11 14:48:09,474 INFO [main]: HiveMetaStore.audit (HiveMetaStore.java:logAuditEvent(368)) - ugi=jzhang ip=unknown-ip-addr cmd=drop_table : db=default tbl=student_bucketed_s1 -- Best Regards Jeff Zhang
Any tutorial document about how to use the example data
Hi, I notice there's one folder example which contains sample data and sample queries. But I didn't find any document about how to use these data and queries. Could anyone point it to me ? Thanks
Where does hive do sampling in order by ?
Order by usually invoke 2 steps (sampling job and repartition job) but hive only run one mr job for order by, so wondering when and where does hive do sampling ? client side ? -- Best Regards Jeff Zhang
Re: Hive CLI question
Bals's right, you can execute any shell command in hive-cli by using "!shell_command;" On Fri, Jan 2, 2015 at 4:30 AM, Bala Krishna Gangisetty wrote: > "!clear" is an another option too. > > --Bala G. > > On Thu, Jan 1, 2015 at 12:23 PM, Mohan Krishna > wrote: > >> Hi Louis >> >> I use "Ctrl+L" as a keyboard short cut to clear Hive screen. >> >> >> >> Thanks >> Mohan >> >> On Fri, Jan 2, 2015 at 1:41 AM, Louis Vincent Frolio >> wrote: >> >>> Does anyone know how to issue a clear screen at the "hive>" prompt? >>> Is this even possible? I am looking for something similar to "system >>> clear" in MySQL. >>> >>> Thank you, Louis. >>> >> >> > -- Best Regards Jeff Zhang
Re: What's official site for howl ?
Thanks for your guys' quick reply, really appreciate that On Thu, May 5, 2011 at 10:13 AM, Alan Gates wrote: > http://incubator.apache.org/hcatalog/ > > Howl has been renamed to HCatalog (due to naming conflicts with an existing > ow2 project called Howl). > > Alan. > > > On May 4, 2011, at 7:04 PM, Jeff Zhang wrote: > > >> Hi all, >> >> Sorry for bothering this mail list. But I only this mail list may relate >> with howl. Just wonder what's official site or mail list for howl. I'd like >> to get more information about howl >> >> >> >> -- >> Best Regards >> >> Jeff Zhang >> > > -- Best Regards Jeff Zhang
What's official site for howl ?
Hi all, Sorry for bothering this mail list. But I only this mail list may relate with howl. Just wonder what's official site or mail list for howl. I'd like to get more information about howl -- Best Regards Jeff Zhang
Find a case that does not make sense in hive
Hi all I create a table with a partition. Then I import data into this table without specify partition, in this case I cannot retrieval data from this table using select statement. But I can retrieve data when importing data with specify partition. So I think since hive do not allow me to retrieve data in the first case, it should not allow me to import data without specify partition -- Best Regards Jeff Zhang
Re: Does hive have batch processing mode ?
Thanks, it works On Thu, Nov 25, 2010 at 3:49 PM, james warren wrote: > Try the following: > % hive -f myhive.file > > cheers, > -James > On Wed, Nov 24, 2010 at 11:37 PM, Jeff Zhang wrote: >> >> Hi all, >> >> I have a bunch of files, and want to import them into one table each >> partition per file. Currently, I have to enter each add partition >> statement in cli. >> >> So I wonder whether hive have a batch processing mode, so that I can >> put the sql statements in one file, and execute the file using one >> command. >> >> Thanks in advance. >> >> -- >> Best Regards >> >> Jeff Zhang > > -- Best Regards Jeff Zhang
Does hive have batch processing mode ?
Hi all, I have a bunch of files, and want to import them into one table each partition per file. Currently, I have to enter each add partition statement in cli. So I wonder whether hive have a batch processing mode, so that I can put the sql statements in one file, and execute the file using one command. Thanks in advance. -- Best Regards Jeff Zhang
Re: How to generate global unique ID?
Please refer https://github.com/twitter/snowflake On Mon, Nov 15, 2010 at 5:09 PM, afancy wrote: > Hi, > Does anybody know how to generate a unique ID in Hive? I have a table: > hive> describe results; > OK > id int > test string > Time taken: 0.091 seconds > > When I insert data into the table results, i want to generate a unqiue Id, > which is like a sequence in RDBMS. > Regards, > afancy -- Best Regards Jeff Zhang
Re: Can i use hive for real time query?
I don't think hive is suitable for real time search, it is suitable for batch-processing with high through-put but high-latency. HBase may be what you need. On Mon, Oct 11, 2010 at 4:02 PM, Guru Prasad wrote: > Hi, > I want to use hive for real time query. > Please anyone explain me how feasible it is? > > Also how many simultaneous queries can hive handle? > > > Thanks & Regards > Guru Prasad > ~guru > > This message is intended only for the use of the addressee and may contain > information that is privileged, confidential and exempt from disclosure > under applicable law. If the reader of this message is not the intended > recipient, or the employee or agent responsible for delivering the message > to the intended recipient, you are hereby notified that any dissemination, > distribution or copying of this communication is strictly prohibited. If you > have received this e-mail in error, please notify us immediately by return > e-mail and delete this e-mail and all attachments from your system. > > -- Best Regards Jeff Zhang
