RE: Question about query result storage

2012-08-09 Thread Venkatesh Kavuluri
You can always do something like INSERT OVERWRITE LOCAL DIRECTORY '/path/' SELECT [] FROM [] which saves the result set on to the given path. Check Hive wiki for more info.https://cwiki.apache.org/confluence/display/Hive/GettingStarted > Date: Thu, 9 Aug 2012 17:42:17 -0400 > From:

Re: Question about query result storage

2012-08-09 Thread Yue Guan
Oh, actually is hive -S -f some_query.q > some_query.log On 08/09/2012 05:41 PM, Yue Guan wrote: We always do something like this: hive -f some_query.q > some_query.log If the output is large, just insert overwrite to some table. On 08/09/2012 05:32 PM, Tom Brown wrote: Team, I'm a new Hive

Re: Question about query result storage

2012-08-09 Thread Yue Guan
We always do something like this: hive -f some_query.q > some_query.log If the output is large, just insert overwrite to some table. On 08/09/2012 05:32 PM, Tom Brown wrote: Team, I'm a new Hive user and I've just run my first large query (a few hours). Unfortunately, I ran it from the CLI, an

Question about query result storage

2012-08-09 Thread Tom Brown
Team, I'm a new Hive user and I've just run my first large query (a few hours). Unfortunately, I ran it from the CLI, and the output was longer than my SSH client allowed for (scroll buffer) so I can't see the first 1/2 of the result. (It also changes tabs to spaces so properly aligning the column

Re: Nested Select Statements

2012-08-09 Thread Bertrand Dechoux
I don't have a hive shell under my hands. What's the planning? It should be doable with a single MapReduce if the cross join is constructed as a 'HashJoin'. Is that the case? I am curious. (You can use 'explain' on your query to know the planning.) Bertrand On Thu, Aug 9, 2012 at 11:08 PM, wro

RE: Nested Select Statements

2012-08-09 Thread richin.jain
Thanks Guys, it worked. From: ext Bertrand Dechoux [mailto:decho...@gmail.com] Sent: Thursday, August 09, 2012 5:03 PM To: user@hive.apache.org Subject: Re: Nested Select Statements Basically a cross join. You would have the same issue with SQL. Bertrand On Thu, Aug 9, 2012 at 10:41 PM, shrikant

Re: Nested Select Statements

2012-08-09 Thread Bertrand Dechoux
Basically a cross join. You would have the same issue with SQL. Bertrand On Thu, Aug 9, 2012 at 10:41 PM, shrikanth shankar wrote: > This should work > > Select ts,id,sum(metric/usage_count) from usage join (select count(*) > usage_count from usage) V on ( 1 = 1) group by ts,id; > > thanks, > Sh

Re: Nested Select Statements

2012-08-09 Thread shrikanth shankar
This should work Select ts,id,sum(metric/usage_count) from usage join (select count(*) usage_count from usage) V on ( 1 = 1) group by ts,id; thanks, Shrikanth On Aug 9, 2012, at 1:33 PM, wrote: > Hi (vers), > > This might be a very basic question for most of you but I am stuck at it for >

Nested Select Statements

2012-08-09 Thread richin.jain
Hi (vers), This might be a very basic question for most of you but I am stuck at it for quite some time now. I have a table with three columns : Describe usage; ts string id string metric double I am trying to do a query like Select ts,id,sum(metric/(select count(*) from usage)) from usage group

RE: Converting rows into dynamic colums in Hive

2012-08-09 Thread richin.jain
Thanks John. Is there a way to do this is excel since I am outputting the table data in csv format (using macros or something else)? Richin -Original Message- From: ext John Meagher [mailto:john.meag...@gmail.com] Sent: Thursday, August 09, 2012 10:11 AM To: user@hive.apache.org Subjec

Re: Hive append support

2012-08-09 Thread Sandeep Reddy P
HI, Can we consider using HBASE for the same? On Thu, Aug 9, 2012 at 1:19 PM, Sandeep Reddy P wrote: > Thank you all for the info. > > > On Thu, Aug 9, 2012 at 12:30 PM, Bob Gause wrote: > >> Hive has no update & delete statements. >> >> You can drop a table, and that is as closes to a delete a

Re: Hive append support

2012-08-09 Thread Sandeep Reddy P
Thank you all for the info. On Thu, Aug 9, 2012 at 12:30 PM, Bob Gause wrote: > Hive has no update & delete statements. > > You can drop a table, and that is as closes to a delete as you get. > > The only "update" you get is to append more data to a table. There is > INSERT OVERWRITE & INSERT. T

Re: Hive append support

2012-08-09 Thread Bob Gause
Hive has no update & delete statements. You can drop a table, and that is as closes to a delete as you get. The only "update" you get is to append more data to a table. There is INSERT OVERWRITE & INSERT. The first will create the first set of rows in the table, the second will append more data

Re: Hive append support

2012-08-09 Thread Bejoy Ks
Hi Sandeep In some scenarios we badly have to implement updates for our use cases. We end up overwriting a partition/sub partition in hive rather than overwriting the whole table to have least performance impact. You need to intelligently decide on partitions based on your use case for having a

Re: Hive append support

2012-08-09 Thread Sandeep Reddy P
Hi Bejoy, Thanks for the link. When you say updates are not supported directly is there any other way we can update data in HDFS/Hive? On Thu, Aug 9, 2012 at 10:30 AM, Bejoy Ks wrote: > Hi Sandeep > > If you are looking at inserting more data into existing tables that has > data, the yes it is

Skipping bad records

2012-08-09 Thread Travis Crawford
Hey hive gurus - I recently had some issues getting Hive to process a partition with bad records, and am curious how others deal with this issue. From searching around, I learned Hive uses the MR-provided bad record skipping functionality, instead of doing anything specific about bad records. The

Re: Hive append support

2012-08-09 Thread Bejoy Ks
Hi Sandeep If you are looking at inserting more data into existing tables that has data, the yes it is supported in hive from 0.8 (i guess). You cn see the INSERT INTO documentation here https://cwiki.apache.org/Hive/languagemanual-dml.html#LanguageManualDML-InsertingdataintoHiveTablesfromquerie

Re: load data inot hive using shell script

2012-08-09 Thread prabhu k
Thanks Jan, it's working fine. On Thu, Aug 9, 2012 at 7:32 PM, Jan Dolinár wrote: > I'd suggest you to read a bit about shell scripts... To make this work you > have to pass the queries to the hive via -e option. The way you wrote it, > interactive cli would be run and the queries would be attem

Re: Converting rows into dynamic colums in Hive

2012-08-09 Thread John Meagher
R is another programming language geared to statistical analysis. It has some features that make it easy to write this sort of data manipulation. http://www.r-project.org/ Here's a sample R function that does something similar to what you need (note: I'm not a real R programmer, so this might be

Re: load data inot hive using shell script

2012-08-09 Thread Jan Dolinár
I'd suggest you to read a bit about shell scripts... To make this work you have to pass the queries to the hive via -e option. The way you wrote it, interactive cli would be run and the queries would be attempted to be executed by shell when cli ends. The correct skript in your case shouild look li

Socket Leaks in Hive 0.8

2012-08-09 Thread Subroto
Hi,When I try to close my HiveMetaStoreClient, it fails. Following is the code snippet from Hive code base which is responsible for closing the connection.public void close() {    isConnected = false;    if ((transport != null) && transport.isOpen()) {      transport.close();    }    try {      if

RE: Reporting tool's error

2012-08-09 Thread yogesh.kumar13
Thanks Nitin :-) No Datameer is on other machine and hadoop in not installed over there. I have setup single node cluster on other and using datameer from other machine. by giving ip address and port no I am trying to connect it. Please suggest Thanks & Regards Yogesh Kumar

Re: Reporting tool's error

2012-08-09 Thread Nitin Pawar
from exception it looks like it is trying to reach the hdfs but on the localhost. is your datameet platform installed on the namenode machine itself? if not then you may want to try changing the hdfs location Thanks, Nitin On Thu, Aug 9, 2012 at 2:29 PM, wrote: > Hi all, > > I am using Datamee

Reporting tool's error

2012-08-09 Thread yogesh.kumar13
Hi all, I am using Datameer to generate report over hive data. I have started hiveserver and trying to import data from another machine (on which datameer installed) over the same network. I have successful created datastore and all tables stored in hive is visible. But when I import them I get

Re: How to turn off logging in hive

2012-08-09 Thread Ravindra
thanks bharat for your response. but I am still not able to reduce the logging. I tried with the level=ERROR but still I see a log of INFO and DEBUG log messages. -- Ravi. *''We do not inherit the earth from our ancestors, we borrow it from our children.'' PROTECT IT !* On Tue, Aug 7, 2012 at 1