Large Scale Table Reprocess

Just finishing up testing with Hive 11 and ORC. Thank you to Owen and all those who have put hard work into this. Just ORC files, when compared to RC files in Hive 9, 10, and 11 saw a huge increase in performance, it was amazing. That said, now we gotta reprocess. We have a large table with lots

anyone ever user filecrush?

I am using the --clone option correctly? hadoop jar filecrush.jar crush.Crush --clone --threshold 1 --compress org.apache.hadoop.io.compress.GzipCodec --input-format org.apache.hadoop.mapred.TextInputFormat --output-format org.apache.hadoop.mapred.SequenceFileOutputFormat --info --verbose /user

Cannot Revoke Privileges

I am trying to run the following query: "revoke show_database to role test_role" The result I get is: Error: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask (state=08S01,code=1) I checked hive.log and this is what I found: 2013

Re: Parse MAP viarable Keys in HIVE

that sure looks like a nice candidate for a UDF, eh? Come to think of it i'm surprised there isn't a jar file out there in openSource land that contains higher order functions already. On Thu, Jul 25, 2013 at 11:20 AM, Arafat, Moiz wrote: > Try this: > > select > distinct arr --- Distinct V

Re: Parse MAP viarable Keys in HIVE

Try this: select distinct arr --- Distinct Values from ( select explode(arr) as arr -- Breaks Array in every column, as separate row . ( select map_keys(KV) as arr -- Returns an array of the Key Names for every row . from )a ) c order by arr; Thanks, Moiz On Jul 25, 2013, at 2:08

Re: Help in debugging Hive Query

Hi Sanjay,   Thanks for taking the time to write all the details. I did a silly mistake. The data type for visit_page_num, i created it as string. The string was causing issues when I am using the max function. A type cast to int in the query worked for me.   Regards, Raj _

Parse MAP viarable Keys in HIVE

Hi All, I have a table in Apache Hive and I am interested in a specific field of the table. Table name: Table1 and field name f1. The field f1 is of type MAP. The table is huge. How could I find the distinct key values of the MAP variable f1 above? I tried: $hive -e "select distinct map_keys(

Re: Help in debugging Hive Query

The query is correct but since u r creating a managed table , that is possibly creating some issue and the records are not all getting created This is what I would propose CHECKPOINT 1 : Is this query running at all ? === Use this option in BOLD and run the QUERY

Re: Problem with the windowing function ntile (Exceptions)

Hi Navis, that's awesome! Thank you for the quick response and the patch. I just tested it against my query which used to fail and it just worked. So you have at least my thumbs-up. Thanks again for your help. Cheers, Lars On Thu, Jul 25, 2013 at 10:01 AM, Navis류승우 wrote: > I've booked this an

Re: PL/SQL to HiveQL translation

The advice I have always seen for your case is to transform the subquery in the WHERE clause into a LEFT OUTER JOIN. On Thu, Jul 25, 2013 at 11:04 AM, Edson Ramiro wrote: > AFAIK, > > Hive supports subqueries only in the FROM clause. > > Maybe you have to split you query into more queries...

Re: PL/SQL to HiveQL translation

AFAIK, Hive supports subqueries only in the FROM clause. Maybe you have to split you query into more queries... https://cwiki.apache.org/confluence/display/Hive/LanguageManual+SubQueries Edson Ramiro On Thu, Jul 25, 2013 at 9:31 AM, Jérôme Verdier wrote: > Hi Bennie, > > I was tryi

Re: PL/SQL to HiveQL translation

Hi Bennie, I was trying some solutions to pass through my problem, and a problem occurs here is the error : FAILED: ParseException line 26:14 cannot recognize input near 'SELECT' 'cal' '.' in expression specification Is AND...BETWEEN ( SELECT. is possible in Hive? 2013/7/25 Bennie Schut

Help in debugging Hive Query

All,   I am trying to determine visits for customer from omniture weblog file using Hive.   Table: omniture_web_data Columns: visid_high,visid_low,evar23,visit_page_num   Sample Data: visid_high,visid_low,evar23,visit_page_num 999,888,1003,10 999,888,1003,14 999,888,1003,6 999,777,1003,12 999,777,

Re: PL/SQL to HiveQL translation

Hi Jerome, Yes it looks like you could stop using GET_SEMAINE and directly joining "calendrier_hebdo" with "calendrier" for example. For "FCALC_IDJOUR" you will have to make a udf so I hope you have some java skills :) The "calendrier" tables suggests you have star schema with a calendar tabl

Re: Problem with the windowing function ntile (Exceptions)

I've booked this and attached patch for it. https://issues.apache.org/jira/browse/HIVE-4932 Could you test with that? thanks. 2013/7/25 Lars Francke : > We're still being bitten by this problem without a workaround. Does > anyone have an idea? > > Thanks, > Lars > > On Wed, Jul 17, 2013 at 11:24

PL/SQL to HiveQL translation

Hi, I need some help to translate a PL/SQL script in HiveQL. Problem : my PL/SQL script is calling two functions. you can see the script below : SELECT in_co_societe as co_societe, 'SEMAINE' as co_type_periode, a.