Drop Partition by ID

2014-07-21 Thread fab wol
Hi everyone, I have the following problem: I have a partitoned managed table (Partition table is a string which represents a date, eg. log-date="2014-07-15"). Unfortunately there is one partition in there like this: log_date=2014-07-15-23%3A45%3A38 (copied from show partitions stmt). This partitio

Re: MSCK REPAIR TABLE

2014-03-27 Thread fab wol
nning a general repair table on the backend knowing you "broke it" up front? just a suggestion! On Thu, Mar 27, 2014 at 3:18 AM, fab wol <mailto:darkwoll...@gmail.com>> wrote: Hey Nitin and everyone else, so let me tell you from memory that the Hive CLI Error wa

Re: MSCK REPAIR TABLE

2014-03-27 Thread fab wol
+01:00 Nitin Pawar : > Without error stack, very hard to get whats wrong > > will it be possible for you to run it via hive cli and grab some logs > through there ? > > > On Thu, Mar 27, 2014 at 3:29 PM, fab wol wrote: > >> Hey Nitin, >> >> HiveServer2 Lo

Re: MSCK REPAIR TABLE

2014-03-27 Thread fab wol
hiveserver2 stopped, pid 2554 Thu Mar 27 10:55:52 CET 2014 hiveserver2 started, pid 2597 Cheers Wolli 2014-03-27 10:04 GMT+01:00 Nitin Pawar : > can you grab more logs from hiveserver2 log file? > > > On Thu, Mar 27, 2014 at 2:31 PM, fab wol wrote: > >> Hey everyone, >

MSCK REPAIR TABLE

2014-03-27 Thread fab wol
Hey everyone, I have a table with currently 5541 partitions. Daily there are 14 partitions added. I will switch the update for the metastore from "msck repair table" to "alter table add partition", since its performing better, but sometimes this might fail, and i need the "msck repair table" comma

Re: Best way to avoid cross join

2014-03-19 Thread fab wol
replied, cross join always uses only one reducer. > > If you want to avoid this can you just try it to make full outer join with > on condition (1 = 1) ? and see if you get your desired result > > > On Wed, Mar 19, 2014 at 4:05 PM, fab wol wrote: > >> anyone? >> &g

Re: Best way to avoid cross join

2014-03-19 Thread fab wol
anyone? still haven't solved this problem. Any help is appreciated. Cheers Wolli 2014-03-14 10:55 GMT+01:00 fab wol : > Hey Nitin, > > in import1 are at least 1.2 mio rows, with almost the same amount of > distinct id's and approxametly 40k distinct keywords. et_keywords

Re: Best way to avoid cross join

2014-03-14 Thread fab wol
; > can you share how many different ids are there and whats the data sizes in > rows? > > > On Wed, Mar 5, 2014 at 7:57 PM, fab wol wrote: > >> hey Yong, >> >> Even without the group by (pure cross join) the query is only using one >> reducer. Even specifyin

Re: Best way to avoid cross join

2014-03-05 Thread fab wol
hey Yong, Even without the group by (pure cross join) the query is only using one reducer. Even specifying more reducers doesn't help: set mapred.reduce.tasks=50; SELECT id1, m.keyword, prep_kw.keyword FROM (select id1, keyword from import1) m CROSS JOIN (SELECT keyword FROM et_ke

Best way to avoid cross join

2014-03-05 Thread fab wol
Hey everyone, before i write a lot of text, i just post something which is already written: http://www.sqlservercentral.com/Forums/Topic1328496-360-1.aspx The first posts adresses a pretty similar problem i also have. Currently my implementation looks like this: SELECT id1, MAX( CASE WHE

Re: NullPointerException on Sample Tables / CDH 4.4

2013-10-11 Thread fab wol
somehow after three days of searching, i just deployed client configuration for all hive roles again, and the error seems to be gone. lets see what the future brings. cheers 2013/10/11 fab wol > hey everyone, > > I've got supplied with a decent ten node CDH 4.4 cluster, only 7 d

NullPointerException on Sample Tables / CDH 4.4

2013-10-11 Thread fab wol
hey everyone, I've got supplied with a decent ten node CDH 4.4 cluster, only 7 days old, and someone tried some HBase stuff on it. I wanted to apply my (on another cluster working) workflow's to that cluster (consisting of HiveQL Scripts and Oozie Workflows) but unfortunately i the following issue

Re: Hive built-in functions not working

2013-06-21 Thread fab wol
On Fri, Jun 21, 2013 at 7:32 AM, fab wol wrote: > >> I'm using Hadoop 0.20.2 with Hive 0.11. I have succesfully inserted into >> hive/hdfs some csv-files in seperate tables. selects and joins work >> flawlessly. When trying to analyse some data, i needed to mak

Hive built-in functions not working

2013-06-21 Thread fab wol
I'm using Hadoop 0.20.2 with Hive 0.11. I have succesfully inserted into hive/hdfs some csv-files in seperate tables. selects and joins work flawlessly. When trying to analyse some data, i needed to make use of the built in functions of hive like: - substr - to_date - rand - etc. for example: