Re: dynamic Partition not splitting properly

use already existing UDFs to split or transform your values the way you want On Fri, Jun 14, 2013 at 12:09 PM, Hamza Asad wrote: > OIC. I got it. Thanx alot nitin :). One more thing i want to ask related > this issue, if old table contains event_date in format "2012-06-24 > 06:04:11.9" then how

Re: dynamic Partition not splitting properly

OIC. I got it. Thanx alot nitin :). One more thing i want to ask related this issue, if old table contains event_date in format "2012-06-24 06:04:11.9" then how can i partition it according to date part only? As partition column does not accepts to_date(event_date) form. On Thu, Jun 13, 2013 at 5

non-string partition column types is it discouraged

I have a question regarding the partition column types in a Hive table. We run hive 0.9.0 in a cloudera distribution and we're having issue trying to connect to hive using the Cloudera Tableau ODBC Connector. I'm unable to use a partition column of type int. Trying to find a solution I chanced upo

Re: Enhancing Query Join to speed up Query

Basically 1. if you join table try to filter out as much as possible in WHERE (to reduce amount of data sent form map to reduce step) 2. if you join big table with small table (< 500 MB) use SELECT /*+ MAPJOIN(small_table) */ hint to avoid reduce step. 3. if you join big table with big table make

Re: Enhancing Query Join to speed up Query

Hi, Can you please paste the output of "explain" for both the queries so that we can know if the predicate is being pushed down in both the cases. Thanks, On Fri, Jun 14, 2013 at 10:05 AM, Naga Vijay wrote: > Hi, > > Here are the test results for the two query options shown in my initial > em

Re: Enhancing Query Join to speed up Query

Hi, Here are the test results for the two query options shown in my initial email - Test Criteria - Both tables have 20 million rows each, 20 partitions, 16 buckets for item_id Query Option 1 - Took 25 minutes, 5 seconds Query Option 2 - Took 24 minutes, 42 seconds Thanks for all the responses!

Re: Issues with Hive 'Like" and multiple '%' wildcard in a pattern

yeah. good one. its a bug alright. where the trouble starts is when a '?' follows a wildcard char. '%' in this case. Looks like hive slavishly mutates a '%' to a '.*' and since you got a '?' after that you get another valid regex looking like '.*?' ('?" being the minimally match directive)

Re: Access context from UDF

If VCs are not on query, it'll not be made in ExecMapper from start. If you can do something with Reporter instance in MR, https://issues.apache.org/jira/browse/HIVE-3628 would be helpful. 2013/5/30 Peter Marron : > Hi, > > > > Using Hive 0.10.0 over Hadoop 1.0.4. > > > > I guess that I know that

Re: Question about weekofyear(string date)

Thanks! On Thu, Jun 13, 2013 at 6:30 PM, Darren Yin wrote: > It's all right here: monday to monday and has a concept of the first > "full" week too. > https://github.com/apache/hive/blob/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFWeekOfYear.java > > > On Thu, Jun 13, 2013 at 2:44 PM, Ro

Re: Enhancing Query Join to speed up Query

You can use "explain" for confirming differences. For inner joins, it would make the same plan. 2013/6/14 Igor Tatarinov : > I would expect no difference because of predicate pushdown. > > igor > decide.com > > > On Thu, Jun 13, 2013 at 11:31 AM, Naga Vijay wrote: >> >> Sure, Will do >> >> >> On

Re: Hive .11 Won't Reduce

This seems conf issue. Have you included all conf path and libs to run hive? Sent from my Galaxy S®III Original message From: Beau Rothrock Date: 06/13/2013 7:25 PM (GMT-06:00) To: "" Subject: Re: Hive .11 Won't Reduce Thanks, but ACLs are disabled. mapred.acls.enable

Re: Hive .11 Won't Reduce

Thanks, but ACLs are disabled. mapred.acls.enabled false -- Beau On Jun 13, 2013, at 2:48 PM, Arafat, Moiz wrote: > Can you See if the id which you are using to run the queries has map reduce > permissions on the cluster ? > > Regards, > Moiz Arafat > > > On Jun 13, 2013, at 1:55 PM,

Re: Question about weekofyear(string date)

It's all right here: monday to monday and has a concept of the first "full" week too. https://github.com/apache/hive/blob/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFWeekOfYear.java On Thu, Jun 13, 2013 at 2:44 PM, Robert Li wrote: > Hi All > > For this UDF, does it consider the week to

Re: Enhancing Query Join to speed up Query

I would expect no difference because of predicate pushdown. igor decide.com On Thu, Jun 13, 2013 at 11:31 AM, Naga Vijay wrote: > Sure, Will do > > > On Thu, Jun 13, 2013 at 10:42 AM, Stephen Sprague wrote: > >> Hi naja, >> test those two versions (or three now) and report back to the group.

Re: Hive .11 Won't Reduce

Can you See if the id which you are using to run the queries has map reduce permissions on the cluster ? Regards, Moiz Arafat On Jun 13, 2013, at 1:55 PM, Beau Rothrock wrote: > Hi Y'all, > > I can connect to hive just fine. I can select rows just fine, but anything > that appears to requir

Question about weekofyear(string date)

Hi All For this UDF, does it consider the week to be from Monday to Monday or first hour of Sunday to the last minute of Saturday for example? Robert -- * Robert Li | Integration and Support Engineer | Kontagent Kontagent is Hiring! | Check us out in th

RE: Issues with Hive 'Like" and multiple '%' wildcard in a pattern

Hive users: It took me a while to create a standalone use case, but here we go: In Oracle, this SQL returns false for both cases: SELECT CASE WHEN '?&icid=mainmoiz%dl' LIKE ('%?icid=main%') THEN 'T' ELSE 'F' END AS case1, CASE WHEN '?&icid=mai

Re: Enhancing Query Join to speed up Query

Sure, Will do On Thu, Jun 13, 2013 at 10:42 AM, Stephen Sprague wrote: > Hi naja, > test those two versions (or three now) and report back to the group. :) > even if some smarty-pants thinks he knows the answer its always good to > confirm things are as they should be. > > > On Wed, Jun 12, 201

Hive .11 Won't Reduce

Hi Y'all, I can connect to hive just fine. I can select rows just fine, but anything that appears to require map/reduce crashes consistently. Anyone know of a fix or workaround for this? I'm using these: hive-0.11.0-bin hadoop-2.0.0-mr1-cdh4.2.0 hive> select count(*) from lookout_analytics.x

Re: Enhancing Query Join to speed up Query

Hi naja, test those two versions (or three now) and report back to the group. :) even if some smarty-pants thinks he knows the answer its always good to confirm things are as they should be. On Wed, Jun 12, 2013 at 11:54 PM, Sanjay Subramanian < sanjay.subraman...@wizecommerce.com> wrote: > Hi

Wrong values returned for nullable columns in hbase tables when accessed via hive

Hi all, I am facing an issue when selecting a nullable column twice in a hive select statement against an hbase table. For rows where that column is null, 2 different values are returned: one null (correct) and the second is the last non-null value. Has anyone seen this issue? The query: hive>

Re: dynamic Partition not splitting properly

If the input column value is NULL or empty string, the row will be put into a special partition, whose name is controlled by the hive parameter hive.exec.default.dynamic.partition.name. The default value is `__HIVE_DEFAULT_PARTITION__`. Basically this partition will contain all "bad" rows whose val

Re: dynamic Partition not splitting properly

when i browse it in browser, all the data is in * event_date=__HIVE_DEFAULT_PARTITION__ *, rest of the

Re: dynamic Partition not splitting properly

what do you mean when you say "it wont split correctly" ? On Thu, Jun 13, 2013 at 5:19 PM, Hamza Asad wrote: > what if i have data of more then 500 days then how can i create partition > on date column by specifying each and every date? (i knw that does not > happens in dynamic partition but on

Re: dynamic Partition not splitting properly

what if i have data of more then 500 days then how can i create partition on date column by specifying each and every date? (i knw that does not happens in dynamic partition but on dynamic partition, it wont splits correctly). On Thu, Jun 13, 2013 at 4:12 PM, Nitin Pawar wrote: > you can partiti

Re: dynamic Partition not splitting properly

you can partition existing table unless the hdfs data is laid out in partitioned fashion. your best bet is create a new partitioned table enable dynamic paritionining read from old table and write into new table you can verify the new partitions by using command "show partitions" On Thu, Jun 13,

Re: dynamic Partition not splitting properly

now i created partition table like *CREATE TABLE new_rc_partition_cluster_table( id int, event_id int, user_id BIGINT, intval_1 int , intval_2 int, intval_3 int, intval_4 int, intval_5 int, intval_6 int, intval_7 int, intval_8 int, intval_9 int, intval_10 int, intval_11

Index not working

I have created simple table as follow *CREATE TABLE events_details( id int, event_id int, user_id BIGINT, event_date string, intval_1 int , intval_2 int, intval_3 int, intval_4 int, intval_5 int, intval_6 int, intval_7 int, intval_8 int, intval_9 int, intval_10 int, in

multi group by with multiple distinct columns come to a wrong result

Currently if there are multiple distinct columns, I think the optimize of HIVE-609 won't work. But the default configure param "hive.optimize.multigroupby.common.distincts" is true so the query still spraying over the distinct columns first and gets an wrong result. Why not make it more intelli

Re: ant maven-build not working in trunk

Thanks Nitin, I will give a try with HIVE-4387. On Thu, Jun 13, 2013 at 12:30 PM, Nitin Pawar wrote: > I just tried a build with both jdk versions > > build = ant clean package > jdk7 on branch-0.10 with patch from HIVE-3384 and it works > jdk6 on trunk without any changes it works > > i created

Re: ant maven-build not working in trunk

I just tried a build with both jdk versions build = ant clean package jdk7 on branch-0.10 with patch from HIVE-3384 and it works jdk6 on trunk without any changes it works i created a new redhat vm and installed sun jdk 6u43 and tried it. It works too. when i try ant maven-build -Dmvn.publish.r