Does Hive 0.6 support underscores in variable names?
Just a thought. The examples I've seen contain letters and dots (".") but
no other characters.
– Lefty
On Sat, Aug 4, 2012 at 12:39 AM, Bejoy KS <bejoy...@yahoo.com> wrote:
> **
> Yes that is the right issue. Variable substitution is not happening. I
> can't say much here as I haven't tried out this on 0.6 . The code on my
> blog post is based on 0.7 or higher version I guess.
> Regards
> Bejoy KS
>
> Sent from handheld, please excuse typos.
> ------------------------------
> *From: * Techy Teck <comptechge...@gmail.com>
> *Date: *Sat, 4 Aug 2012 00:35:20 -0700
> *To: *<user@hive.apache.org>; <bejoy...@yahoo.com>
> *ReplyTo: * user@hive.apache.org
> *Subject: *Re: Passing date as command line arguments
>
> Hi Bejoy,
>
> I found something very strange. When I actually viewed the Hive log file,
> I found something like this- See the thing in red color. Why is it like so?
>
> *SessionStart SESSION_ID="rjamal_201208041433" TIME="1344065620956"*
> *QueryStart QUERY_STRING=" select * from lip_data_quality where dt = '
> ${hiveconf:start_date}'"
> QUERY_ID="rjamal_20120804143333_3719afde-117d-4c6b-a322-0cff6e4b4c5d"
> TIME="1344065629740"*
> *Counters
> plan="{"queryId":"rjamal_20120804143333_3719afde-117d-4c6b-a322-0cff6e4b4c5d","queryType":null,"queryAttributes":{"queryString":"
> select * from lip_data_quality where dt =
> '${hiveconf:start_date}'"},"queryCounters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"false","started":"true"}"
> TIME="1344065629743"*
> *QueryEnd QUERY_STRING=" select * from lip_data_quality where dt =
> '${hiveconf:start_date}'"
> QUERY_ID="rjamal_20120804143333_3719afde-117d-4c6b-a322-0cff6e4b4c5d"
> QUERY_RET_CODE="0" QUERY_NUM_TASKS="0" TIME="1344065629745"*
> *Counters
> plan="{"queryId":"rjamal_20120804143333_3719afde-117d-4c6b-a322-0cff6e4b4c5d","queryType":null,"queryAttributes":{"queryString":"
> select * from lip_data_quality where dt =
> '${hiveconf:start_date}'"},"queryCounters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"true","started":"true"}"
> TIME="1344065629745"*
>
> On Sat, Aug 4, 2012 at 12:30 AM, Techy Teck <comptechge...@gmail.com>wrote:
>
>> I cannot try that on Higher version of Hive as I am working in a company
>> where they have installed only Hive 0.6 on the Ares cluster.
>>
>> :(
>>
>>
>>
>>
>> On Sat, Aug 4, 2012 at 12:29 AM, Bejoy KS <bejoy...@yahoo.com> wrote:
>>
>>> **
>>>
>>> Try it on a higher version of hive and let me know if that doesn't work
>>> still. 0.9 should be good.
>>>
>>>
>>> Regards
>>> Bejoy KS
>>>
>>> Sent from handheld, please excuse typos.
>>> ------------------------------
>>> *From: * Techy Teck <comptechge...@gmail.com>
>>> *Date: *Sat, 4 Aug 2012 00:26:44 -0700
>>> *To: *<user@hive.apache.org>; <bejoy...@yahoo.com>
>>> *ReplyTo: * user@hive.apache.org
>>> *Subject: *Re: Passing date as command line arguments
>>>
>>> Yes it is String for me here.
>>>
>>> This is the table structure by the way-
>>>
>>> *create table lip_data_quality*
>>> *( buyer_id bigint,*
>>> * total_chkout bigint,*
>>> * total_errpds bigint*
>>> * )*
>>> * partitioned by (dt string)*
>>> *row format delimited fields terminated by '\t'*
>>> *stored as sequencefile*
>>> *location '/apps/hdmi-technology/b_apdpds/lip-data-quality'*
>>> *;*
>>>
>>> in which I am using dt in the where clause in my query. I am running *Hive
>>> 0.6.*
>>>
>>>
>>>
>>> On Sat, Aug 4, 2012 at 12:24 AM, Bejoy KS <bejoy...@yahoo.com> wrote:
>>>
>>>> **
>>>> I tried the same query on my end, It is working fine for me without any
>>>> issues.
>>>>
>>>> By de way the data type for 'dt' is String itself right?
>>>> Regards
>>>> Bejoy KS
>>>>
>>>> Sent from handheld, please excuse typos.
>>>> ------------------------------
>>>> *From: * Techy Teck <comptechge...@gmail.com>
>>>> *Date: *Sat, 4 Aug 2012 00:21:37 -0700
>>>> *To: *<user@hive.apache.org>; <bejoy...@yahoo.com>
>>>> *ReplyTo: * user@hive.apache.org
>>>> *Subject: *Re: Passing date as command line arguments
>>>>
>>>> I was not able to understand. So what changes I need to make? This is
>>>> my below *hivetest1.hql-*
>>>> *
>>>> *
>>>> *select * from lip_data_quality where dt = '${hiveconf:start_date}';*
>>>> *
>>>> *
>>>> And by this I am executing the above hql file-
>>>>
>>>> *hive -f hivetest1.hql -hiveconf start_date=20120709*
>>>> *
>>>> *
>>>> *
>>>> *
>>>> Any thoughts what changes I need to make?
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Sat, Aug 4, 2012 at 12:16 AM, Bejoy KS <bejoy...@yahoo.com> wrote:
>>>>
>>>>> **
>>>>> Yes.
>>>>>
>>>>> From the logs the query being executed is
>>>>> select * from lip_data_quality where dt=20120709
>>>>>
>>>>> But here the dt is not in quotes. It should be like
>>>>> select * from lip_data_quality where dt='20120709';
>>>>> Regards
>>>>> Bejoy KS
>>>>>
>>>>> Sent from handheld, please excuse typos.
>>>>> ------------------------------
>>>>> *From: * Techy Teck <comptechge...@gmail.com>
>>>>> *Date: *Sat, 4 Aug 2012 00:07:07 -0700
>>>>> *To: *<user@hive.apache.org>; Bejoy Ks<bejoy...@yahoo.com>
>>>>> *ReplyTo: * user@hive.apache.org
>>>>> *Subject: *Re: Passing date as command line arguments
>>>>>
>>>>> I found this thing from the Hive log. This is what you are looking
>>>>> for?-
>>>>>
>>>>> *SessionStart SESSION_ID="rjamal_201208041359" TIME="1344063565655"*
>>>>> *QueryStart QUERY_STRING="select * from lip_data_quality where
>>>>> dt=20120709"
>>>>> QUERY_ID="rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486"
>>>>> TIME="1344063597488"*
>>>>> *Counters
>>>>> plan="{"queryId":"rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486","queryType":null,"queryAttributes":{"queryString":"select
>>>>> * from lip_data_quality where dt=20120709"},"queryCo*
>>>>> *unters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"false","started":"true"}"
>>>>> TIME="1344063597501"*
>>>>> *QueryEnd QUERY_STRING="select * from lip_data_quality where
>>>>> dt=20120709"
>>>>> QUERY_ID="rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486"
>>>>> QUERY_RET_CODE="0" QUERY_NUM_TASKS="0" TIME="134406359*
>>>>> *7503"*
>>>>> *Counters
>>>>> plan="{"queryId":"rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486","queryType":null,"queryAttributes":{"queryString":"select
>>>>> * from lip_data_quality where dt=20120709"},"queryCo*
>>>>> *unters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"true","started":"true"}"
>>>>> TIME="1344063597504"*
>>>>> *
>>>>> *
>>>>> *
>>>>> *
>>>>>
>>>>>
>>>>>
>>>>> On Fri, Aug 3, 2012 at 11:58 PM, Techy Teck
>>>>> <comptechge...@gmail.com>wrote:
>>>>>
>>>>>> Yes that query gives me the result back correctly.
>>>>>>
>>>>>> How can I see the hive log Bejoy? Basically I am not sure which hive
>>>>>> log and what is the path where I can see that thing?
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Fri, Aug 3, 2012 at 11:52 PM, Bejoy Ks <bejoy...@yahoo.com> wrote:
>>>>>>
>>>>>>> Hi Techy
>>>>>>>
>>>>>>> Are you getting the results for the same query directly from CLI, I
>>>>>>> mean does the below query return correct results
>>>>>>> select * from lip_data_quality where dt = '20120709';
>>>>>>>
>>>>>>> If the above query returns results, can you check the hive logs and
>>>>>>> see what is the actual query being executed?
>>>>>>>
>>>>>>> Regards,
>>>>>>> Bejoy KS
>>>>>>>
>>>>>>> ------------------------------
>>>>>>> *From:* Techy Teck <comptechge...@gmail.com>
>>>>>>> *To:* user@hive.apache.org
>>>>>>> *Cc:* Vijay Dirisala <tec...@gmail.com>
>>>>>>> *Sent:* Saturday, August 4, 2012 12:11 PM
>>>>>>> *Subject:* Re: Passing date as command line arguments
>>>>>>>
>>>>>>> Thanks Vijay for the suggestion. I also tried that and it still
>>>>>>> didn't worked for me.
>>>>>>>
>>>>>>> Any thoughts why it is not working?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Aug 3, 2012 at 11:32 PM, Vijay <tec...@gmail.com> wrote:
>>>>>>>
>>>>>>> There can't be a space in the variable name. Try this:
>>>>>>> ${hiveconf:start_date}
>>>>>>>
>>>>>>> On Fri, Aug 3, 2012 at 11:06 PM, Techy Teck <comptechge...@gmail.com>
>>>>>>> wrote:
>>>>>>> > Can anyone help me out on this?
>>>>>>> >
>>>>>>> > I am stuck on this and I have no clue what wrong I am doing?
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>> > On Fri, Aug 3, 2012 at 3:48 PM, Techy Teck <
>>>>>>> comptechge...@gmail.com> wrote:
>>>>>>> >>
>>>>>>> >> I have my below query in test1.hql file. I am trying to pass the
>>>>>>> date (dt)
>>>>>>> >> as the command line argument.
>>>>>>> >>
>>>>>>> >>
>>>>>>> >> select * from lip_data_quality where dt = '${hiveconf:
>>>>>>> start_date}';
>>>>>>> >>
>>>>>>> >>
>>>>>>> >> So whenever I try to run the above test1.hql file like this-
>>>>>>> >>
>>>>>>> >> hive -f hivetest1.hql -hiveconf start_date=20120709
>>>>>>> >>
>>>>>>> >> I get zero records back. But the data is there in that table for
>>>>>>> that
>>>>>>> >> date. Why is it so? Something wrong I am doing?
>>>>>>> >>
>>>>>>> >> Can anyone help me out here? I was following Bejoy's Article here
>>>>>>> >>
>>>>>>> http://kickstarthadoop.blogspot.com/2011/10/include-values-during-execution-time-in.html
>>>>>>> >>
>>>>>>> >>
>>>>>>> >
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
