Hive Patch update
Hello, The following patch was released on 25th February, 2013, Monday: https://issues.apache.org/jira/browse/HIVE-3235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#issue-tabs Although it says resolution: unresolved, I was wondering that if it is available in the current stable release version of Hive. How do I use this function to get the jobId of the mapred tasks submitted to the Hadoop cluster by the hive server, if I submit a Hive query remotely using the Hive client. I need the jobId in my application that is calling the Hive server using the hive-client to monitor the progress of query submitted to the hive server. Or is there a way to achieve this using the hadoop job notification service? Thank you for the help. -- Regards, Ouch Whisper 010101010101
Re: Hive Patch update
if it says unresolved it means it is not part of the code base else you will have the release version there. Currently there is just one API for this cluster status in the apis so you may want to try set a jobname for each query you fire through your application and then get the respective job from the jobtracker api On Fri, Mar 1, 2013 at 7:35 PM, Panshul Whisper ouchwhis...@gmail.comwrote: Hello, The following patch was released on 25th February, 2013, Monday: https://issues.apache.org/jira/browse/HIVE-3235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#issue-tabs Although it says resolution: unresolved, I was wondering that if it is available in the current stable release version of Hive. How do I use this function to get the jobId of the mapred tasks submitted to the Hadoop cluster by the hive server, if I submit a Hive query remotely using the Hive client. I need the jobId in my application that is calling the Hive server using the hive-client to monitor the progress of query submitted to the hive server. Or is there a way to achieve this using the hadoop job notification service? Thank you for the help. -- Regards, Ouch Whisper 010101010101 -- Nitin Pawar
Request to add me onto the list
Hi, I request you to kindly add me onto this list. -- Thanks and Regards, Sachin Sudarshana
Finding maximum across a row
Hi, Is there any function/method to find the maximum across a row in hive? Suppose i have a table like this: ColA ColB ColC 2 5 7 3 2 1 I want the function to return 7 1 Its urgently required. Any help would be greatly appreciated! -- Thanks and Regards, Sachin Sudarshana
regexp_replace with unicode chars
I would like to remove unicode chars that are outside the Basic Multilingual Plane [1] I thought select regexp_replace(some_column,[^\\u-\\u],\ufffd) from my_table would work but while the regexp does work the replacement str does not (I can paste in the literal �, which you may or may not be able to see here but it somehow did not fell right) I saw Deans previous post on using octals [2] but I think \ufffd is outside the allowable range. Cheers, Tom [1] http://en.wikipedia.org/wiki/Plane_%28Unicode%29#Basic_Multilingual_Plane [2] http://grokbase.com/t/hive/dev/131a4n562y/unicode-character-as-delimiter
Re: regexp_replace with unicode chars
I think this should work, but you might investigate using the translate function instead. I suspect it will provide much better performance than using regexps. Also, Are you planning to do this once to create your final tables? If so, the performance overhead won't matter much. dean On Fri, Mar 1, 2013 at 10:52 AM, Tom Hall thattommyh...@gmail.com wrote: I would like to remove unicode chars that are outside the Basic Multilingual Plane [1] I thought select regexp_replace(some_column,[^\\u-\\u],\ufffd) from my_table would work but while the regexp does work the replacement str does not (I can paste in the literal �, which you may or may not be able to see here but it somehow did not fell right) I saw Deans previous post on using octals [2] but I think \ufffd is outside the allowable range. Cheers, Tom [1] http://en.wikipedia.org/wiki/Plane_%28Unicode%29#Basic_Multilingual_Plane [2] http://grokbase.com/t/hive/dev/131a4n562y/unicode-character-as-delimiter -- *Dean Wampler, Ph.D.* thinkbiganalytics.com +1-312-339-1330
Re: Finding maximum across a row
Hi Sachin You could get the detailed ateps from hive wiki itself https://cwiki.apache.org/Hive/hiveplugins.html Regards Bejoy KS Sent from remote device, Please excuse typos -Original Message- From: Sachin Sudarshana sachin.sudarsh...@gmail.com Date: Fri, 1 Mar 2013 22:37:54 To: user@hive.apache.org; bejoy...@yahoo.com Reply-To: user@hive.apache.org Subject: Re: Finding maximum across a row Hi Bejoy, I am new to UDF in Hive. Could you send me any link/tutorials on where i can be able to learn about writing the UDF? Thanks! On Fri, Mar 1, 2013 at 10:22 PM, bejoy...@yahoo.com wrote: ** Hi Sachin AFAIK There isn't one at the moment. But you can easily achieve this using a custom UDF. Regards Bejoy KS Sent from remote device, Please excuse typos -- *From: * Sachin Sudarshana sachin.sudarsh...@gmail.com *Date: *Fri, 1 Mar 2013 22:16:37 +0530 *To: *user@hive.apache.org *ReplyTo: * user@hive.apache.org *Subject: *Finding maximum across a row Hi, Is there any function/method to find the maximum across a row in hive? Suppose i have a table like this: ColA ColB ColC 2 5 7 3 2 1 I want the function to return 7 1 Its urgently required. Any help would be greatly appreciated! -- Thanks and Regards, Sachin Sudarshana -- Thanks and Regards, Sachin Sudarshana
Re: regexp_replace with unicode chars
Thanks Dean, I dont think translate would work as the set of things to remove is massive. Yeah, it's a one-off cleanup job while exporting to try redshift on our datasets. My guess is it's something about the way hive handles strings? Tried \\ufffd as the replacement str but no joy either. Cheers again, Tom On 1 March 2013 17:08, Dean Wampler dean.wamp...@thinkbiganalytics.comwrote: I think this should work, but you might investigate using the translate function instead. I suspect it will provide much better performance than using regexps. Also, Are you planning to do this once to create your final tables? If so, the performance overhead won't matter much. dean On Fri, Mar 1, 2013 at 10:52 AM, Tom Hall thattommyh...@gmail.com wrote: I would like to remove unicode chars that are outside the Basic Multilingual Plane [1] I thought select regexp_replace(some_column,[^\\u-\\u],\ufffd) from my_table would work but while the regexp does work the replacement str does not (I can paste in the literal �, which you may or may not be able to see here but it somehow did not fell right) I saw Deans previous post on using octals [2] but I think \ufffd is outside the allowable range. Cheers, Tom [1] http://en.wikipedia.org/wiki/Plane_%28Unicode%29#Basic_Multilingual_Plane [2] http://grokbase.com/t/hive/dev/131a4n562y/unicode-character-as-delimiter -- *Dean Wampler, Ph.D.* thinkbiganalytics.com +1-312-339-1330
Re: regexp_replace with unicode chars
Anyone know if translate takes ranges, like some implementations? e.g.,
translate ('[a-z]', '[A-Z]')
Of course, that probably doesn't work for non-ascii characters.
On Fri, Mar 1, 2013 at 11:24 AM, Tom Hall thattommyh...@gmail.com wrote:
Thanks Dean,
I dont think translate would work as the set of things to remove is
massive.
Yeah, it's a one-off cleanup job while exporting to try redshift on our
datasets.
My guess is it's something about the way hive handles strings? Tried
\\ufffd as the replacement str but no joy either.
Cheers again,
Tom
On 1 March 2013 17:08, Dean Wampler dean.wamp...@thinkbiganalytics.comwrote:
I think this should work, but you might investigate using the translate
function instead. I suspect it will provide much better performance than
using regexps. Also, Are you planning to do this once to create your final
tables? If so, the performance overhead won't matter much.
dean
On Fri, Mar 1, 2013 at 10:52 AM, Tom Hall thattommyh...@gmail.comwrote:
I would like to remove unicode chars that are outside the Basic
Multilingual Plane [1]
I thought
select regexp_replace(some_column,[^\\u-\\u],\ufffd) from
my_table
would work but while the regexp does work the replacement str does not
(I can paste in the literal �, which you may or may not be able to see here
but it somehow did not fell right)
I saw Deans previous post on using octals [2] but I think \ufffd is
outside the allowable range.
Cheers,
Tom
[1]
http://en.wikipedia.org/wiki/Plane_%28Unicode%29#Basic_Multilingual_Plane
[2]
http://grokbase.com/t/hive/dev/131a4n562y/unicode-character-as-delimiter
--
*Dean Wampler, Ph.D.*
thinkbiganalytics.com
+1-312-339-1330
--
*Dean Wampler, Ph.D.*
thinkbiganalytics.com
+1-312-339-1330
RE: Finding maximum across a row
Seems like you would want the results as: 7 3 Here is one way to get the maximum value for a collection of columns with a case statement. (this not done in Hive but hive case statements should work same way) ## data in table xyz select * from xyz ; c1 | c2 | c3 ++ 2 | 1 | 3 3 | 2 | 1 1 | 2 | 3 (3 rows) ## Use case statements select case when (c1 c2) and (c1 c3 ) then c1 when (c2 c1) and (c2 c3 ) then c2 else c3 end max_col from xyz ; ## results max_col - 3 3 3 (3 rows) From: Sachin Sudarshana [mailto:sachin.sudarsh...@gmail.com] Sent: Friday, March 01, 2013 11:47 AM To: user@hive.apache.org Subject: Finding maximum across a row Hi, Is there any function/method to find the maximum across a row in hive? Suppose i have a table like this: ColA ColB ColC 2 5 7 3 2 1 I want the function to return 7 1 Its urgently required. Any help would be greatly appreciated! -- Thanks and Regards, Sachin Sudarshana
Re: regexp_replace with unicode chars
The translate UDF does take care of non-ascii characters. It uses
codepoints instead of characters.
Here is the unit test to demonstrate that:
https://github.com/apache/hive/blob/trunk/ql/src/test/queries/clientpositive/udf_translate.q#L36
But you guys are right. It doesn't solve Tom's original problem
because it doesn't take ranges.
I created HIVE-4100 for improving regex_replace UDF.
Mark
On Fri, Mar 1, 2013 at 9:31 AM, Dean Wampler
dean.wamp...@thinkbiganalytics.com wrote:
Anyone know if translate takes ranges, like some implementations? e.g.,
translate ('[a-z]', '[A-Z]')
Of course, that probably doesn't work for non-ascii characters.
On Fri, Mar 1, 2013 at 11:24 AM, Tom Hall thattommyh...@gmail.com wrote:
Thanks Dean,
I dont think translate would work as the set of things to remove is
massive.
Yeah, it's a one-off cleanup job while exporting to try redshift on our
datasets.
My guess is it's something about the way hive handles strings? Tried
\\ufffd as the replacement str but no joy either.
Cheers again,
Tom
On 1 March 2013 17:08, Dean Wampler dean.wamp...@thinkbiganalytics.com
wrote:
I think this should work, but you might investigate using the translate
function instead. I suspect it will provide much better performance than
using regexps. Also, Are you planning to do this once to create your final
tables? If so, the performance overhead won't matter much.
dean
On Fri, Mar 1, 2013 at 10:52 AM, Tom Hall thattommyh...@gmail.com
wrote:
I would like to remove unicode chars that are outside the Basic
Multilingual Plane [1]
I thought
select regexp_replace(some_column,[^\\u-\\u],\ufffd) from
my_table
would work but while the regexp does work the replacement str does not
(I can paste in the literal �, which you may or may not be able to see here
but it somehow did not fell right)
I saw Deans previous post on using octals [2] but I think \ufffd is
outside the allowable range.
Cheers,
Tom
[1]
http://en.wikipedia.org/wiki/Plane_%28Unicode%29#Basic_Multilingual_Plane
[2]
http://grokbase.com/t/hive/dev/131a4n562y/unicode-character-as-delimiter
--
Dean Wampler, Ph.D.
thinkbiganalytics.com
+1-312-339-1330
--
Dean Wampler, Ph.D.
thinkbiganalytics.com
+1-312-339-1330
doubt with LEFT OUTER JOIN
Hi, Does LEFT OUTER JOIN returns zero rows when RIGHT SIDE table doesn't have at least one record that matches JOIN condition in Hive? Regards, Kumar
Re: doubt with LEFT OUTER JOIN
I just tried an experiment where the right-hand table was empty. It worked fine. Could you post more details, like the query, create table statements, etc? What version of Hive? On Fri, Mar 1, 2013 at 2:13 PM, kumar mr kumar...@aol.com wrote: Hi, Does LEFT OUTER JOIN returns zero rows when RIGHT SIDE table doesn't have at least one record that matches JOIN condition in Hive? Regards, Kumar -- *Dean Wampler, Ph.D.* thinkbiganalytics.com +1-312-339-1330
Re: installing hive-0.10.0 from source
In a fresh checkout today, I found identical RELEASE_NOTES.txt files in four Hive branches: - branch-0.8-r2 - branch-0.10 - ptf-windowing - trunk Branch 0.9's release notes are updated, but branch 0.10 still has the release notes from branch 0.8-r2. So I've opened a JIRA: https://issues.apache.org/jira/browse/HIVE-4107 -- Update Hive 0.10.0 RELEASE_NOTES.txt. – Lefty Leverenz On Tue, Feb 26, 2013 at 2:16 PM, Eric Chu e...@rocketfuel.com wrote: To update this thread, I also did a git clone and checked out branch-0.10. The release notes there also says it's 0.8.0, and the git log doesn't have many of the changes mentioned in the Jira change log for 0.10.0https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12320745styleName=TextprojectId=12310843.in http://hive.apache.org/releases.html. But the lib folder (after I did the ant package) shows *-0.10.0-*.jar files, so I suppose it is 0.10 and that the RELEASE_NOTES.txt is outdated. This is a little confusing. It'd be great if someone from Apache could review/update the build from source instruction or the release notes. Eric On Tue, Feb 26, 2013 at 11:38 AM, Eric Chu e...@rocketfuel.com wrote: Hi, I tried to build Hive0.10.0 from source by doing the following: svn co http://svn.apache.org/repos/asf/hive/trunk hive sudo ant package It built fine and I got the dist folder as expected, but inside that folder the RELEASE_NOTES.txt says the version is 0.8.0. How do I get 0.10.0 source? I'm looking at https://github.com/apache/hivenow but the instruction there points back to https://cwiki.apache.org/confluence/display/Hive/GettingStarted Thanks, Eric
