Hiveserver2 Beeline command clarification

2013-08-13 Thread Sanjay Subramanian
Hi guys I just hooked up hivservrer2 to ldap. In beeline I realized you can login like the following (don't need to define "org.apache.hive.jdbc.HiveDriver") beeline> !connect jdbc:hive2://dev-thdp5:1 sanjay.subraman...@wizecommerce.com scan complete in 2ms Connecting to jdbc:hive2://dev-th

Re: LZO output compression

2013-08-13 Thread Sanjay Subramanian
Check this class where these are defined http://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.1/src/mapred/org/apache/hadoop/mapreduce/lib/output/FileOutputFormat.java From: w00t w00t mailto:w00...@yahoo.de>> Reply-To: "user@hive.apache.org" mailto:user@hiv

Re: Hive and Lzo Compression

2013-08-13 Thread Sanjay Subramanian
Hi I think the CREATE TABLE without the STORED AS clause will not give any errors while creating the table. However when you query that table and since that table contains .lzo files , you would get errors. With external tables , u r separating the table creation(definition) from the data. So

Getting the raw hive column types from HiveResultSetMetaData

2013-08-13 Thread Mingyu Kim
Hi all, I'd like to inspect ResultSet returned by Hive JDBC and to be able to reconstruct the complex types (I.e. Map, struct and array). However, ResultSet and ResultSetMetaData only returns the column type as string for these complex types, making it impossible to distinguish an array "[1,2,3]"

Re: Numbers display in Hive CLI

2013-08-13 Thread pandees waran
Sure , let me explore hive beeline client  — Sent from Mailbox for iPad On Tue, Aug 13, 2013 at 11:24 PM, Stephen Sprague wrote: > Yeah. I would think i'd be a useful feature to have in the client - but > probably not the Hive CLI client. The Hive client seems pretty bare bones > and my guess it

Re: Passing mapreduce configuration parameters to hive udf

2013-08-13 Thread Abdelrahman Shettia
Hi Jon, Please refer to the following document: https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Select#LanguageManualSelect-REGEXColumnSpecification Hope this helps. Thanks -Abdelrahman On Tue, Aug 13, 2013 at 9:13 AM, Jon Bender wrote: > Hi there, > > I'm trying to pass some

Re: Numbers display in Hive CLI

2013-08-13 Thread Stephen Sprague
Yeah. I would think i'd be a useful feature to have in the client - but probably not the Hive CLI client. The Hive client seems pretty bare bones and my guess it'll probably stay that way. The Beeline client, however, looks to be where these kinds of bells and whistles probably could/should be ad

Re: Numbers display in Hive CLI

2013-08-13 Thread pandees waran
Thanks Stephen! I shall check this . My requirement is controlling the formatting in session level using some properties set. Looks like, there's no such as of now . Would this be a good feature in hive cli? If many people think so, then I can file a feature request. — Sent from Mailbox for iPad

Passing mapreduce configuration parameters to hive udf

2013-08-13 Thread Jon Bender
Hi there, I'm trying to pass some external properties to a UDF. In the MapReduce world I'm used to extending Configured in my classes, but in my UDF class when initializing a new Configuration object or HiveConf object it doesn't inherit any of those properties. I see it in the Job Configuration

Re: Numbers display in Hive CLI

2013-08-13 Thread Stephen Sprague
well... a good 'ol search (let's not use the word google) of "hive udf" we find this: https://cwiki.apache.org/Hive/languagemanual-udf.html#LanguageManualUDF-StringFunctionsand there's a reference to a function called format_number(). or did you really want the *hive CLI* to format the number? if

Re: LZO output compression

2013-08-13 Thread w00t w00t
Oh, I could get it working using these settings: SET hive.exec.compress.output=true; SET mapred.output.compression.codec=com.hadoop.compression.lzo.LzopCodec; But I have one question, where maybe on of you can help me with an explaination: As I am running Hadoop 1.1.* why do I need the old comma

LZO output compression

2013-08-13 Thread w00t w00t
Hello, I am running Hortonworks 1.2 using Hadoop 1.1.2.21 and Hive 0.10.0.21. I set up LZO compression and can read LZO compressed data without problems. My next try was to test output compression. Therefore, I created the following small script: -

Re: Nice hive notes and cheat sheets from Minwoo Kim

2013-08-13 Thread Minwoo Kim
Thanks for mentioning my blog. I just uploaded a presentation about hive hook on my blog. http://julingks.tistory.com/entry/Apache-Hive-Hooks I hope this will be useful when wanting to use hive hook. Thanks, Minwoo Kim 2013/8/2 Sanjay Subramanian > http://julingks.tistory.com/category/Hi

Re: Hive and Lzo Compression

2013-08-13 Thread w00t w00t
Thanks for your replies and the link. I could get it working, but wondered why the CREATE TABLE statement worked without the STORED AS Clause as well...that's what puzzles me a bit... But I will use the STORED AS Clause to be on the safe side. Von: Lefty Le