RE: Tableau connectivity available on KR

Sorry for the spam. This was meant as internal Yahoo announcement. Olga From: Mohammad Islam [mailto:misla...@yahoo.com] Sent: Monday, September 30, 2013 3:53 PM To: user@hive.apache.org Subject: Re: Tableau connectivity available on KR Olga. I'm sure it was not intended for me and a lot of us.

Re: Tableau connectivity available on KR

Olga. I'm sure it was not intended for me  and a lot of us. "hive-u...@hadoop.apache.org" made it happened. From: Olga L. Natkovich To: "kryptonite-u...@yahoo-inc.com" ; "hive-u...@hadoop.apache.org" ; "ygrid-sandbox-annou...@yahoo-inc.com" ; "ygrid-produc

Re: UDF error?

ok I found the reason, as I modified the jar file, though I re-ran "ADD .MyUdf.jar; create temporary function ; ", it doesn't take effect. I have to get out of hive session, then rerun these again. On Mon, Sep 30, 2013 at 1:47 PM, Yang wrote: > I wrote a super simple UDF, but got s

Re: UDF error?

Here is an example of a no arg that will return a different value for each row: https://code.google.com/p/gbif-occurrencestore/source/browse/trunk/occurrence-store/src/main/java/org/gbif/occurrencestore/hive/udf/UuidUDF.java Hope this helps, Tim On Mon, Sep 30, 2013 at 10:59 PM, Yang wrote: >

Re: UDF error?

thanks! at first I did have a no-arg evaluate(), but somehow select myfunction(), field1, field2 from mytable ; spits out the same value for myfunction() for each row. so I was wondering whether the UDF got called only 1 time, because the hive compiler sees that the argument is void, so that all

Tableau connectivity available on KR

Dear Grid Users, Hadoop Services team is happy to announce that Tableau is now supported on KR. Please, come give it a try and provide your feedback. The steps to connect with Tableau are described here: http://twiki.corp.yahoo.com/view/Grid/HiveServer2BITools. In addition, we also provide su

Re: UDF error?

That class is: https://code.google.com/p/gbif-occurrencestore/source/browse/trunk/occurrence-store/src/main/java/org/gbif/occurrencestore/hive/udf/UDFRowSequence.java Cheers, Tim On Mon, Sep 30, 2013 at 10:55 PM, Tim Robertson wrote: > It's been ages since I wrote one, but the differences to mi

Re: UDF error?

It's been ages since I wrote one, but the differences to mine: a) I use LongWritable: public LongWritable evaluate(LongWritable startAt) { b) I have annotations on the class (but I think they are just for docs) @Description(name = "row_sequence", value = "_FUNC_() - Returns a generated row sequ

UDF error?

I wrote a super simple UDF, but got some errors: UDF: package yy; import org.apache.hadoop.hive.ql.exec.UDF; import java.util.Random; import java.util.UUID; import java.lang.management.*; public class MyUdf extends UDF { static Random rand = new Random(System.currentTimeMillis() + Thread

Re: how to treat an existing partition data file as a table?

thanks guys, I found that the table is not partitioned, so I guess no way out... On Mon, Sep 30, 2013 at 9:31 AM, Olga L. Natkovich wrote: > You need to specify a table partition from which you want to sample. > > ** ** > > Olga > > ** ** > > *From:* Yang [mailto:tedd...@gmail.com]

Re: Want query to use more reducers

Thanks. mapred.reduce.tasks and hive.exec.reducers.max seem to have fixed the problem. It is now saturating the cluster and running the query super fast. Excellent! On Sep 30, 2013, at 12:28 , Sean Busbey wrote: > Hey Keith, > > It sounds like you should tweak the settings for how Hive hand

Re: Converting from textfile to sequencefile using Hive

Thanks Sean, that is exactly what I want. On Mon, Sep 30, 2013 at 3:09 PM, Sean Busbey wrote: > S, > > Check out these presentations from Data Science Maryland back in May[1]. > > 1. working with Tweets in Hive: > > > http://www.slideshare.net/JoeyEcheverria/analyzing-twitter-data-with-hadoop-2

Re: Want query to use more reducers

Hey Keith, It sounds like you should tweak the settings for how Hive handles query execution[1]: 1) Tune the guessed number of reducers based on input size = hive.exec.reducers.bytes.per.reducer Defaults to 1G. Based on your description, it sounds like this is probably still at default. In thi

Re: Converting from textfile to sequencefile using Hive

S, Check out these presentations from Data Science Maryland back in May[1]. 1. working with Tweets in Hive: http://www.slideshare.net/JoeyEcheverria/analyzing-twitter-data-with-hadoop-20929978 2. then pulling stuff out of Hive to use with Mahout: http://files.meetup.com/6195792/Working%20With%

Re: Converting from textfile to sequencefile using Hive

Hi Nitin, No offense taken. Thank you for your response. Part of this is also trying to find the right tool for the job. I am doing queries to determine the cuts of tweets that I want, then doing some modest normalization (through a python script) and then I want to create sequenceFiles from that

Re: Converting from textfile to sequencefile using Hive

are you using hive to just convert your text files to sequence files? If thats the case then you may want to look at the purpose why hive was developed. If you want to modify data or process data which does not involve any kind of analytics functions on a routine basis. If you want to do a data m

Want query to use more reducers

I have a query that doesn't use reducers as efficiently as I would hope. If I run it on a large table, it uses more reducers, even saturating the cluster, as I desire. However, on smaller tables it uses as low as a single reducer. While I understand there is a logic in this (not using multipl

Re: Hive Query via Hue, Only column headers in downloaded CSV or XSL results, sometimes

+ hue-user thanks Prasad On Mon, Sep 30, 2013 at 11:05 AM, Martin, Nick wrote: > Mark - is the Hive table you're using for this fairly wide? If so, are you > doing a "select * from table_name limit 10"? > > We ran some tests this morning on one of the Hive tables giving us some > fits and if

RE: Hive Query via Hue, Only column headers in downloaded CSV or XSL results, sometimes

Mark - is the Hive table you're using for this fairly wide? If so, are you doing a "select * from table_name limit 10"? We ran some tests this morning on one of the Hive tables giving us some fits and if we limit the select to ~20 columns and put the limit on the query we get the returns fairly

Converting from textfile to sequencefile using Hive

Hi, I have a lot of tweets saved as text. I created an external table on top of it to access it as textfile. I need to convert these to sequencefiles with each tweet as its own record. To do this, I created another table as a sequencefile table like so - CREATE EXTERNAL TABLE tweetseq( tweet ST

RE: Doing FSCK throws error

The script for this table is add jar json-serde-1.1.3-jar-with-dependencies.jar; list jars; CREATE EXTERNAL TABLE IF NOT EXISTS table1 ( instance_type string, category string, session_id string, nonce string, user_id string, properties array>, instance map, true_as_of_secs string ) PARTITIONED

Doing FSCK throws error

Hi, On executing "MSCK REPAIR TABLE table1", I get the below error. FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask What can possibly be the error. Thanks, Shouvnik This message is for the designated recipient only and may c

RE: Not able to execute this query

Thanks Nitin. I am able to create the table in HUE now. You are right. There was no directory and no permission accordingly. Thanks, Shouvanik From: Nitin Pawar [mailto:nitinpawar...@gmail.com] Sent: Monday, September 30, 2013 9:08 PM To: Haldar, Shouvanik Cc: user@hive.apache.org Subject: Re:

RE: how to treat an existing partition data file as a table?

You need to specify a table partition from which you want to sample. Olga From: Yang [mailto:tedd...@gmail.com] Sent: Sunday, September 29, 2013 1:39 PM To: hive-u...@hadoop.apache.org Subject: how to treat an existing partition data file as a table? we have a huge table, including browsing

Re: Error - loading data into tables

Is this /home/strorage/... a hdfs directory? I think its a normal filesystem directory. Try running this load data local inpath '*/home/storage/mount1/tabled.txt' INTO TABLE TEST;*" On Mon, Sep 30, 2013 at 7:13 PM, Manickam P wrote: > Hi, > > I'm getting the below error while loading the data

Re: Not able to execute this query

Do the hue user have permissions to access '/user/hue'? is that directory existing ? On Mon, Sep 30, 2013 at 8:58 PM, wrote: > Hi Nitin, > > ** ** > > Thanks. That answers my previous query. But, if I add LOCATION > '/user/hue/' string below, I get a big fat exception in beeswax. > > *

Converting from textfile to sequencefile using Hive

Hi, I have a lot of tweets saved as text. I created an external table on top of it to access it as textfile. I need to convert these to sequencefiles with each tweet as its own record. To do this, I created another table as a sequencefile table like so - CREATE EXTERNAL TABLE tweetseq( tweet ST

RE: Not able to execute this query

Hi Nitin, Thanks. That answers my previous query. But, if I add LOCATION '/user/hue/' string below, I get a big fat exception in beeswax. Thanks, Shouvanik From: Nitin Pawar [mailto:nitinpawar...@gmail.com] Sent: Monday, September 30, 2013 8:22 PM To: user@hive.apache.org Subject: Re: Not able

Re: Not able to execute this query

I am really not sure what your entire query is but the below one works . If possible share your entire ddl and mask or hide cols if there is something you can not share create table test1( col3 int, col4 string) PARTITIONED BY (col1 timestamp, col2 timestamp) CLUSTERED BY(col3) SORTED BY(col3

RE: Not able to execute this query

Hi, Have you used HUE WEB console. Actually I have not used same columns. But, when I give a query, I get that error.! Please help? Thanks, Shouvanik From: Nitin Pawar [mailto:nitinpawar...@gmail.com] Sent: Monday, September 30, 2013 7:34 PM To: user@hive.apache.org Subject: Re: Not able to ex

RE: Hive Query via Hue, Only column headers in downloaded CSV or XSL results, sometimes

Hi Mark - we hit this issue as well. We use Hue as the Hive front-end for our users and this is a pretty big roadblock for them. We're on Hue 2.2 and Hive 11. If you figure out a fix let me know :) -Original Message- From: Sunderlin, Mark [mailto:mark.sunder...@teamaol.com] Sent: Mo

Re: Not able to execute this query

you are trying to bucket and partition on same column? I could create a hive table if I change the bucketing column to non-partition column On Mon, Sep 30, 2013 at 7:23 PM, wrote: > When I executing the query to create table in HIVE, I am getting this > error. > > ** ** > > 'NoneType' obj

Not able to execute this query

When I executing the query to create table in HIVE, I am getting this error. 'NoneType' object has no attribute 'columns' The table script below create external table test1( - ) PARTITIONED BY (col1 timestamp, col2 timestamp) CLUSTERED BY(col1) SORTED BY(col1 ASC)

Error - loading data into tables

Hi, I'm getting the below error while loading the data into hive table. return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask I used " LOAD DATA INPATH '/home/storage/mount1/tabled.txt' INTO TABLE TEST;" this query to insert into table. Thanks, Manickam P

RE: Hive Query via Hue, Only column headers in downloaded CSV or XSL results, sometimes

Hmm.. No replies on this one? Is no one use Hue? :-) That would be interesting to know .. if not Hue, how are others exposing Hive to "end users?" without given them a direct login to a node on the cluster? --- Mark E. Sunderlin Data Architect | AOL NETWORKS BDM P: 703-265-6935 | C: 540-3

RE: unable to create a table in hive

Thanks man. I added hive site and it worked. Thanks, Manickam P From: Nitin Pawar Sent: ‎30-‎09-‎2013 05:35 PM To: user@hive.apache.org Subject: Re: unable to create a table in hive hive-site.xml will be

Re: unable to create a table in hive

hive-site.xml will be placed under your hive conf directory. anyway, try using location flag to your ddl like below CREATE TABLE TABLE_A (EMPLOYEE_ID INT, EMPLOYEE_NAME STRING, EMPLOYEE_LOCATION STRING, EMPLOYEE_DEPT STRING) ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' STORED AS TEXTFILE LOCATION

RE: unable to create a table in hive

Hi, I have given below the script i used. I've not used any hive site xml here. CREATE TABLE TABLE_A (EMPLOYEE_ID INT, EMPLOYEE_NAME STRING, EMPLOYEE_LOCATION STRING, EMPLOYEE_DEPT STRING) ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' STORED AS TEXTFILE; Thanks, Manickam P Date: Mon, 30 Sep 20

Re: unable to create a table in hive

Can you share your create table ddl and hive warehouse directory setting from hive-site.xml ? On Mon, Sep 30, 2013 at 4:57 PM, Manickam P wrote: > Guys, > > when i try to create a new table in hive i am getting the below error. > *FAILED: Error in metadata: MetaException(message:Got exception:

unable to create a table in hive

Guys, when i try to create a new table in hive i am getting the below error. FAILED: Error in metadata: MetaException(message:Got exception: java.io.FileNotFoundException /user)FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask I've created direcotries in hdfs lik

Re: Load Timestamp data type fom local file

Hi Claudio, When you do a select * from table there is no mapreduce in place. What hive does is it uses the hdfs api and reads your files and displays the data by a tab separated columns list. If the data is wrongly populated, hive will show the entire set into first column and rest of the colum

Re: Load Timestamp data type fom local file

Thanks Nitin for the reply, if I run the query "SELECT * FROM momis_test_a_3" I get an empty result set with no errors. Instead I would expect all the results. My best guess is that because of timestamp data the whole dataset is not able to be loaded. But since I don't have any errors (of any kin

Re: Load Timestamp data type fom local file

Sorry but I could not understand the issues you are facing. When you loaded data, did select col from table for the timestamp column, what error did you get? what data did you get? this is the default datetime format "-MM-dd hh:mm:ss." Looking at your sample data seems to match the format

Load Timestamp data type fom local file

Hello, For unit testing, I would like to load from a local file data that has several columns, one is also Timestamp. The command I use is "LOAD DATA LOCAL INPATH... ". Unfortunately that column does not allow me to load all the dataset. I have no error in the log of my local apache hive server,

issue about remote hive client

hi,all: i run hive client in seperate box ,but all job submit from the client is local job,why? ,i try it from hive-server2 running box ,the job will submit as distribute job