Re: Loading data only into one node

Hi, The only way to do this is to set the replication factor to 1. dfs.replication 1 You have to set this property to 1 and upload the file to HDFS locally on the DD where you want it to be stored. Still no guarantee that it will end up there. But why would you want to do this? It total

Re: Something wrong with my query to get TOP 3?

2012-07-09 21:43:29 >>>>>>> 1345653 330760137950 2012-07-09 21:40:29 >>>>>>> 1345653 330760137950 2012-07-09 21:41:29 >>>>>>> 1345653 110909316904 2012-07-09 21:29:06 >>>>>>> 1345

Re: Something wrong with my query to get TOP 3?

Hi, I more or less had the same problem and finally got it down by introducing a second subquery. This will guarantee that the rank function is invoked on the reduce phase and that the rank results are properly sorted. I guess something like this: *SELECT bid, pid, rank FROM * *(SELECT bi

Re: Something wrong with my query to get TOP 3?

Hi, I more or less had the same problem and finally got it down by introducing a second subquery. This will guarantee that the rank function is invoked on the reduce phase and that the rank results are properly sorted. I guess something like this: *SELECT bid, pid, rank FROM * *(SELECT bi

Re: Hive Web Services

gt; > Now I want to run queries like displaying Hive's table data, is it > possible to do it from Hive Web services. If so then how? > > Please suggest > > Regards > Yogesh Kumar Dhari > -- > *From:* Jasper Knulst [jasper.knu...@incentro.com] &g

Re: Hive Web Services

Hi Yogesh, Can you tell us what you eventually had to do to get the HWI running ? (seen your earlier mail topic where you had some trouble to get it going) Was it the relative path? Thanks Jasper 2012/7/18 > Hi all, > > I have installed hadoop and hive and its working well through terminal,

Re: UPLOADING .XLS FILES

Hi Yogesh, I guess this can't be done directly on Excel files since Excel file use proprietary MS data formatting. You would have to know all the ins and outs of the native Excel format to make it work (and write some custom class for it. Why not convert to CSV before uploading the files to HDFS?

Re: Invalid Function rank in HiveQL

Hi Raihan, You should use 'rank(buyer_id)' in the order by clause on line 9 in stead of the alias 'rk'. I had the same problem, strangely, the alias is not resolved when it is in the order by clause. Other thing, I had some issues when I used this exact same set up for ranking results, that the r

Re: Passing parameters to initialize UDF

to > not check the output schema somehow, those would also be helpful. > > Thanks! > -- Met vriendelijke groet, *Jasper Knulst* Consultant *|* Incentro Business Intelligence Gildeweg 5b, Nootdorp The Netherlands, 2632 BD *E:* jasper.knu...@incentro.com *T:* +31157640750 *M:

Re: Making UDFs "permanent"

Hi Denny, I asked the same question a few days ago and got this reference to another question: "If you want to make your temporary function permanent , you have to patch hive source code. Please refer to this discussion http://mail-archives.apache.org/mod_mbox/hive-user/201101.mbox/%3caanlktimbx1

Create permanent UDF's not using "Create temporary function"

Hi I have been successfully using UDF's in Hive for a while. So I started wondering what it would take to use non-temporary functions so you don't have to bother going through the same commands "add jars" & "create temporary function" for every session. Why is it not possible to register function

Re: Loading files from a directory

Hi Mayank, Can you post your Hive DDL script. Otherwise its not possible to help you. Jasper Knulst 2012/6/21 Mayank Bansal > Hi, > > ** ** > > I am trying to create an external table in hive, by referring to a > directory in hadoop containing multiple files of the sa

Re: Reading XML Files

rde2.RegexSerDe actually lives) add jar /pathto/hive/lib/hive_serde-0.7.0-cdh3u4.jar; Or you copy these jar to $HADOOP_HOME/lib and restart job/tasktrackers -- Jasper Knulst 2012/6/20 Tucker, Matt > Has anyone had success reading XML files in Hive? I’ve been looking at > the cloud9 XML

Re: Hive server not starting...on EC2 Ubuntu 10.04 instance

Hi Periya, Try removing both .lck (lock) files in the metastore_db folder. The derby db can only support one user. So if you have used hive from cli it is probably for another user. Cheers, Jasper Op 18 dec. 2011 17:42 schreef "Periya.Data" het volgende: > Hi Vivek, >Tried doing with sudo a

Re: Partitioning EXTERNAL TABLE without copying or moving files

Hi Vince, Hive partitioning can only exist by issueing new directories in HDFS. There is no way to partition the data in a Hive table without adding extra filepaths/dirs in HDFS. For an external table you have to redistribute the data yourself in corresponding filepaths and add the new partition

Re: Data loading from Datanode

Hi Keshav, What you want is not possible I guess. You can't submit anything into HDFS without the namenode. Datanodes reports their local blocks into the namenode. If the namenode does not know them it will instruct the datanode to delete them. But whats the point? If you submit local files to HDF

Re: Scheduling Hive Jobs (Oozie vs. Pentaho vs. something else)

e actual experience using either of these (or something >> else) to schedule Hive jobs? >> >> William Kornfeld >> Baynote >> >> > -- *Jasper Knulst* Consultant *|* Incentro Den Haag Gildeweg 5B 2632 BD Nootdorp The Netherlands *E:* jasper.knu...@incentro.com *T:* +31157640750 *M: *+31619667511 *W:* www.incentro.com [image: Logo Incentro]

Re: FW: Mysql metastore configuration error.

; > mysql> use metastore; > > mysql> show tables; > > > > The table abcd is not there. The table is not being stored in the mysql > metastore db. > > So how come on Hive CLI, when I do “select * from abcd” it shows the data > in the table. And

Managing Hive logging detail (per Hive statement) in /tmp/${user.name}/

Hi, Does anyone know how I can reduce the level of logging of the Hive History file which are created per Hive statement executed in the (default) location of /tmp/${user.name}/ hive_job_log_hdfs_201109122109_675419678.txt. The level of detail is very high by default but I can't find a way to tune

Re: Pentaho issue with hive

Hi *Siddharth, *Did you also install the PHD (Pentaho Distribution for Hadoop) on the cluster side? On my cluster (with PHD installed) there are a lot of Pentaho specific jars in /usr/lib/hadoop-0.20/lib for instance: hive-exec-0.7.0-pentaho-1.0.0.jar hive-jdbc-0.7.0-pentaho-1.0.0.jar hive-met

Re: Pentaho issue with hive

Hi Siddharth, What versions of Hive and Pentaho are you on ? GR. Jasper 2011/9/12 Siddharth Tiwari > Hi users,I face this issue while connecting to hive > started hive using: HIVE_PORT=1 hive --service hiveserver > Starting hive thrift server > > No information after it > netstat shows: >

Re: hi .... any one know how to solv the result run in hive and script hive

nology 日嘉 / Harold Chen > > MSN: chen0...@hotmail.com > > SKYPE: chen0727 > > Mobil: 886-937545215 > > Tel: 886-2-8798-2988 #222**** > > Fax:886-2-8751-5499 > > ** ** > -- Kind Regards \ Met Vriendelijke Groet,

Re: Custom Serde with thorn

have you tried? 2011/5/9 Jasper Knulst > Hi Ankit, > > > I got this in my java mapper code > > String oldSeperator = "�"; //the thorn as java sees it > String newSeperator = "~"; > > In Eclipse it shows as �, which is the standard java way of

Can Hive 0.7 Rebuild partitions ?

Hi, I have a partitioned external table on Hive 0.7. New subfolders are regularly added to the base table HDFS folder. I now have to perform this scan myself and let an external tool create new partitions by generating and firing ALTER TABLE ADD PARTITION commands. Is there an easier way to have

Re: Use spacial chars in hive split

Hi, To my knowledge Hive currently only supports single byte separators in the DELIMITED BY clause. So you can only pick one of the first 128 ASCII characters. Or use a custom Serde to map your data. Jasper Op 18 mei 2011 05:46 schreef "wd" het volgende: > Hi, > > Can I use a spacial char like

Re: Custom Serde with thorn

gt; Ankit > -- Kind Regards \ Met Vriendelijke Groet, Jasper Knulst BI Consultant VLC Den Haag Gildeweg 5B 2632 BD Nootdorp M: +31 (0)6 19 66 75 11 T: +31 (0)15 764 07 50 Skype: jasper_knulst_vlc <>

Re: Custom Serde with thorn

; every platform and software sees it differently. Jasper 2011/5/9 ankit bhatnagar > Hi Jasper, > > How did you find - 'þ' > > My browser shows this - � > > Ankit > -- Kind Regards \ Met Vriendelijke Groet, Jasper Knulst BI Consultant VLC Den Haag

Re: Custom Serde with thorn

writing a test case for deserializer and kind of confused by the > fact it works fine as I pass the line to the deserializer, however when i > run it on hive the line is not split into columns and table inside hive has > thorn as it is. > > Any help would be appreciated. > > Th

Insert Overwrite a partition

Hi, I have this burning question. Normally I would test it myself, but I don't have access to a cluster right now. I wonder what happens if you do a INSERT OVERWRITE TABLE X PARTITION (B="some_value")... after you have renamed all the files (like 00_0 to ren_00_0 etc.) that were in su

Re: Hive SQL Job Tracker

Hi, What you want should be possible. Do you have a hiveserver (hive thrift) up and running? Check out the logs of the hive server Jasper 2011/5/3 Stuart Scott > Hi, > > > > Wonder if anyone can help please? > > We have set up Hive and are running SQL queries against this using a JDBC > client

Re: can I use hive dynamic partition while loading data into tables?

x27;US', type); >> type is the dynamic partition key in the raw data? >> >> This will be very cool! If is supported, I will not have to category the >> raw data according to the type column. >> >> >&g

Hive map task attemps 33+

Hi, Today I had to kill quite a large hive generated MR job. The progress on the mappers was reversed halfway (so actually declining). When I got to the local mapped logs from the TT I saw that there were up to 33 task attempts on the mappers that stalled and it looked like this would go on foreve

Re: Text output separator for Hive Output

Thanks Edward, That worked. 2011/4/7 Edward Capriolo > On Thu, Apr 7, 2011 at 8:38 AM, Jasper Knulst > wrote: > > Hi, > > How would I set the field separator for Hive output to files? > > I see that the default is a space (or tab, don't know exactly) but I &g

Text output separator for Hive Output

Hi, How would I set the field separator for Hive output to files? I see that the default is a space (or tab, don't know exactly) but I would like to use another character to facilitate loading of the output in another system. I already tried set mapred.textoutputformat.separator='~'; But this

Re: Problem Hive table RegexSerDe and Hadoop MR

> jobs > > > > On Apr 5, 2011, at 3:50 PM, Jasper Knulst wrote: > > > Hi(ve), > > > > I created a table like this; > > > > create table testtable (veld1 STRING,veld2 STRING,veld3 STRING) ROW > FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.

Problem Hive table RegexSerDe and Hadoop MR

tion: java.lang.ClassNotFoundException: org.apache.hadoop.hive.contrib.serde2.RegexSerDe" I already copied the hive serde jar (in my case hive-serde-0.7.0-CDH3B4.jar) to $HADOOP_HOME/lib and restarted jobtracker/tasktrackers but that doesn't help. Cheers Jasper -- Kind Regards \ Met Vriendelijke G