Re: Writing hive column headers in 'Insert overwrite query'
r u looking for hive -e "set hive.cli.print.header=true; < query> " > output On Tuesday, January 12, 2016 10:14 PM, Sreenath wrote: Hi All, Is there a way we can write the hive column headers also along with the output when we are overwriting a query's output to an HDFS or local directory ? -- Sreenath S KamathBangalorePh No:+91-9590989106
Re: How to increase the mapper number in my case
check these> mapred.max.split.size > mapred.min.split.size On Friday, January 8, 2016 6:41 PM, Todd wrote: Hi, I have Hadoop (2.6.0, pseudo distributed mode) and Hive (1.2.1) installed on my local machine. I have a table A,its underlying file takes up 8 HDFS blocks. When I run a query like select count(1) from A >From the result, I see only 1 mapper task ,I thought it should be equal to the >block numbers. How could I configure to make the mapper number to be equal to the block numbers? Thanks!
Re: Query in HQL
My bad .. i looked up the code and hive really transforms everything to lower case. On Wednesday, June 3, 2015 10:02 PM, Renuka Be wrote: Hi Ankit, Thanks for your reply. I have tried as you suggested it throws below error. FAILED: ParseException line 1:21 cannot recognize input near 'as' ''CustomerId'' 'from' in selection target Query: select customerid as 'CustomerId' from Customers Regards,Renuka On Thu, Jun 4, 2015 at 12:49 AM, Ankit Bhatnagar wrote: modify to say > 'CustomerId' On Wednesday, June 3, 2015 9:44 PM, Renuka Be wrote: Hi Folks, Here is my query, select customerid as CustomerId from Customers It returns table with column name as 'customerid', ' CustomerId' in the query was not reflected into table. What shall i do to display column name as 'CustomerId'. Regards,Renuka
Re: Query in HQL
modify to say > 'CustomerId' On Wednesday, June 3, 2015 9:44 PM, Renuka Be wrote: Hi Folks, Here is my query, select customerid as CustomerId from Customers It returns table with column name as 'customerid', ' CustomerId' in the query was not reflected into table. What shall i do to display column name as 'CustomerId'. Regards,Renuka
Re: HCatInputFormat combine splits
try thesemapred.max.split.size= mapred.min.split.size= mapreduce.input.fileinputformat.split.maxsize= mapreduce.input.fileinputformat.split.minsize= On Thursday, May 14, 2015 11:04 AM, Pradeep Gollakota wrote: The following property has been to no effect. mapreduce.input.fileinputformat.split.maxsize = 67108864 I'm still getting 1 Mapper per file. On Thu, May 14, 2015 at 10:27 AM, Ankit Bhatnagar wrote: you can explicitly set the split size On Wednesday, May 13, 2015 11:37 PM, Pradeep Gollakota wrote: Hi All, I'm writing an MR job to read data using HCatInputFormat... however, the job is generating too many splits. I don't have this problem when running queries in Hive since it combines splits by default. Is there an equivalent in MR so that I'm not generating thousands of mappers? Thanks,Pradeep
Re: HCatInputFormat combine splits
you can explicitly set the split size On Wednesday, May 13, 2015 11:37 PM, Pradeep Gollakota wrote: Hi All, I'm writing an MR job to read data using HCatInputFormat... however, the job is generating too many splits. I don't have this problem when running queries in Hive since it combines splits by default. Is there an equivalent in MR so that I'm not generating thousands of mappers? Thanks,Pradeep
Re: FAILED: Error in metadata: MetaException
Query is pretty simple CREATE TABLE SRC(ID INT, NAME STRING); Yes HADOOP_HOME is set From: Nitin Pawar mailto:nitinpawar...@gmail.com>> Date: Friday, August 16, 2013 12:49 PM To: Ankit Bhatnagar mailto:ank...@yahoo-inc.com>> Cc: "user@hive.apache.org<mailto:user@hive.apache.org>" mailto:user@hive.apache.org>> Subject: Re: FAILED: Error in metadata: MetaException Not sure what's wrong in your query but have you set variable "HADOOP_HOME" ? its deprecated in hadoop but needed in hive (as per my understanding) On Sat, Aug 17, 2013 at 12:55 AM, Ankit Bhatnagar mailto:ank...@yahoo-inc.com>> wrote: Creating new table From: Nitin Pawar mailto:nitinpawar...@gmail.com>> Reply-To: "user@hive.apache.org<mailto:user@hive.apache.org>" mailto:user@hive.apache.org>> Date: Friday, August 16, 2013 12:17 PM To: "user@hive.apache.org<mailto:user@hive.apache.org>" mailto:user@hive.apache.org>> Subject: Re: FAILED: Error in metadata: MetaException Can you tell us what were you trying to do? create table? or executing a query on a table someone already created ? On Sat, Aug 17, 2013 at 12:32 AM, Ankit Bhatnagar mailto:ank...@yahoo-inc.com>> wrote: Hi folks, FAILED: Error in metadata: MetaException(message:file:/user/hive/warehouse/src is not a directory or unable to create one) drwxrwxrwx - hadoop supergroup 0 2013-08-16 18:31 /user/hive/warehouse/src Is it some permission issue? Ankit -- Nitin Pawar -- Nitin Pawar
Re: FAILED: Error in metadata: MetaException
Creating new table From: Nitin Pawar mailto:nitinpawar...@gmail.com>> Reply-To: "user@hive.apache.org<mailto:user@hive.apache.org>" mailto:user@hive.apache.org>> Date: Friday, August 16, 2013 12:17 PM To: "user@hive.apache.org<mailto:user@hive.apache.org>" mailto:user@hive.apache.org>> Subject: Re: FAILED: Error in metadata: MetaException Can you tell us what were you trying to do? create table? or executing a query on a table someone already created ? On Sat, Aug 17, 2013 at 12:32 AM, Ankit Bhatnagar mailto:ank...@yahoo-inc.com>> wrote: Hi folks, FAILED: Error in metadata: MetaException(message:file:/user/hive/warehouse/src is not a directory or unable to create one) drwxrwxrwx - hadoop supergroup 0 2013-08-16 18:31 /user/hive/warehouse/src Is it some permission issue? Ankit -- Nitin Pawar
FAILED: Error in metadata: MetaException
Hi folks, FAILED: Error in metadata: MetaException(message:file:/user/hive/warehouse/src is not a directory or unable to create one) drwxrwxrwx - hadoop supergroup 0 2013-08-16 18:31 /user/hive/warehouse/src Is it some permission issue? Ankit
Re: Custom Serde with thorn
Hi Jasper, could you please share your MR program. I am not able to grab this character Ankit
Re: Custom Serde with thorn
Hi Jasper, How did you find - 'þ' My browser shows this - � Ankit
Fwd: Custom Serde with thorn
Hi I am facing a weird issue with the file parsing. My log files have a thorn 'þ' as separator. I tried writing a test case for deserializer and kind of confused by the fact it works fine as I pass the line to the deserializer, however when i run it on hive the line is not split into columns and table inside hive has thorn as it is. Any help would be appreciated. Thanks Ankit
Custom Serde with thorn
Hi I am facing a weird issue with the file parsing. My log files have a thorn 'þ' as separator. I tried writing a test case for deserializer and kind of confused by the fact it works fine as I pass the line to the deserializer, however when i run it on hive the line is not split into columns and table inside hive has thorn as it is. Any help would be appreciated. Thanks Ankit
Re: Exception while running hive queries
Hi Please check your hive-site.xml. Looks like issue with the cluster or config. Ankit
Custom Serde
Hi All, Whats the best way to add custom serde jar. looks like add jar says added successfully and I am able to list that jar in hive interface However Create table fails to find the class. Thanks Ankit
Re: csv import with quotes
Hi Larry, add jar -- successful However create table my_table(a string, b string, c string) > row format serde 'com.test.CSVSerde' > stored as textfile; when I execute the query-I get error Error in metadata: Cannot validate serde: com.vantage.hive.CSVSerde FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask Any thoughts Ankit
Custom SerDe
Hi All,
Just wondering if someone can throw some light on the fact how are the
column names and types are provided to Deserializer while creating table.
@Override
public void initialize(Configuration conf, Properties tbl)
throws SerDeException {
}
I understand that above accept that info - but my question will be how is
this initialize() method getting invoked with Custom SerDe while executing
query.
create table my_table(a string, b string, ...)
row format serde '..CustomSerde'
stored as textfile
;
Thanks
-Ankit
csv import with quotes
Hi All, I am planning to join some csv files using hive. referring to the developer guide on wiki quotes are not supported. --- Hive currently use these SerDe classes to serialize and deserialize data: - MetadataTypedColumnsetSerDe: This SerDe is used to read/write delimited records like CSV, tab-separated control-A separated records (sorry, quote is not supported yet.) Did anybody write some Deserializer for the same? Thanks Ankit
