re: Create function using custom UDF

Referring to https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-Create/DropFunction You don’t need to add jar every time when using Permanent Functions. Make sure to create your function following this CREATE FUNCTION [db_name.]function_name AS class_name

Re: Hive - how to skip empty inner fields

Lukas, This is more of a data modelling request. Can you create a staging table which will source all relevant data (on important columns) you desire to query and nothing more than that. It would be more of a redundant job to add a NON NULL clause to subsequent queries, which is not a great

Re: hive on Tez - merging orc files

You can download the branch-0.14 source code from https://github.com/apache/hive/tree/branch-0.14, apply HIVE-9529-branch-1.0.0.patch from https://issues.apache.org/jira/browse/HIVE-9529 and compile it using “mvn clean install -DskipTests -Phadoop-2,dist”. This will generate tar file under

Re: Hive - how to skip empty inner fields

Hi, Thanks for reply. What i needed is custom serde that will serialize results of queries to JSON without empty fields. The serde can be configured using hive.fetch.output.serde (in Hive 10). It works well on console but it does not work in Hue (Hue GUI throws Requested unknown parameter '2'

Re: Hive - how to skip empty inner fields

Solved: The custom serde must extend DelimitedJSONSerDe Lukas On 24.4.2015 10:01, Lukas Nalezenec wrote: Hi, Thanks for reply. What i needed is custom serde that will serialize results of queries to JSON without empty fields. The serde can be configured using hive.fetch.output.serde (in Hive

答复: Create function using custom UDF

The using statement support multiple resources,so you can create function with “using jar HDFS://path/to/A.jar, jar HDFS://path/to/B.jar,…” 发件人: Buntu Dev [mailto:buntu...@gmail.com] 发送时间: 2015年4月24日 15:18 收件人: user@hive.apache.org 主题: Re: Create function using custom UDF Thanks Loudongfeng for

Re: Clear up Hive scratch directory

Thanks Martin Can you also mention steps you did to reclaim the hdfs data from temporary data ? On Fri, Apr 24, 2015 at 12:21 PM, Martin Benson martin.ben...@jaywing.com wrote: Hi All, I just wanted to feedback that it does appear to be safe - I emptied the directory manually, without

Re: java.lang.NullPointerException when using INSERT INTO ORC table with ACID property

Hi, Any one encountered the below error while updating to ORC table with ACID property. java.lang.NullPointerException at org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat.java:265) at

Re: create statement is not working

To be more specific, it happens occasionally . On Fri, Apr 24, 2015 at 10:59 PM, Bhagwan S. Soni bhgwnsson...@gmail.com wrote: there is no syntax error, everything is in place which is required for this query. When I'm executing this statement it won't make any progress. some time it takes

Re: create statement is not working

assuming you have the same issue as HIVE-10242https://issues.apache.org/jira/browse/HIVE-10242 you need to wait for next release (1.2) which includes a fix(which only applies if you are using DBTxnManager) alternatively, you can disable concurrency if that is acceptable Eugene From: Bhagwan

create statement is not working

Hi Hive Users, I'm using Hive's 13th Cloudera version. I'm facing an issue while running any of the create statement. Other operations like DML and drop, alter are working fine. below is the sample statement which i'm trying to run CREATE EXTERNAL TABLE IF NOT EXISTS PROCESS.aggregated_rspns (

Re: create statement is not working

there is no syntax error, everything is in place which is required for this query. When I'm executing this statement it won't make any progress. some time it takes vary long time to complete it and sometime it just got hanged for undefined time. On Fri, Apr 24, 2015 at 10:10 PM, gabriel balan

Re: create statement is not working

Hi What's the error message? if you get FAILED: SemanticException [Error 10072]: Database does not exist: PROCESS then run create schema if not exists process; After that the DDL is accepted just fine on my hive-0.13.1-cdh5.3.0. hth GB On 4/24/2015 12:25 PM, Bhagwan S. Soni wrote: Hi

RE: MapredContext not available when tez enabled

Cause found. I had ... limit 5 in the query. Once I take it out, the query runs fine. -Original Message- From: Frank Luo [mailto:j...@merkleinc.com] Sent: Wednesday, April 22, 2015 10:50 AM To: user@hive.apache.org Subject: RE: MapredContext not available when tez enabled Gopal,

Re: Create function using custom UDF

Thanks Loudongfeng for pointing that out, that has worked after providing the JAR HDFS path. If there are any JARs that this UDF depends on then 'add jar dependent jar' still seem to be needed or updating the hive-site.xml is another option. On Thu, Apr 23, 2015 at 11:43 PM, Loudongfeng

hive on Tez - merging orc files

Hi, Is there anyone using hortonworks sandbox 2.2? I am trying to use hive on Tez on the sandbox. I set the running engine in hive-site.xml to Tez. property namehive.execution.engine/name valuetez/value /property Then I ran the script that alters a table to merge small

Re: hive on Tez - merging orc files

Hi This has been fixed recently https://issues.apache.org/jira/browse/HIVE-9529. Merging is triggered in two different ways. INSERT/CTAS can trigger merging of small files and CONCATENATE can trigger merging of small files. The later had a bug which generated MR task instead of TEZ task which

RE: create statement is not working

Hm It looks OK. However, I personally would not call a database “process”. It could be reserved work somewhere or an application clashing with that word (outside of Hive) hive create database process; OK Time taken: 0.415 seconds hive CREATE EXTERNAL TABLE IF NOT EXISTS

Re: create statement is not working

This is the same which I'm trying to explain, there is nothing wrong in create statement. This issue is related to infrastructure which I'm trying to resolve and i need help if some has already faced this issue earlier. As create statement does not display any log so I'm not able to trace it. On

RE: create statement is not working

That is a valid point I just create a database called table in hive which IMO does not make sense or should not be allowed hive create database table; OK Time taken: 0.488 seconds Now try the same in Sybase given that table is keyword! 1 create database table on datadev01_HDD =

Re: create statement is not working

Do you have hive.support.concurrency=true and long running insert overwrite statements running concurrently? If so, you may be hitting something something like https://issues.apache.org/jira/browse/HIVE-10242 From: Mich Talebzadeh m...@peridale.co.ukmailto:m...@peridale.co.uk Reply-To:

ORC file across multiple HDFS blocks

hi, Guys, I am working on directly READ ORC files from HDFS cluster, and hopefully to leverage HDFS local shortcuit READ ( http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/ShortCircuitLocalReads.html) as much as possible According to ORC design, each ORC file usually contain

RE: create statement is not working

Hi Eugene, What this with regard to the following thread of mine? org.apache.hadoop.hive.ql.lockmgr.LockException: No record of lock could be found, may have timed out Thanks Mich Talebzadeh http://talebzadehmich.wordpress.com Author of the books A Practitioner's Guide to

Re: create statement is not working

No, I was suggesting a possible explanation for why Create Table may be blocked. that was the issue in HIVE-10242, but there were no exceptions – it just appeared as a hang. From: Mich Talebzadeh m...@peridale.co.ukmailto:m...@peridale.co.uk Reply-To: