Re: ORA-8177 with Hive transactions

2015-09-24 Thread Steve Howard
is that the reader thinks the data hasn't changed, when it has. For our needs, that isn't a huge issue. Are we missing something? Any ideas? Thanks, Steve On Fri, Sep 18, 2015 at 3:39 PM, Steve Howard <stevedhow...@gmail.com> wrote: > I think ROWDEPENDENCIES on an Oracle table al

Re: Transaction deadlocks

2015-09-23 Thread Steve Howard
<user@hive.apache.org> > Date: Wednesday, September 23, 2015 at 8:15 AM > To: "user@hive.apache.org" <user@hive.apache.org> > Subject: Re: Transaction deadlocks > > that’s indeed a problem. > > From: Steve Howard <stevedhow...@gmail.com> > Reply-To:

Re: Transaction deadlocks

2015-09-22 Thread Steve Howard
} else { this.deadlockCnt = 0; this.retryNum = 0; } } Thanks, Steve On Tue, Sep 22, 2015 at 6:22 PM, Eugene Koifman <ekoif...@hortonworks.com> wrote: > What version of Hive are you running? In the current codebase at least, > TxnHandler.checkRetryable()

Re: Transaction deadlocks

2015-09-22 Thread Steve Howard
retryLimit + ") > reached. Last error: " + getMessage(e)); > this.retryNum = 0; > } > else > { > this.deadlockCnt = 0; > this.retryNum = 0; > } > } > > Thanks, > > Steve > > On Tue, Sep 22, 2015 at 6

Transaction deadlocks

2015-09-22 Thread Steve Howard
Thread A… "HiveServer2-Background-Pool: Thread-35" #35 prio=5 os_prio=0 tid=0x7fd150e4 nid=0x2c97 runnable [0x7fd146e0a000] java.lang.Thread.State: RUNNABLE at java.net.SocketInputStream.socketRead0(Native Method) at

Re: mapjoin with left join

2015-09-22 Thread Steve Howard
Hi Gopal/All, Yep, I absolutely understand the limitation of what we are trying to do. We will try the settings you suggested. Thanks, Steve On Tue, Sep 22, 2015 at 1:44 PM, Gopal Vijayaraghavan wrote: > > > select small.* from small s left join large l on s.id

Re: ORA-8177 with Hive transactions

2015-09-18 Thread Steve Howard
e/HIVE-11831> and > https://issues.apache.org/jira/browse/HIVE-11833 that try to address this. > We can do a patch similar to the first one; can you file a JIRA? > > From: Steve Howard <stevedhow...@gmail.com> > Reply-To: "user@hive.apache.org" <user@hive.apache

ORA-8177 with Hive transactions

2015-09-18 Thread Steve Howard
While troubleshooting an issue with transactions shortly after enabling them, I noticed the following in an Oracle trace, which is our metastore for hive... ORA-8177: can't serialize access for this transaction These were thrown on "insert into HIVE_LOCKS..." Traditionally in Oracle, if an

mapjoin with left join

2015-09-11 Thread Steve Howard
We would like to utilize mapjoin for the following SQL construct: select small.* from small s left join large l on s.id = l.id where l.id is null; We can easily fit small into RAM, but large is over 1TB according to optimizer stats. Unless we set hive.auto.convert.join.noconditionaltask.size =

Logging configuration

2015-09-08 Thread Steve Howard
We are having an issue for which we would like to enable logging. The component is org.apache.hadoop.hive.ql.exec.persistence.HybridHashTableContainer. We have added entries to both hive-log4.properties and hive-exec-log4j.properties as follows: log4j.appender.ts=org.apache.log4j.FileAppender

Re: Permission denied error when starting HiveServer2

2015-09-07 Thread Steve Howard
strace is your friend if you are on Linux. Try the following from the shell in which you are starting hive... strace -f -e trace=file service hive-server2 start 2>&1 | grep ermission You should see the file it can't read/write. On Mon, Sep 7, 2015 at 8:13 AM, Daniel Haviv <

Simulating a login trigger in hive

2015-07-21 Thread Steve Howard
We would like to assign a YARN queue to a user upon login. Is there any way to do this outside of the box? If not, is anyone aware of any development effort to do this? It sounds like it would be pretty simple to extend the Connection class to lookup a queue in a custom table in the metastore

Re: hive on tez error

2015-06-15 Thread Steve Howard
What does netstat -anp | grep 9000 show? On Mon, Jun 15, 2015 at 3:47 PM, Sateesh Karuturi sateesh.karutu...@gmail.com wrote: iam using hive 1.0.0 and tez 0.5.2.. whenever iam trying to open the hive getting following error: Exception in thread main java.lang.RuntimeException:

Re: Create ORC Table on Tez Failed

2015-06-04 Thread Steve Howard
I would look in the hive server log, as well as the name node log. That should have a full stack trace. On Jun 4, 2015, at 4:13 AM, Vijay Bhoomireddy vijaya.bhoomire...@whishworks.com wrote: Hi, We are trying to create a Hive ORC Table with Tez. DDL command for the same is

Re: Change hive column size

2015-05-28 Thread Steve Howard
Hi Udit, That JIRA is five years old and applies to hive 0.5. Newer releases are far larger... https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Types Thanks, Steve Sent from my iPad On May 28, 2015, at 9:08 PM, Udit Mehta ume...@groupon.com wrote: Hi, Per this ticket:

Re: Executing HQL files from JAVA application.

2015-03-25 Thread Steve Howard
I would argue that executing arbitrary code from a random remote server has just increased your security scope footprint in terms of the need to control another access point. Purely out of curiosity, is there a compelling architectural reason or environment limitation that results in your need

Re: Delete ORC partition

2015-03-24 Thread Steve Howard
table my_tbl drop partition (date='2014-01-02 00%3A00%3A00.0') ; On Tue, Mar 24, 2015 at 9:35 AM, Steve Howard stevedhow...@gmail.com wrote: Do you have a typo in the partition name? There is a space in the list you have between day and hour, but not in your drop statement. Also %3A is hex

Re: Delete ORC partition

2015-03-23 Thread Steve Howard
Do you have a typo in the partition name? There is a space in the list you have between day and hour, but not in your drop statement. Also %3A is hex for the : character, but you don't have that in you partition name to get dropped. Sent from my iPad On Mar 23, 2015, at 11:46 PM, Megha Garg

Re: Executing HQL files from JAVA application.

2015-03-21 Thread Steve Howard
There are more elegant ways I am sure, but you could also use a java.io.BufferedReader and read the file content into a string and execute it much as you would a hard coded SQL statement in your class. Sent from my iPad On Mar 21, 2015, at 5:04 AM, Amal Gupta amal.gup...@aexp.com wrote: Hi

sqoop import to hive being killed by resource manager

2015-03-12 Thread Steve Howard
Hi All, We have not been able to get what is in the subject line to run. This is on hive 0.14. While pulling a billion row table from Oracle using 12 splits on the primary key, each job continually runs out of memory such as below... 15/03/13 00:22:23 INFO mapreduce.Job: Task Id :