Re: [VOTE] Sponsoring Howl as an Apache Incubator project

:11 PM, Alan Gates wrote: Yes, it adds Input and Output formats for MapReduce and load and store functions for Pig. In the future it we expect it will continue to add more additional layers. Alan. On Feb 3, 2011, at 2:49 PM, John Sichi wrote: But Howl does layer on some additional code

Re: What's official site for howl ?

http://incubator.apache.org/hcatalog/ Howl has been renamed to HCatalog (due to naming conflicts with an existing ow2 project called Howl). Alan. On May 4, 2011, at 7:04 PM, Jeff Zhang wrote: Hi all, Sorry for bothering this mail list. But I only this mail list may relate with howl.

Re: Performance: hive+hbase integration query against the row_key

On Sep 11, 2012, at 7:00 AM, bharath vissapragada wrote: Hey, Hive does all kinds of parsing , metadata lookups, query tree building and stuff before executing the query. Not sure if this all was included in those 36 seconds ! Also what hive does is, it builds a scan object with

Re: [VOTE] Amend Hive Bylaws + Add HCatalog Submodule

Most excellent. I'll start the vote in the HCatalog PPMC to approve this, and assuming that passes I'll then start a vote in the IPMC per the guidelines at http://incubator.apache.org/guides/graduation.html#subproject Alan. On Feb 4, 2013, at 2:27 PM, Carl Steinbach wrote: The following

Re: Does Hive support collation?

No, Hive does not support collation at this time. Alan. On Mar 18, 2013, at 9:09 PM, Jon Klein wrote: Hi, I'm using Hive for dealing with some international characters. Does Hive have collation support so I can specify case sensitivity, ascent sensitivity or width sensitity for string

Fwd: Hadoop In Seoul 2013 Conference Calls For Speakers

Begin forwarded message: From: Edward J. Yoon edwardy...@apache.org Date: May 21, 2013 1:29:06 AM PDT To: gene...@hadoop.apache.org Subject: Hadoop In Seoul 2013 Conference Calls For Speakers Reply-To: gene...@hadoop.apache.org Hi, I'm planning the Hadoop In Seoul 2013 Open

Fwd: DesignLounge @ HadoopSummit

Begin forwarded message: From: Eric Baldeschwieler eri...@hortonworks.com Date: June 11, 2013 10:46:25 AM PDT To: common-...@hadoop.apache.org common-...@hadoop.apache.org Subject: DesignLounge @ HadoopSummit Reply-To: common-...@hadoop.apache.org Hi Folks, We thought we'd try

Re: Hive Architecture - Execution on nodes

On Jul 18, 2013, at 1:40 PM, Tzur Turkenitz wrote: Hello, Just finished reading the Hive-Architecture pdf, and failed to find the answers I was hoping for. So here I am, hoping this community will shed some light. I think I know what the answers will be, I need that bolted down and

Re: Hive-0.11.0 HCatalog configuration

On Jul 23, 2013, at 1:03 AM, nabhajit wrote: Hi, I am trying to configure Hcatalog , which is now part of Hive-0.11.0. Do I have to make changes to the permission of the following files? $HCAT_HOME/bin/hcat and $HCAT_HOME/sbin/webhcat-server.sh Yes, this is a known issue and has

Re: Large Scale Table Reprocess

A table can definitely have partitions with different input formats/serdes. We test this all the time. Assuming your old data doesn't stay for ever and most of your queries are on more recent data (which is usually the case) I'd advise you to not reprocess any data, just alter the table to

Re: Large Scale Table Reprocess

appreciate it :) On Fri, Jul 26, 2013 at 3:29 PM, Alan Gates ga...@hortonworks.com wrote: A table can definitely have partitions with different input formats/serdes. We test this all the time. Assuming your old data doesn't stay for ever and most of your queries are on more recent

Re: joins in hcatalog

HCatalog itself does not perform joins. But you can do the joins in Pig or MapReduce using tables read via HCatalog. Alan. On Aug 2, 2013, at 1:17 PM, Adeel Qureshi wrote: is it possible to perform joins via hcatalog api

Re: hive 0.11 on MRv1 Vs. YARN

Hive 0.11 does run faster on YARN than MR1, but that's due solely to YARN being faster than MR1. Hive 0.11 doesn't take any advantage of YARN. Lots of work is going on to change that so that future versions of Hive will take advantage of YARN (see all the work in the Tez branch for example).

Re: DISCUSS: Hive language manual to be source control managed

I'm definitely +1 on moving the language doc to source control. Being able to have versions of the language manual that match versions of Hive would be very valuable. Alan. On Sep 1, 2013, at 2:39 PM, Edward Capriolo wrote: Over the past few weeks I have taken several looks over documents

Re: DISCUSS: Hive language manual to be source control managed

On Sep 4, 2013, at 10:38 PM, Lefty Leverenz wrote: snip Let me split another hair: my converted files actually belong to Hortonworks, and Alan deserves credit for assigning that task to me (thanks, Alan). I'd better return the files so they can contribute them. I don't think we

Re: How to prevent user drop table in Hive metadata?

See https://issues.apache.org/jira/browse/HIVE-5837 for a JIRA addressing this. Also, you can use the StorageBasedAuthorizationProvider in Hive, which bases metadata security on file security. So if the user doesn't have permissions to remove the directory that stores the table data, they

Re: Help on loading data stream to hive table.

You shouldn’t need to write each record to a separate file. Each Storm bolt should be able to write to it’s own file, appending records as it goes. As long as you only have one writer per file this should be fine. You can then close the files every 15 minutes (or whatever works for you) and

Re: [DISCUSS] Proposed Changes to the Apache Hive Project Bylaws

One other benefit in rotating chairs is that it exposes more of Hive’s PMC members to the board and other Apache old timers. This is helpful in getting better integrated into Apache and becoming a candidate for Apache membership. It is also an excellent education in the Apache Way for those

Re: Help on loading data stream to hive table.

with that partition. How can i share the file writer across different bolt? should I instruct data within the same partition to the same bolt? Thanks, Chen On Fri, Jan 3, 2014 at 3:27 PM, Alan Gates ga...@hortonworks.com wrote: You shouldn’t need to write each record to a separate file. Each

Re: Using Hive metastore as general purpose RDBMS

HCatalog is definitely not designed for this purpose. Could you explain your use case more fully? Is this indexing for better query planning or faster file access? If so, you might look at some of the work going on in ORC, which is storing indices of its data in the format itself for these

Re: hive13 release date

Harish, who has volunteered to be release manager for 0.13 has stated he’d like to branch this week and then give a couple of weeks to stabilize the branch. So that would put the release somewhere in early March. Alan. On Feb 14, 2014, at 11:45 AM, Suhas Satish suhas.sat...@gmail.com wrote:

Re: Hive 13

We do not have a firm release date yet. The branch has been cut. I think Harish said he’d like to have a first RC early next week. It usually takes 1 to 2 weeks after the first RC, depending on any show stoppers found in it, etc. Alan. On Mar 19, 2014, at 6:50 AM, Bryan Jeffrey

Re: Meta data tables - Hive

Hive does not have a traditional SQL information schema. Instead it uses MySQL style show/describe. So it has show tables, etc. See https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-Show Alan. On Apr 21, 2014, at 7:10 AM, Ravi Prasad

Re: largest table last in joins

Join ordering is not yet part of the Hive optimizer. There is integration work being done with the Optiq framework that will address this, but it is not complete yet. Hopefully at least an initial integration will be available in the next Hive release. Alan. On May 2, 2014, at 5:36 AM,

Re: ACID Support in Hive

As noted in the JIRA, we hope to have it ready in the next release of Hive. As to when that will be, it is up to the Hive community. Usually Hive does a release every 4-6 months, with the last one being in April so there will probably be another in the fall. Alan. On Jun 16, 2014, at 9:04

Re: bug in hive

Up until Hive 0.13 locks in Hive were really advisory only, since as you note any user can remove any other user's lock. In Hive 0.13 a new type of locking was introduced, see https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions#HiveTransactions-LockManager This new locking is

Re: bug in hive

? Moreover hive does not release locks in 0.10 when hive session is killed . User has to explicitly unlock a table. Can i specify any sort of max expiry time while taking a lock. Thanks Shushant Alan Gates mailto:ga...@hortonworks.com September 20, 2014 at 7:41 Up until Hive 0.13 locks in Hive were

Re: Oracle backend for Hive Metastore

What version of Hive are you using? Did you explicitly create the tables in the metastore via the Oracle script or depend on DataNucleus to do it for you? Alan. Rahul Channe mailto:drah...@googlemail.com September 22, 2014 at 10:47 Hi All, I am using oracle as hive metastore. I could see

Re: [ANNOUNCE] New Hive PMC Member - Alan Gates

Thanks everyone for the congratulations. I look forward to being able to serve Hive as part of the PMC. Alan. Carl Steinbach mailto:c...@apache.org October 27, 2014 at 15:38 I am pleased to announce that Alan Gates has been elected to the Hive Project Management Committee. Please join me

Re: UPDATE in Hive -0.14.0

You have to use ORC as the table format (it's the only format that supports update and delete at this point), and you have to explicitly mark the table as transactional. You must also bucket the table. For example: create table HiveTest (...) clustered by (_col_) into _num_ buckets stored

Re: hive transactions

Usually this means the metastore database was not properly upgraded. You can check this by logging into your RDBMS and listing all the tables. If you see a table named TXNS then the proper tables exist. Did you install Hive 0.14 fresh or upgrade from a previous version? If you upgraded did

Re: Where is the base directory of a transaction table?

The base directories will only exist after compaction has run. When you say you set hive.compactor.initiator.on (=true I hope) and hive.compactor.worker.threads, did you did that in your metastore process? If so, did you restart the metastore after changing the config values? Alan. vic0777

Re: UPDATE implementation

Yes. Alan. Brian Jeltema mailto:brian.jelt...@digitalenvoy.net December 3, 2014 at 11:02 I’m anticipating using UPDATE statements in Hive 0.14. In my use case, I may need to perform 30 or so updates at a time. Will each UPDATE result in an MR job doing a full partition scan? Brian -- Sent

Re: Where is the base directory of a transaction table?

/configuration At 2014-12-03 09:59:34, Alan Gates ga...@hortonworks.com wrote: Alan Gates mailto:ga...@hortonworks.com December 2, 2014 at 17:59 The base directories will only exist after compaction has run. When you say you set hive.compactor.initiator.on (=true I hope

Re: hive transactions

From: Alan Gates mailto:ga...@hortonworks.com Sent: ‎12/‎3/‎2014 9:51 AM To: user@hive.apache.org mailto:user@hive.apache.org Subject: Re: hive transactions Usually this means the metastore database was not properly upgraded. You can check

Re: relationship between a hive query and a tez dag

Several. Each Hive query becomes one Tez DAG. Alan. Xiaoyong Zhu mailto:xiaoy...@microsoft.com December 15, 2014 at 22:55 Hi experts I know that Hive could be run on Tez in a DAG format -- if I have a complex Hive script which is consisted of several Hive queries, will it be translated to

Re: relationship between a hive query and a tez dag

? Xiaoyong *From:*Alan Gates [mailto:ga...@hortonworks.com] *Sent:* Wednesday, December 17, 2014 5:27 AM *To:* user@hive.apache.org *Subject:* Re: relationship between a hive query and a tez dag Several. Each Hive query becomes one Tez DAG. Alan. -- Sent with Postbox http://www.getpostbox.com

Re: Hive 0.14 SQL Inserts Complex Data Types

Hive does not support literals for complex types, so it is not possible to use them in INSERT...VALUES clauses. Alan. Skyler Beckman mailto:skyler.beck...@livetv.net December 17, 2014 at 9:59 With 0.14 is it possible to use the INSERT…VALUES statement to insert data into a table with

Re: relationship between a hive query and a tez dag

tb3; And this script will be executed exactly in 1 2 3 4 order even 2 and 3 do not have relationship with each other and 4 does not have relation with all query 1,2,3? Xiaoyong *From:*Alan Gates [mailto:ga...@hortonworks.com] *Sent:* Thursday, December 18, 2014 8:05 AM *To:* Xiaoyong Zhu

Re: unsubscribe

All, if you wish to unsubscribe send email to user-unsubscr...@hive.apache.org Alan. Krishnan K mailto:kkrishna...@gmail.com February 16, 2015 at 17:00 unsubscribe Alex.Wang mailto:alex.w.y.w...@gmail.com February 16, 2015 at 16:10 unsubscribe Best regards. Santhosh Thomas

Re: Hive 14 performance and scalability?

Mike, I've tested it with inserts, updates, and deletes in the 100k rows per partition range, and with multiple but not hundreds of users. The thing to keep in mind is the transactional changes don't affect tables that you don't explicitly mark as transactional, so this shouldn't affect

Re: Hive 0.14 SQL Inserts Complex Data Types

system. Thank You. Skyler Beckman mailto:skyler.beck...@livetv.net December 19, 2014 at 8:50 Thanks *From:*Alan Gates [mailto:ga...@hortonworks.com] *Sent:* Friday, December 19, 2014 11:35 AM *To:* user@hive.apache.org *Subject:* Re: Hive 0.14 SQL Inserts Complex Data Types Hive does

Re: Does Hive 1.0.0 still support commandline

Hive CLI and HiveServer2/beeline are both in Hive 1.0. Alan. DU DU mailto:will...@gmail.com February 9, 2015 at 8:54 According to the release note of Hive 1.0.0, the HiveServer1 is removed. Can we still use command line in 1.0.0? -- Thanks, Dayong

Re: writing to partitions with HCatWriter

This sounds like a bug in the HCatWriter. You should file a JIRA so we can track it. Alan. Nathan Bamford mailto:nathan.bamf...@redpoint.net February 13, 2015 at 13:50 Hi all, I'm using HCatWriter in a java program to write records to a partitioned Hive table. It works great, but I

Re: SHOW COMPACTIONS fail with remote metastore

When you say but run sql the job is ok. what do you mean? Which version of Hive are you on? Were there active compactions when you did this? Alan. r7raul1...@163.com mailto:r7raul1...@163.com March 17, 2015 at 23:56 15/03/18 22:54:46 [main]: WARN metastore.RetryingMetaStoreClient:

Re: How to specify column list in hive insert into query?

This feature is not available in Hive 1.1. It will be added in 1.2 (it has already been checked into trunk). In 1.1 and previous your values clause has to contain a value for every column in the table in the proper order. So you would change your insert below to be insert into person

Re: insert table error

What Hive version are you using? What error message did you get? Alan. zhangjp mailto:smart...@hotmail.com March 13, 2015 at 4:13 case fail CREATE TABLE students (name VARCHAR(64), age INT, gpa DECIMAL(3, 2)) CLUSTERED BY (age) INTO 2 BUCKETS STORED AS ORC; INSERT INTO TABLE students

Re: Updates/deletes with OrcRecordUpdater

Your table definition looks fine, and no you shouldn't service the recIdField in the table itself. Without seeing your writing code it's hard to know why you're hitting this, but some info that may be of use. Hive itself uses a pseudo column to store the recIdInfo when it reads an ACID row

Re:

If you want off of the list send email to user-unsubscr...@hive.apache.org Alan. jake lawson mailto:jacobj...@gmail.com March 25, 2015 at 15:45 Stop emailing me

Re: Delete ORC partition

Are you getting an error or does the partition just not get deleted? If you get an error message can you share it? What version of Hive are you using? Alan. Megha Garg mailto:megha.ga...@gmail.com March 23, 2015 at 5:43 Hi, I am new to hive. I have created one ORC table with partitioning

Re: column selection for insert

Specifying which columns in the target table to insert into (emp in your case) is not supported in 0.14. You have to insert into all the columns in the table and they have to match the order of items in the select or values clause. This will be supported in 1.2. Alan. Srinivas Thunga

Re: Reading tables will lock entire database in Hive 0.13

What lock manager are you using? Alan. Jim Green mailto:openkbi...@gmail.com February 25, 2015 at 17:24 Hi Team, Here is old thread about this issue: http://qnalist.com/questions/5091386/doubt-about-locking-mechanism-in-hive I met the same issue that when reading one table in session A, we

Re: Hive optimiser seems not to use for a covered query

Hive's optimizer never uses the indexes. Indexes aren't fully implemented in Hive. Alan. Mich Talebzadeh mailto:m...@peridale.co.uk March 27, 2015 at 1:33

Re: Adding update/delete to the hive-hcatalog-streaming API

The missing piece for adding update and delete to the streaming API is a primary key. Updates and deletes in SQL work by scanning the table or partition where the record resides. This is assumed to be ok since we are not supporting transactional workloads and thus update/deletes are assumed

Re: Adding update/delete to the hive-hcatalog-streaming API

when issuing the deletes/updates? Apologies if I'm misunderstanding something. Thanks - Elliot. On Thursday, March 26, 2015, Alan Gates alanfga...@gmail.com mailto:alanfga...@gmail.com wrote: Alan Gates mailto:alanfga...@gmail.com March 26, 2015 at 14:48 The missing piece for adding update

Re: Adding update/delete to the hive-hcatalog-streaming API

than streaming. I am not aware of anyone working on it at the moment. Alan. Thanks - Elliot. On Thursday, March 26, 2015, Alan Gates alanfga...@gmail.com mailto:alanfga...@gmail.com wrote: Alan Gates mailto:alanfga...@gmail.com March 26, 2015 at 15:30 Are you saying that when the records

Re: how to set column level privileges

Column level permissions was added to Hive default authorization in HIVE-5837. That is why the TBL_COL_PRIV tables exists in the metastore. The problem with default auth is it isn't really secure, as anyone can grant anybody (including themselves) any privilege. But Allen is correct that

Re: org.apache.hadoop.hive.ql.lockmgr.LockException: No record of lock could be found, may have timed out

What lock or transaction manager are you using? Alan. Mich Talebzadeh mailto:m...@peridale.co.uk April 23, 2015 at 8:19 Hi all, Trying to do a direct load from RDBMS to Hive (not using Sqoop). It sends data in files of rows at a time. Concurrency is enabled. Using Oracle database as

Re: Transactional table read lifecycle

Whether you obtain a read lock depends on the guarantees you want to make to your readers. Obtaining the lock will do a couple of things your uses might want: 1) It will prevent DDL statements such as DROP TABLE from removing the data while they are reading it. 2) It will prevent the compactor

Re: Orc file and Hive Optimiser

Mich Talebzadeh mailto:m...@peridale.co.uk April 19, 2015 at 12:32 Finally this is more of a speculative question. If we have ORC files that provide good functionality, is there any reason why one should deploy a columnar database such as Hbase or Cassandra If Hive can do the job as well?

Re: ORC file across multiple HDFS blocks

No, you don't want to be designing ORC files to not cross block boundaries. Engines in Hadoop (MapReduce, Tez, etc.) are all built to handle the fact that files tend to cross blocks and hence nodes. There is value in lining up stripe size and HDFS block size so that your stripes don't

Re: ACID ORC file reader issue with uncompacted data

() returns: warehouse/test_table/continent=Asia/country=India Partition keys derived as: 'continent=Asia' (INCORRECT) Cheers - Elliot. On 30 April 2015 at 17:40, Alan Gates alanfga...@gmail.com mailto:alanfga...@gmail.com wrote: Are you using OrcInputFormat.getReader to get a reader

Re: External Table with unclosed orc files.

. What information should I capture to help with this issue? From: Alan Gates ga...@apache.org mailto:ga...@apache.org Reply-To: user@hive.apache.org mailto:user@hive.apache.org user@hive.apache.org mailto:user@hive.apache.org, ga...@apache.org mailto:ga...@apache.org ga...@apache.org mailto:ga

Re: Hive support for concurrency with Oracle Metastore

All locks in Hive are on database, table, or partition level. There are no row level locks. When using DbTxnManager the locking is chosen to be as fine grained as possible (ie only partitions are locked when we can a priori know the partitions that will be used in the query). Alan. Mich

Re: External Table with unclosed orc files.

It will fail. Orc writes info in the footers that are required to properly read the file. If close hasn't been called, then that footer hasn't been written yet. Alan. Grant Overby (groverby) mailto:grove...@cisco.com April 14, 2015 at 20:46 What will Hive do if querying an external table

Re: External Table with unclosed orc files.

Grant Overby (groverby) wrote: Thanks for the link to the hive streaming bolt. We rolled our own bolt many moons ago to utilize hive streaming. We’ve tried it against 0.13 and 0.14 . Acid tables have been a real pain for us. We don’t believe they are production ready. At least in our use

Re: admin user in hive

Use the new SQL standard authorization rather than the default authorization. See https://cwiki.apache.org/confluence/display/Hive/SQL+Standard+Based+Hive+Authorization Alan. Megha Garg mailto:megha.ga...@gmail.com April 6, 2015 at 4:08 Hi, I want to enable authentication+authorization on

Re: A simple insert stuck in hive

If you're seeing it list progress (or attempted progress) as here, this isn't a locking issue. All locks are obtained before the job is submitted to Hadoop. Alan. Mich Talebzadeh mailto:m...@peridale.co.uk April 7, 2015 at 14:09 Hi, Today I have noticed the following issue. A simple

Re: HIve Joins vs Pig Joins

These results are fairly old. Both Hive and Pig have changed significantly since then. Many of Hive's internal operators have been rewritten. Both can now run over Tez or Spark. So at this point it's hard to say which is better. The best option is to develop some tests based on your use

Re: delta file compact take no effect

Which version of Hive are you running? A number of deadlock issues were resolved in HIVE-10500 which was released in Hive 1.2. Based on your log it appears it recovered properly from the deadlocks and did manage to compact. Alan. r7raul1...@163.com mailto:r7raul1...@163.com June 17, 2015

Re: delta file compact take no effect

See https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions#HiveTransactions-Configuration Compaction is initiated by the thrift metastore server. You need to set the values labeled metastore in the above page in the hive-site.xml for your metastore server. Alan.

Re: delta file compact take no effect

Is the config you give on your metastore or your client? The worker thread and initiator must be started on the metastore. Alan. r7raul1...@163.com mailto:r7raul1...@163.com June 16, 2015 at 22:38 Any help?

Re: Error communicating with metastore

Did you run the hive metastore upgrade scripts for your oracle instance? This error message usually means the transaction related tables have not been created in your database. Somewhere in your distribution there should be a set of upgrade scripts. Look for scripts of the form:

Re: Hive Concurrency support

Are you using a lock manager, and if so which one? I believe the ZooKeeper lock manager does not allow simultaneous writes. The lock manager that comes with the DbTxnManager does, but you can't use that without also using transactions. Alan. Suyog Parlikar mailto:suyogparli...@gmail.com

Re: Unsubscribe

To unsbuscribe send email to user-unsubscr...@hive.apache.org Alan. Nurdan Ahat mailto:nurdan.a...@gmail.com August 24, 2015 at 6:38

Re: Error communicating with metastore

and able to see the table TXNS. I've also removed the hdfs location /user/hive/warehouse and created a fresh one. But still I'm facing this issue. Alan Gates mailto:alanfga...@gmail.com August 3, 2015 at 8:29 Did you run the hive metastore upgrade scripts for your oracle instance? This error

Re: Error communicating with metastore

(TIOStreamTransport.java:127) ... 24 more Let me know if there is anything to be taken care in the configuration or setup. Alan Gates mailto:alanfga...@gmail.com August 4, 2015 at 16:10 Ok, the next step is to look at the logs from your Hive metastore server and see exactly what's happening

Re: Unsigned Data Type Support

No, we don't natively support this in Hive. Alan. saurabh mailto:mpp.databa...@gmail.com July 23, 2015 at 3:27 Hi All, Is there any data type in hive which only stores positive values and converts the negative values to ZERO? This can be handled while data processing e.g. CASE statement or

Re: Question about hive-jdbc

The way to keep track of when things are getting done in Hive is to check the JIRA, https://issues.apache.org/jira/browse/HIVE I'm not aware of anyone working on those issues at the moment, but a search of the JIRA will tell you if anyone has filed a bug on it. Alan. Hafiz Mujadid

Re: clarification please

Ashok Kumar October 28, 2015 at 22:43 hi gurus, kindly clarify the following please * Hive currently does not support indexes or indexes are not used in the query Mostly true. There is a create index, but Hive does not use the resulting index by

Re: insert timestamp values in Hive

Actually, for INSERT VALUES you don't have to have a transactional table (you do to use UPDATE or DELETE). So I would expect this to work as is. What happens if you do: create table foo (x int); insert into foo values (5); select * from foo; Do you get 5 or null? This will tell whether the

Re: Locking when using the Metastore/HCatalog APIs.

Answers inlined. Elliot West October 22, 2015 at 6:40 I notice from the Hive locking wiki page that locks may be acquired for a range of HQL DDL operations. I wanted to know how the locking scheme mapped

Re: hive locking doubt

You are correct that DbTxnManager does not support the explicit locking of tables. Instead it obtains locks based on SQL statements that are being executed. If you use the DummyTxnManager (the default) and set concurrency to true and the lock manager to ZooKeeperHiveLockManager then your

Re: ORC tables loading

The reads and writes both happen in parallel, so as more nodes are available for read and write, at least in this case, the time stays roughly the same. Alan. James Pirz November 16, 2015 at 21:23 Hi, I am using Hive 1.2 with ORC tables on Hadoop 2.6 on a

Re: View definition information

It should certainly be possible. Can you file a JIRA adding this as a new feature, and if you're so inclined feel free to contribute a patch to add this. Alan. Rachna Jotwani Bakhru October 14, 2015 at 16:47 We are currently using the HCatalog API to get the Hive

Re: truncating tables via hcatalog api?

That's correct, HCatClient doesn't provide that feature at this time. It would be easy enough to add if you want to provide a patch for it. Alan. Nathan Bamford October 6, 2015 at 12:14 Hello all, The product I work on using the HCatalog api

Re: Help me, please

Seems like the best place to start would be with email to the infra team. Alan. Navis Ryu October 7, 2015 at 0:29 After long vacation, I've found my account on apache JIRA is disabled. I think I've logged in somewhere and forgot to logout or it was just hacked

Re: HiveMetaStoreClient

Accessing the RDBMS directly isn't recommended. We don't view the table layout as part of the public API and sometimes change things between releases. With the thrift API we at least try to keep it backwards compatible. Alan. Dr Mich Talebzadeh mailto:m...@peridale.co.uk August 26, 2015

Re: HiveMetaStoreClient

Yes, sorry, I wasn't clear. As others have stated, HiveMetaStoreClient uses the thrift API. So using that should be fine. Alan. Subroto Sanyal mailto:ssan...@datameer.com August 26, 2015 at 10:07 Hi Jerrick, HiveMetastoreClient will be using thrift communication internally via the

Re: transactional table + vectorization + where = bug

I am not aware of this issue. Please file a JIRA, and if it does turn out to be a duplicate we can mark it as such. Alan. Furcy Pin September 19, 2015 at 2:36 Hi, We bumped into a bug when using vectorization on a transactional table. Here is a minimal

Re: How to register permanent function during hive thrift server is running

No restart of the thrift service should be required. Alan. Todd December 3, 2015 at 3:12 Hi, I am using Hive 0.14.0, and have hive thrift server running.During its running, I would use “create function” to add a permanent function, Does hive support this **without

Re: Hive partition load

Yes, you can load different partitions simultaneously. Alan. Suyog Parlikar December 17, 2015 at 5:02 Hello everyone, Can we load different partitions of a hive table simultaneously. Is there any locking issues in that if yes what are they? Please find

Re: Indexes in Hive

The issue with this is that HDFS lacks the ability to co-locate blocks. So if you break your columns into one file per column (the more traditional column route) you end up in a situation where 2/3 of the time only one of your columns is being locally read, which results in a significant

Re: Immutable data in Hive

Traditionally data in Hive was write once (insert) read many. You could append to tables and partitions, add new partitions, etc. You could remove data by dropping tables or partitions. But there was no updates of data or deletes of particular rows. This was what was meant by immutable.

Re: Difference between ORC and RC files

ORC offers a number of features not available in RC files: * Better encoding of data. Integer values are run length encoded. Strings and dates are stored in a dictionary (and the resulting pointers then run length encoded). * Internal indexes and statistics on the data. This allows for more

Re: Attempt to do update or delete using transaction manager that does not support these operations. (state=42000,code=10294)

Also note that transactions only work with MR or Tez as the backend. The required work to have them work with Spark hasn't been done. Alan. Mich Talebzadeh December 22, 2015 at 9:43 Dropped and created table tt as follows: drop table if exists tt; create table

Re: Attempt to do update or delete using transaction manager that does not support these operations. (state=42000,code=10294)

s email is virus free, therefore neither Peridale Ltd, its subsidiaries nor their employees accept any responsibility. *From:*Alan Gates [mailto:alanfga...@gmail.com] *Sent:* 22 December 2015 20:39 *To:* user@hive.apache.org *Subject:* Re: Attempt to do update or delete using transactio

Re: Loop if table is not empty

Have you looked at the new procedural HPL/SQL available in recent Hive? If you are using an older version of Hive you can check out hplsql.org, which allows you to install it separately. Alan. Thomas Achache December 28, 2015 at 2:30 Hi everyone, I am running

Re: Query performance correlated to increase in delta files?

Are you running the compactor as part of your metastore? It's occasionally compacts the delta files in order to reduce read time. See https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions for details. Alan. Sai Gopalakrishnan November

Re: Spark support for update/delete operations on Hive ORC transactional tables

This JIRA https://issues.apache.org/jira/browse/HIVE-12366 moved the heartbeat logic from the engine to the client. AFAIK this was the only issue preventing working with Spark as an engine. That JIRA was released in 2.0. I want to stress that to my knowledge no one has tested this combination

  1   2   >