Re: Hive compaction didn't launch

One more thing. I'm using Apache Storm to stream data in Hive. And when I turned off Storm topology compactions started to work properly. On Tue, Jul 26, 2016 at 6:28 PM, Igor Kuzmenko wrote: > I'm using Hive 1.2.1 transactional table. Inserting data in it via Hive > Streaming API. After some ti

Re: Hive compaction didn't launch

I don’t know the details of how the storm application that streams into Hive works, but this sounds like the transaction batches weren’t getting closed. Compaction can’t happen until those batches are closed. Do you know how you had storm configured? Also, you might ask separately on the stor

Re: Hive compaction didn't launch

Thanks for reply, Alan. My guess with Storm was wrong. Today I get same behavior with running Storm topology. Anyway, I'd like to know, how can I check that transaction batch was closed correctly? On Wed, Jul 27, 2016 at 8:09 PM, Alan Gates wrote: > I don’t know the details of how the storm appl

Hive on spark

Hi All, I need to configure hive cluster based on spark engine (yarn). I already have a running hadoop cluster. Can someone point me to relevant documentation? TIA. Thanks, Mudit

Re: Hive on spark

You mean you want to run Hive using Spark as the execution engine which uses Yarn by default? Something like below hive> select max(id) from oraclehadoop.dummy_parquet; Starting Spark Job = 8218859d-1d7c-419c-adc7-4de175c3ca6d Query Hive on Spark job[1] stages: 2 3 Status: Running (Hive on Spark

Re: Hive on spark

Yes Mich,exactly. Thanks, Mudit From: Mich Talebzadeh Reply-To: Date: Thursday, July 28, 2016 at 1:08 AM To: user Subject: Re: Hive on spark You mean you want to run Hive using Spark as the execution engine which uses Yarn by default? Something like below hive> select max(id) from or

Re: Hive on spark

mudit, this link can guide you - https://cwiki.apache.org/confluence/display/Hive/Hive+on+Spark%3A+Getting+Started Thanks, Karthik On Thu, Jul 28, 2016 at 8:54 AM, Mudit Kumar wrote: > Yes Mich,exactly. > > Thanks, > Mudit > > From: Mich Talebzadeh > Reply-To: > Date: Thursday, July 28, 2016

Re: Hive on spark

Hi, I made a presentation in London on 20th July on this subject:. In that I explained how to make Spark work as an execution engine for Hive. Query Engines for Hive, MR, Spark, Tez and LLAP – Considerations ! See if I can send the pre