RE: make best use of VCore in Hive

In my opinion, this ultimately becomes a resource balance issue that you'll need to test. You have a fixed amount of memory (although you haven't said what it is). As you increase the number of tasks, the available memory per task will decrease. If the tasks run out of memory, they will either

make best use of VCore in Hive

Hi, Currently we are doing join 2-3 big tables and couple of Left Joins. We are running on 40 node cluster, During query execution, we could see all the memory has been utilized completely (100%), which is perfect. But Number of VCore used are less than 50%. Is there a way to increase usage o

RE: Best way of Unpivoting of hiva table data. Any Analytic function for unpivoting

collect_list(col) will give you an array with all of the data from that column However, the scalability of this approach will have limits. -Original Message- From: mahender bigdata [mailto:mahender.bigd...@outlook.com] Sent: Monday, March 28, 2016 5:47 PM To: user@hive.apache.org Subject:

Best way of Unpivoting of hiva table data. Any Analytic function for unpivoting

Hi, Has any one implemented Unpivoting of Hive external table data. We would like Convert Columns into Multiple Rows. We have external table, which holds almost 2 GB of Data. is there best and quicker way of Converting columns into Row. Any Analytic functions available in Hive to do Unpivoting

Re: Automatic Update statistics on ORC tables in Hive

It would be useful to have a script that could be scheduled as part of a low priority background job, to update stats at least where none are available, and a report in the Hive GUI on stats per table. Encountered a Tez oo memory issue due to the lack of auto updated stats recently. Cheers, An

Re: Automatic Update statistics on ORC tables in Hive

Hi Alan, Thanks for the clarification. I gather you are referring to the following notes in Jira "Given the work that's going on in HIVE-11160 and HIVE-12763 I don't think it makes sense to conti

Re: Automatic Update statistics on ORC tables in Hive

I resolved that as Won’t Fix. See the last comment on the JIRA for my rationale. Alan. > On Mar 28, 2016, at 03:53, Mich Talebzadeh wrote: > > Thanks. This does not seem to be implemented although the Jira says resolved. > It also mentions the timestamp of the last update stats. I do not see

TRYING TO CONNECT TO METASTORE...ASSISTANCE REQUESTED

localhost:/usr/local/hive# bin/hive --service metastore & [5] 27950 root@localhost:/usr/local/hive# Starting Hive Metastore Server SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/usr/local/hive/lib/hive-jdbc-2.0.0-standalone.jar!/org/slf4j/impl/StaticLoggerBin

Re: Automatic Update statistics on ORC tables in Hive

Thanks. This does not seem to be implemented although the Jira says resolved. It also mentions the timestamp of the last update stats. I do not see it yet. Regards, Mich Dr Mich Talebzadeh LinkedIn * https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw