Re: migrate hive cli to beeline
Should be reasonable for me. You should use hive for version as new as possible, Memory leak issue was included in hive's jira. For our production, hiveserver2 of 2.1.1 could run for log lifetime. 2017-11-22 15:04 GMT+08:00 游垂凯 <yo...@jpush.cn>: > Hello everyone: > Recently,I want to migrate hive cli to beeline，but i'm reffered to > cloudera official site about hive beeline(https://www.cloudera. > com/documentation/enterprise/5-8-x/topics/cdh_ig_hive_ > install.html#concept_alp_4kl_3q), it describe that the beeline cli should > use a heap size at least 2GB, hiveserver2 and the hive metastore require > sufficient memory to run correctly. > I want to know that the hiveserver and hive metastore cloudera > recommand memory is really true? Thanks for your help. > Number of Concurrent ConnectionsHiveServer2 Heap Size Recommended RangeHive > Metastore Heap Size Recommended Range > > Up to 40 concurrent connections > > Cloudera recommends splitting HiveServer2 into multiple instances and > load-balancing once you start allocating over 16 GB to HiveServer2. This > reduces the impact of Java garbage collection on active processing by the > service. > 12 - 16 GB 12 - 16 GB > Up to 20 concurrent connections 6 - 12 GB 10 - 12 GB > Up to 10 concurrent connections 4 - 6 GB 4 - 10 GB > One connection 4 GB 4 GB由于本人英文太弱，不过我已经在补习英语过程中。我把我想描述的英文内容用中文再说一遍吧。 > 最近，我想从hive cli迁移使用beeline，但是看到cloudare官网这样描述hiveserver2 > https://www.cloudera.com/documentation/enterprise/5-8- > x/topics/cdh_ig_hive_install.html#concept_alp_4kl_3q > <https://www.cloudera.com/documentation/enterprise/5-8-x/topics/cdh_ig_hive_install.html#concept_alp_4kl_3q，不知道官网关于hiveserver2以及hive> > 不知道官网写的并发连接数与hivesever2以及metastore内存配置是否是真的？不知道有 > 没有人测试过，因为我听有的开发同事说，hiveserve2开启后时间久了因性能原因可能需要重启。希望得到帮助谢谢。 > > > youck > -- 王海华
Re: migrate hive cli to beeline
Hello, As the table explains, it really depends on the number of concurrent connections you will have on your HiveServer2, but I confirm that you can experience OOM failures on either the HiveServer2 or the Metastore if too many concurrent queries are run simultaneously. For larger clusters, as Cloudera recommends, you will have to use multiple HiveServer2 instances with load-balancing. Performances on the Metastore's SQL backend can also become a bottleneck at some point. Also, when using beeline to retrieve large amount of data as your query output, don't forget to use this option, or it will hang and ultimately fail: "beeline --incremental=true" 2017-11-22 8:04 GMT+01:00 游垂凯 <yo...@jpush.cn>: > Hello everyone: > Recently,I want to migrate hive cli to beeline，but i'm reffered to > cloudera official site about hive beeline(https://www.cloudera. > com/documentation/enterprise/5-8-x/topics/cdh_ig_hive_ > install.html#concept_alp_4kl_3q), it describe that the beeline cli should > use a heap size at least 2GB, hiveserver2 and the hive metastore require > sufficient memory to run correctly. > I want to know that the hiveserver and hive metastore cloudera > recommand memory is really true? Thanks for your help. > Number of Concurrent ConnectionsHiveServer2 Heap Size Recommended RangeHive > Metastore Heap Size Recommended Range > > Up to 40 concurrent connections > > Cloudera recommends splitting HiveServer2 into multiple instances and > load-balancing once you start allocating over 16 GB to HiveServer2. This > reduces the impact of Java garbage collection on active processing by the > service. > 12 - 16 GB 12 - 16 GB > Up to 20 concurrent connections 6 - 12 GB 10 - 12 GB > Up to 10 concurrent connections 4 - 6 GB 4 - 10 GB > One connection 4 GB 4 GB由于本人英文太弱，不过我已经在补习英语过程中。我把我想描述的英文内容用中文再说一遍吧。 > 最近，我想从hive cli迁移使用beeline，但是看到cloudare官网这样描述hiveserver2 > https://www.cloudera.com/documentation/enterprise/5-8- > x/topics/cdh_ig_hive_install.html#concept_alp_4kl_3q > <https://www.cloudera.com/documentation/enterprise/5-8-x/topics/cdh_ig_hive_install.html#concept_alp_4kl_3q，不知道官网关于hiveserver2以及hive> > 不知道官网写的并发连接数与hivesever2以及metastore内存配置是否是真的？不知道有 > 没有人测试过，因为我听有的开发同事说，hiveserve2开启后时间久了因性能原因可能需要重启。希望得到帮助谢谢。 > > > youck >
migrate hive cli to beeline
Hello everyone: Recently,I want to migrate hive cli to beeline，but i'm reffered to cloudera official site about hive beeline(https://www.cloudera.com/documentation/enterprise/5-8-x/topics/cdh_ig_hive_install.html#concept_alp_4kl_3q), it describe that the beeline cli should use a heap size at least 2GB, hiveserver2 and the hive metastore require sufficient memory to run correctly. I want to know that the hiveserver and hive metastore cloudera recommand memory is really true? Thanks for your help. | Number of Concurrent Connections | HiveServer2 Heap Size Recommended Range | Hive Metastore Heap Size Recommended Range | | Up to 40 concurrent connections Cloudera recommends splitting HiveServer2 into multiple instances and load-balancing once you start allocating over 16 GB to HiveServer2. This reduces the impact of Java garbage collection on active processing by the service. | 12 - 16 GB | 12 - 16 GB | | Up to 20 concurrent connections | 6 - 12 GB | 10 - 12 GB | | Up to 10 concurrent connections | 4 - 6 GB | 4 - 10 GB | | One connection | 4 GB | 4 GB | 由于本人英文太弱，不过我已经在补习英语过程中。我把我想描述的英文内容用中文再说一遍吧。 最近，我想从hive cli迁移使用beeline，但是看到cloudare官网这样描述hiveserver2 https://www.cloudera.com/documentation/enterprise/5-8-x/topics/cdh_ig_hive_install.html#concept_alp_4kl_3q 不知道官网写的并发连接数与hivesever2以及metastore内存配置是否是真的？不知道有 没有人测试过，因为我听有的开发同事说，hiveserve2开启后时间久了因性能原因可能需要重启。希望得到帮助谢谢。 youck
