[
https://issues.apache.org/jira/browse/PHOENIX-3553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15796840#comment-15796840
]
James Taylor commented on PHOENIX-3553:
---------------------------------------
The patch looks fine, but I'm confused as to why
DefaultStatisticsCollector.initGuidePostDepth() would be called during a minor
compaction given this check in UngroupedAggregateRegionObserver:
{code}
@Override
public InternalScanner preCompact(final
ObserverContext<RegionCoprocessorEnvironment> c, final Store store,
final InternalScanner scanner, final ScanType scanType) throws
IOException {
// Compaction and split upcalls run with the effective user context of
the requesting user.
// This will lead to failure of cross cluster RPC if the effective user
is not
// the login user. Switch to the login user context to ensure we have
the expected
// security context.
return User.runAsLoginUser(new
PrivilegedExceptionAction<InternalScanner>() {
@Override
public InternalScanner run() throws Exception {
TableName table =
c.getEnvironment().getRegion().getRegionInfo().getTable();
InternalScanner internalScanner = scanner;
if (scanType.equals(ScanType.COMPACT_DROP_DELETES)) {
try {
long clientTimeStamp =
TimeKeeper.SYSTEM.getCurrentTime();
StatisticsCollector stats =
StatisticsCollectorFactory.createStatisticsCollector(
c.getEnvironment(), table.getNameAsString(),
clientTimeStamp,
store.getFamily().getName());
internalScanner =
stats.createCompactionScanner(c.getEnvironment(), store, scanner);
{code}
> Zookeeper connection should be closed immediately after
> DefaultStatisticsCollector's collecting stats done
> ----------------------------------------------------------------------------------------------------------
>
> Key: PHOENIX-3553
> URL: https://issues.apache.org/jira/browse/PHOENIX-3553
> Project: Phoenix
> Issue Type: Bug
> Affects Versions: 4.9.0
> Reporter: Yeonseop Kim
> Labels: stats, zookeeper
> Fix For: 4.10.0
>
> Attachments: PHOENIX-3553.patch
>
>
> In every minor compaction job of HBase,
> org.apache.phoenix.schema.stats.DefaultStatisticsCollector.initGuidePostDepth()
> is called,
> and SYSTEM.CATALOG table is open to get guidepost width via
> htable = env.getTable(
>
> SchemaUtil.getPhysicalTableName(PhoenixDatabaseMetaData.SYSTEM_CATALOG_NAME_BYTES,
> env.getConfiguration()));
> This function call creates one zookeeper connection to get cluster id.
> DefaultStatisticsCollector doesn't close this zookeeper connection
> immediately after get guidepost width, and the zookeeper connection remains
> alive until HRegion is closed.
> This is not a problem with small number of Regions, but when number of Region
> is large and upsert operation is frequent, the number of zookeeper connection
> gradually increases to hundreds, and the zookeeper server nodes experience
> short of available TCP/IP ports.
> This zookeeper connection should be closed immediately after get guidepost
> width.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
