[
https://issues.apache.org/jira/browse/TRAFODION-2017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15304195#comment-15304195
]
ASF GitHub Bot commented on TRAFODION-2017:
-------------------------------------------
Github user DaveBirdsall commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/505#discussion_r64923467
--- Diff: core/sql/ustat/hs_globals.cpp ---
@@ -546,10 +557,13 @@ NABoolean HSGlobalsClass::setHBaseCacheSize(double
sampleRatio)
Int64 workableCacheSize = (Int64)(sampleRatio * calibrationFactor);
if (workableCacheSize < 1)
workableCacheSize = 1; // can't go below 1 unfortunately
+ else if (workableCacheSize > 50)
+ workableCacheSize = 50;
- Int32 max = getDefaultAsLong(HBASE_NUM_CACHE_ROWS_MAX);
- if ((workableCacheSize < 10000) && // don't bother if 10000 works
- (max == 10000)) // don't do it if user has already set this CQD
+ // if the user himself set the CQD, don't do anything
+ NADefaults &defs = ActiveSchemaDB()->getDefaults();
+ if (defs.getProvenance(HBASE_NUM_CACHE_ROWS_MAX) ==
+ NADefaults::INIT_DEFAULT_DEFAULTS)
--- End diff --
Thanks, @selvaganesang. Will change in the next rework.
> Tune HBase row cache sizes so UPDATE STATS completes
> ----------------------------------------------------
>
> Key: TRAFODION-2017
> URL: https://issues.apache.org/jira/browse/TRAFODION-2017
> Project: Apache Trafodion
> Issue Type: Bug
> Components: sql-cmp
> Affects Versions: 2.0-incubating
> Environment: All, though prevalent on clusters with larger tables
> and/or heavier loads
> Reporter: David Wayne Birdsall
> Assignee: David Wayne Birdsall
> Fix For: 2.1-incubating
>
>
> UPDATE STATISTICS often fails with HBase socket timeout exception and/or
> scanner timeout exception when run with sampling on larger tables or clusters
> with heavy concurrent workloads.
> We have experimented in the past with setting various CQDs on large tables to
> reduce these failures, however we were loathe to set them all the time due to
> fears that this would lengthen elapsed time in non-failure scenarios.
> Some recent work by Carol Pearson however shows that the increase in elapsed
> time is negligible for smaller tables and in failure scenarios the failure
> does not occur quickly, so paying a small penalty in elapsed time to increase
> the probability of success seems a better trade-off.
> Carol's work involves tables of less than 1 billion rows. The existing CQD
> logic is still required for larger tables. But for tables of less than 1
> billion rows, she recommends setting HBASE_ROWS_CACHED_MIN and
> HBASE_ROWS_CACHED_MAX to '50'. This JIRA is written to cover this change.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
