[
https://issues.apache.org/jira/browse/DRILL-6830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16686818#comment-16686818
]
ASF GitHub Bot commented on DRILL-6830:
---------------------------------------
vvysotskyi commented on issue #1524: DRILL-6830: Remove
Hook.REL_BUILDER_SIMPLIFY handler after use
URL: https://github.com/apache/drill/pull/1524#issuecomment-438724649
@lushuifeng, you are right that the problem connected with
`TestCaseNullableTypes#testCaseNullableTypesVarchar` is in Calcite. The initial
goal for adding `Hook.REL_BUILDER_SIMPLIFY.add(Hook.propertyJ(false));` was to
avoid failures connected with treating empty strings as null feature, but looks
like it was fixed in another place.
The interesting thing is that problem connected with
`TestCaseNullableTypes#testCaseNullableTypesVarchar` failure was fixed after
1.17 version, so when Drill is rebased onto Calcite 1.18,
`Hook.REL_BUILDER_SIMPLIFY.add(Hook.propertyJ(false));` may be removed.
@ihuzenko, since you are working on Calcite rebase, could you please also
take care of it?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Hook.REL_BUILDER_SIMPLIFY handler didn't removed cause performance degression
> -----------------------------------------------------------------------------
>
> Key: DRILL-6830
> URL: https://issues.apache.org/jira/browse/DRILL-6830
> Project: Apache Drill
> Issue Type: Bug
> Components: Query Planning & Optimization
> Affects Versions: 1.14.0
> Reporter: shuifeng lu
> Assignee: shuifeng lu
> Priority: Major
> Fix For: 1.15.0
>
> Attachments: Screen Shot 2018-11-06 at 16.14.16.png
>
>
> Planning performance degression has been observed that the duration of
> planning increased from 30ms to 160ms after running drill a long period of
> time(say a month).
> RelBuilder.simplify never becomes true if Hook.REL_BUILDER_SIMPLIFY handlers
> are not removed.
> Here is some clue (after running 40 days):
> Hook.get takes 8ms per-invocation, it may be called serveral times per query.
> ---[8.816063ms] org.apache.calcite.tools.RelBuilder:<init>()
> +---[0.020218ms] java.util.ArrayDeque:<init>()
> +---[0.018493ms] java.lang.Boolean:valueOf()
> +---[8.341566ms] org.apache.calcite.runtime.Hook:get()
> +---[0.008489ms] java.lang.Boolean:booleanValue()
> +---[min=5.21E-4ms,max=0.015832ms,total=0.025233ms,count=12]
> org.apache.calcite.plan.Context:unwrap()
> +---[min=3.83E-4ms,max=0.009494ms,total=0.014516ms,count=13]
> org.apache.calcite.util.Util:first()
> +---[0.006892ms] org.apache.calcite.plan.RelOptCluster:getPlanner()
> +---[0.009104ms] org.apache.calcite.plan.RelOptPlanner:getExecutor()
> +---[min=4.8E-4ms,max=0.002277ms,total=0.002757ms,count=2]
> org.apache.calcite.plan.RelOptCluster:getRexBuilder()
> ---[min=4.91E-4ms,max=0.004586ms,total=0.005077ms,count=2]
> org.apache.calcite.rex.RexSimplify:<init>()
> The top instances in JVM
> num #instances #bytes class name
> ----------------------------------------------
> 1: 116333 116250440 [B
> 2: 890126 105084536 [C
> 3: 338062 37415944 [Ljava.lang.Object;
> 4: 1715004 27440064 org.apache.calcite.runtime.Hook$4
> 5: 803909 19293816 java.lang.String
> !Screen Shot 2018-11-06 at 16.14.16.png!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
