[GitHub] [incubator-druid] himanshug commented on a change in pull request #7716: AggregatorUtil should cache parsed expression to avoid memory problem (OOM/FGC) when Expression is used in metricsSpec
himanshug commented on a change in pull request #7716: AggregatorUtil should
cache parsed expression to avoid memory problem (OOM/FGC) when Expression is
used in metricsSpec
URL: https://github.com/apache/incubator-druid/pull/7716#discussion_r287079210
##
File path:
processing/src/main/java/org/apache/druid/query/aggregation/AggregatorUtil.java
##
@@ -196,7 +246,7 @@ static BaseFloatColumnValueSelector
makeColumnValueSelectorWithFloatDefault(
if (fieldName != null) {
return metricFactory.makeColumnValueSelector(fieldName);
} else {
- final Expr expr = Parser.parse(fieldExpression, macroTable);
+ final Expr expr = parseIfAbsent(fieldExpression, macroTable);
Review comment:
sent a PR to your branch, pls merge that. I meant to remove cache altogether
and tie lifetime of parsed expression to the object holding it.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org
[GitHub] [incubator-druid] himanshug commented on a change in pull request #7716: AggregatorUtil should cache parsed expression to avoid memory problem (OOM/FGC) when Expression is used in metricsSpec
himanshug commented on a change in pull request #7716: AggregatorUtil should
cache parsed expression to avoid memory problem (OOM/FGC) when Expression is
used in metricsSpec
URL: https://github.com/apache/incubator-druid/pull/7716#discussion_r286188705
##
File path: core/src/main/java/org/apache/druid/math/expr/Parser.java
##
@@ -75,6 +75,9 @@ public static Expr parse(String in, ExprMacroTable
macroTable)
@VisibleForTesting
static Expr parse(String in, ExprMacroTable macroTable, boolean withFlatten)
{
+if (log.isDebugEnabled()) {
Review comment:
log.debug(..) has this check inside it , you can use string format instead
of string concatenation and remove this check.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org
[GitHub] [incubator-druid] himanshug commented on a change in pull request #7716: AggregatorUtil should cache parsed expression to avoid memory problem (OOM/FGC) when Expression is used in metricsSpec
himanshug commented on a change in pull request #7716: AggregatorUtil should
cache parsed expression to avoid memory problem (OOM/FGC) when Expression is
used in metricsSpec
URL: https://github.com/apache/incubator-druid/pull/7716#discussion_r286190859
##
File path:
processing/src/main/java/org/apache/druid/query/aggregation/AggregatorUtil.java
##
@@ -196,7 +246,7 @@ static BaseFloatColumnValueSelector
makeColumnValueSelectorWithFloatDefault(
if (fieldName != null) {
return metricFactory.makeColumnValueSelector(fieldName);
} else {
- final Expr expr = Parser.parse(fieldExpression, macroTable);
+ final Expr expr = parseIfAbsent(fieldExpression, macroTable);
Review comment:
would be better to not have explicit cache added in this class. Instead, if
you change arguments of this(and other similar methods) to...
```
static BaseFloatColumnValueSelector
makeColumnValueSelectorWithFloatDefault(
final ColumnSelectorFactory metricFactory,
@Nullable final String fieldName,
@Nullable final Expr fieldExpression,
final float nullValue
)
```
and let caching happen in `SimpleXXAggregatorFactory` classes . Don't parse
in the constructor of those classes because those objects could be created in
many places where they wouldn't need parsed expression, so do the parsing
lazily.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org
[GitHub] [incubator-druid] himanshug commented on a change in pull request #7716: AggregatorUtil should cache parsed expression to avoid memory problem (OOM/FGC) when Expression is used in metricsSpec
himanshug commented on a change in pull request #7716: AggregatorUtil should
cache parsed expression to avoid memory problem (OOM/FGC) when Expression is
used in metricsSpec
URL: https://github.com/apache/incubator-druid/pull/7716#discussion_r286191367
##
File path: core/src/main/java/org/apache/druid/math/expr/Parser.java
##
@@ -75,6 +75,9 @@ public static Expr parse(String in, ExprMacroTable
macroTable)
@VisibleForTesting
static Expr parse(String in, ExprMacroTable macroTable, boolean withFlatten)
{
+if (log.isDebugEnabled()) {
Review comment:
also, not sure if you really need this log at all.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org
