[jira] [Resolved] (IMPALA-11284) INSERT query with concat operator fails with 'Function not set in thrift node' error
[
https://issues.apache.org/jira/browse/IMPALA-11284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Riza Suminto resolved IMPALA-11284.
---
Fix Version/s: Impala 4.3.0
Resolution: Fixed
> INSERT query with concat operator fails with 'Function not set in thrift
> node' error
>
>
> Key: IMPALA-11284
> URL: https://issues.apache.org/jira/browse/IMPALA-11284
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 4.1.0
>Reporter: Abhishek Rawat
>Assignee: Michael Smith
>Priority: Critical
> Fix For: Impala 4.3.0
>
>
> *Steps to Reproduce:*
> {code:java}
> DROP TABLE t2;
> CREATE TABLE t2(c0 BOOLEAN, c1 STRING) STORED AS ICEBERG;
> INSERT INTO t2(c0, c1) VALUES ( TRUE, ( 'abc' ||('927160245' || 'Q') ) );
> Error: Function not set in thrift node{code}
> Looks like a regression introduced by IMPALA-6590.
> fn_ was previously serialized during rewrite in :
> {code:java}
> treeToThriftHelper:FunctionCallExpr(Expr).treeToThriftHelper(TExpr) line: 866
> FunctionCallExpr(Expr).treeToThrift() line: 844
> FeSupport.EvalExprWithoutRowBounded(Expr, TQueryCtx, int) line: 188
> LiteralExpr.createBounded(Expr, TQueryCtx, int) line: 210
> FoldConstantsRule.apply(Expr, Analyzer) line: 66
> ExprRewriter.applyRuleBottomUp(Expr, ExprRewriteRule, Analyzer) line: 85
> ExprRewriter.applyRuleRepeatedly(Expr, ExprRewriteRule, Analyzer) line: 71
> ExprRewriter.rewrite(Expr, Analyzer) line: 55
> SelectList.rewriteExprs(ExprRewriter, Analyzer) line: 100
> SelectStmt.rewriteExprs(ExprRewriter) line: 1189
> ValuesStmt(SetOperationStmt).rewriteExprs(ExprRewriter) line: 467
> InsertStmt.rewriteExprs(ExprRewriter) line: 1119
> AnalysisContext.analyze(StmtMetadataLoader$StmtTableCache,
> AuthorizationContext) line: 537 {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-11284) INSERT query with concat operator fails with 'Function not set in thrift node' error
[
https://issues.apache.org/jira/browse/IMPALA-11284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Smith reassigned IMPALA-11284:
--
Assignee: Riza Suminto (was: Michael Smith)
> INSERT query with concat operator fails with 'Function not set in thrift
> node' error
>
>
> Key: IMPALA-11284
> URL: https://issues.apache.org/jira/browse/IMPALA-11284
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 4.1.0
>Reporter: Abhishek Rawat
>Assignee: Riza Suminto
>Priority: Critical
> Fix For: Impala 4.3.0
>
>
> *Steps to Reproduce:*
> {code:java}
> DROP TABLE t2;
> CREATE TABLE t2(c0 BOOLEAN, c1 STRING) STORED AS ICEBERG;
> INSERT INTO t2(c0, c1) VALUES ( TRUE, ( 'abc' ||('927160245' || 'Q') ) );
> Error: Function not set in thrift node{code}
> Looks like a regression introduced by IMPALA-6590.
> fn_ was previously serialized during rewrite in :
> {code:java}
> treeToThriftHelper:FunctionCallExpr(Expr).treeToThriftHelper(TExpr) line: 866
> FunctionCallExpr(Expr).treeToThrift() line: 844
> FeSupport.EvalExprWithoutRowBounded(Expr, TQueryCtx, int) line: 188
> LiteralExpr.createBounded(Expr, TQueryCtx, int) line: 210
> FoldConstantsRule.apply(Expr, Analyzer) line: 66
> ExprRewriter.applyRuleBottomUp(Expr, ExprRewriteRule, Analyzer) line: 85
> ExprRewriter.applyRuleRepeatedly(Expr, ExprRewriteRule, Analyzer) line: 71
> ExprRewriter.rewrite(Expr, Analyzer) line: 55
> SelectList.rewriteExprs(ExprRewriter, Analyzer) line: 100
> SelectStmt.rewriteExprs(ExprRewriter) line: 1189
> ValuesStmt(SetOperationStmt).rewriteExprs(ExprRewriter) line: 467
> InsertStmt.rewriteExprs(ExprRewriter) line: 1119
> AnalysisContext.analyze(StmtMetadataLoader$StmtTableCache,
> AuthorizationContext) line: 537 {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-12441) Simplify local toolchain development
[ https://issues.apache.org/jira/browse/IMPALA-12441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17765303#comment-17765303 ] Michael Smith commented on IMPALA-12441: We should update the Wiki. I need to figure out how to get access. > Simplify local toolchain development > > > Key: IMPALA-12441 > URL: https://issues.apache.org/jira/browse/IMPALA-12441 > Project: IMPALA > Issue Type: Task >Reporter: Michael Smith >Assignee: Michael Smith >Priority: Major > > Testing updates to https://github.com/cloudera/native-toolchain currently > takes some work. > https://cwiki.apache.org/confluence/display/IMPALA/Building+native-toolchain+from+scratch+and+using+with+Impala > mentions it's for advanced users only, and the instructions don't quite work > anymore. They also don't work very well when switching between different > branches. > For aarch64 builds we make this somewhat simpler. Expand on that to make it > easier to work with local native-toolchain checkouts. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-12414) Add scripts to run Trino in the dev environment
[ https://issues.apache.org/jira/browse/IMPALA-12414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17765332#comment-17765332 ] ASF subversion and git services commented on IMPALA-12414: -- Commit cf819f967b2ba01e9903c8f69fea0ad09e35e34e in impala's branch refs/heads/master from Zoltan Borok-Nagy [ https://gitbox.apache.org/repos/asf?p=impala.git;h=cf819f967 ] IMPALA-12414: Add scripts to run Trino in the dev environment This patch set adds a couple of scripts that makes it easy to run Trino in the dev environment. At first one need to build our custom Trino docker image via: testdata/bin/build-trino-docker-image.sh Then we can run the Trino container via: testdata/bin/run-trino.sh One can connect to the Trino CLI by: testdata/bin/trino-cli.sh Trino will be configured to access our HMS and HDFS. We add the 'hive' and 'iceberg' catalogs for Trino. Legacy Hive tables can be accessed via the 'hive' catalog, while Iceberg tables (only the ones reside in HiveCatalog) can be accessed via the 'iceberg' catalog. E.g.: trino> use iceberg.functional_parquet; trino:functional_parquet> select count(*) from iceberg_mixed_file_format; _col0 --- 3 (1 row) Testing * no tests added * later we might always run Trino as part of the minicluster and add interop tests with it Change-Id: I49818c7a95e23988b3fbc3d31b4c7fa738e0d952 Reviewed-on: http://gerrit.cloudera.org:8080/20444 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins > Add scripts to run Trino in the dev environment > --- > > Key: IMPALA-12414 > URL: https://issues.apache.org/jira/browse/IMPALA-12414 > Project: IMPALA > Issue Type: Bug >Reporter: Zoltán Borók-Nagy >Assignee: Zoltán Borók-Nagy >Priority: Major > > Add some scripts to conveniently run Trino in the minicluster environment. > Maybe later we could run Trino with the other minicluster components by > default. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-11284) INSERT query with concat operator fails with 'Function not set in thrift node' error
[
https://issues.apache.org/jira/browse/IMPALA-11284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17765333#comment-17765333
]
ASF subversion and git services commented on IMPALA-11284:
--
Commit 130a55e5269ea023b43ef2c0b495989cb0759800 in impala's branch
refs/heads/master from Abhishek Rawat
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=130a55e52 ]
IMPALA-11284: Do non-optional rewrites for || and Between predicate
IMPALA-6590 disabled expression rewrites for ValuesStmt. However,
CompoundVerticalBarExpr (||) cannot be executed directly without
rewrite. This is because it could either be an OR operation with boolean
arguments or CONCAT function call with string arguments.
Backend cannot evaluate a BetweenPredicate and relies on rewriting
BetweenPredicate into a conjunctive or disjunctive CompoundPredicate.
This patch enables non-optional expression rewrites for ValuesStmt with
CompoundVerticalBarExpr or BetweenPredicate.
Testing:
- Extended ExprRewriterTest and Planner test to have values clause
with || and Between predicate
Change-Id: I99b8b33bf6468d12b9e26f0a6e744feb7072619c
Reviewed-on: http://gerrit.cloudera.org:8080/18581
Reviewed-by: Michael Smith
Reviewed-by: Daniel Becker
Tested-by: Riza Suminto
> INSERT query with concat operator fails with 'Function not set in thrift
> node' error
>
>
> Key: IMPALA-11284
> URL: https://issues.apache.org/jira/browse/IMPALA-11284
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 4.1.0
>Reporter: Abhishek Rawat
>Assignee: Riza Suminto
>Priority: Critical
> Fix For: Impala 4.3.0
>
>
> *Steps to Reproduce:*
> {code:java}
> DROP TABLE t2;
> CREATE TABLE t2(c0 BOOLEAN, c1 STRING) STORED AS ICEBERG;
> INSERT INTO t2(c0, c1) VALUES ( TRUE, ( 'abc' ||('927160245' || 'Q') ) );
> Error: Function not set in thrift node{code}
> Looks like a regression introduced by IMPALA-6590.
> fn_ was previously serialized during rewrite in :
> {code:java}
> treeToThriftHelper:FunctionCallExpr(Expr).treeToThriftHelper(TExpr) line: 866
> FunctionCallExpr(Expr).treeToThrift() line: 844
> FeSupport.EvalExprWithoutRowBounded(Expr, TQueryCtx, int) line: 188
> LiteralExpr.createBounded(Expr, TQueryCtx, int) line: 210
> FoldConstantsRule.apply(Expr, Analyzer) line: 66
> ExprRewriter.applyRuleBottomUp(Expr, ExprRewriteRule, Analyzer) line: 85
> ExprRewriter.applyRuleRepeatedly(Expr, ExprRewriteRule, Analyzer) line: 71
> ExprRewriter.rewrite(Expr, Analyzer) line: 55
> SelectList.rewriteExprs(ExprRewriter, Analyzer) line: 100
> SelectStmt.rewriteExprs(ExprRewriter) line: 1189
> ValuesStmt(SetOperationStmt).rewriteExprs(ExprRewriter) line: 467
> InsertStmt.rewriteExprs(ExprRewriter) line: 1119
> AnalysisContext.analyze(StmtMetadataLoader$StmtTableCache,
> AuthorizationContext) line: 537 {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-12441) Simplify local toolchain development
[ https://issues.apache.org/jira/browse/IMPALA-12441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17765335#comment-17765335 ] ASF subversion and git services commented on IMPALA-12441: -- Commit 4be517e150148bd852d1fd106a4e4b1adf1229f1 in impala's branch refs/heads/master from Michael Smith [ https://gitbox.apache.org/repos/asf?p=impala.git;h=4be517e15 ] IMPALA-12441: Simplify local toolchain development If NATIVE_TOOLCHAIN_HOME is set, that will be used to provide the native toolchain instead of the default in IMPALA_TOOLCHAIN. Overrides IMPALA_TOOLCHAIN_PACKAGES_HOME and sets SKIP_TOOLCHAIN_BOOTSTRAP=true. Adds IMPALA_TOOLCHAIN_REPO, IMPALA_TOOLCHAIN_BRANCH, and IMPALA_TOOLCHAIN_COMMIT_HASH so everything is clear about what toolchain is used for this Impala commit. If NATIVE_TOOLCHAIN_HOME does not yet exist, buildall.sh will clone the repo and checkout the commit hash mentioned above before building. Also skips downloading Kudu if SKIP_TOOLCHAIN_BOOTSTRAP is true as Kudu is built from native-toolchain. Normalizes aarch64 logic, which skipped Kudu because it would always build native-toolchain locally. Change-Id: I3a9e51b7f54c738d8cc01b32428ac88a344de376 Reviewed-on: http://gerrit.cloudera.org:8080/20267 Tested-by: Impala Public Jenkins Reviewed-by: Michael Smith > Simplify local toolchain development > > > Key: IMPALA-12441 > URL: https://issues.apache.org/jira/browse/IMPALA-12441 > Project: IMPALA > Issue Type: Task >Reporter: Michael Smith >Assignee: Michael Smith >Priority: Major > > Testing updates to https://github.com/cloudera/native-toolchain currently > takes some work. > https://cwiki.apache.org/confluence/display/IMPALA/Building+native-toolchain+from+scratch+and+using+with+Impala > mentions it's for advanced users only, and the instructions don't quite work > anymore. They also don't work very well when switching between different > branches. > For aarch64 builds we make this somewhat simpler. Expand on that to make it > easier to work with local native-toolchain checkouts. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-6590) Disable expr rewrites and codegen for VALUES() statements
[
https://issues.apache.org/jira/browse/IMPALA-6590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17765334#comment-17765334
]
ASF subversion and git services commented on IMPALA-6590:
-
Commit 130a55e5269ea023b43ef2c0b495989cb0759800 in impala's branch
refs/heads/master from Abhishek Rawat
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=130a55e52 ]
IMPALA-11284: Do non-optional rewrites for || and Between predicate
IMPALA-6590 disabled expression rewrites for ValuesStmt. However,
CompoundVerticalBarExpr (||) cannot be executed directly without
rewrite. This is because it could either be an OR operation with boolean
arguments or CONCAT function call with string arguments.
Backend cannot evaluate a BetweenPredicate and relies on rewriting
BetweenPredicate into a conjunctive or disjunctive CompoundPredicate.
This patch enables non-optional expression rewrites for ValuesStmt with
CompoundVerticalBarExpr or BetweenPredicate.
Testing:
- Extended ExprRewriterTest and Planner test to have values clause
with || and Between predicate
Change-Id: I99b8b33bf6468d12b9e26f0a6e744feb7072619c
Reviewed-on: http://gerrit.cloudera.org:8080/18581
Reviewed-by: Michael Smith
Reviewed-by: Daniel Becker
Tested-by: Riza Suminto
> Disable expr rewrites and codegen for VALUES() statements
> -
>
> Key: IMPALA-6590
> URL: https://issues.apache.org/jira/browse/IMPALA-6590
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 2.8.0, Impala 2.9.0, Impala 2.10.0, Impala 2.11.0
>Reporter: Alexander Behm
>Assignee: Wenzhe Zhou
>Priority: Major
> Labels: perf, planner, ramp-up, regression
> Fix For: Impala 4.1.0
>
>
> The analysis of statements with big VALUES clauses like INSERT INTO
> VALUES is slow due to expression rewrites like constant folding. The
> performance of such statements has regressed since the introduction of expr
> rewrites and constant folding in IMPALA-1788.
> We should skip expr rewrites for VALUES altogether since it mostly provides
> no benefit but can have a large overhead due to evaluation of expressions in
> the backend (constant folding). These expressions are ultimately evaluated
> and materialized in the backend anyway, so there's no point in folding them
> during analysis.
> Similarly, there is no point in doing codegen for these exprs in the backend
> union node.
> *Workaround*
> {code}
> SET ENABLE_EXPR_REWRITES=FALSE;
> SET DISABLE_CODEGEN=TRUE;
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-12441) Simplify local toolchain development
[ https://issues.apache.org/jira/browse/IMPALA-12441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Smith resolved IMPALA-12441. Fix Version/s: Impala 4.3.0 Resolution: Fixed > Simplify local toolchain development > > > Key: IMPALA-12441 > URL: https://issues.apache.org/jira/browse/IMPALA-12441 > Project: IMPALA > Issue Type: Task >Reporter: Michael Smith >Assignee: Michael Smith >Priority: Major > Fix For: Impala 4.3.0 > > > Testing updates to https://github.com/cloudera/native-toolchain currently > takes some work. > https://cwiki.apache.org/confluence/display/IMPALA/Building+native-toolchain+from+scratch+and+using+with+Impala > mentions it's for advanced users only, and the instructions don't quite work > anymore. They also don't work very well when switching between different > branches. > For aarch64 builds we make this somewhat simpler. Expand on that to make it > easier to work with local native-toolchain checkouts. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-12402) Make CatalogdMetaProvider's cache concurrency level configurable
[ https://issues.apache.org/jira/browse/IMPALA-12402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17765415#comment-17765415 ] Maxwell Guo commented on IMPALA-12402: -- I have already modified the commit message and together with the merge conflict . [~MikaelSmith] > Make CatalogdMetaProvider's cache concurrency level configurable > > > Key: IMPALA-12402 > URL: https://issues.apache.org/jira/browse/IMPALA-12402 > Project: IMPALA > Issue Type: Improvement > Components: fe >Reporter: Maxwell Guo >Assignee: Maxwell Guo >Priority: Minor > Labels: pull-request-available > > when the cluster contains many db and tables such as if there are more than > 10 tables, and if we restart the impalad , the local cache_ > CatalogMetaProvider's need to doing some loading process. > As we know that the goole's guava cache 's concurrencyLevel os set to 4 by > default. > but if there is many tables the loading process will need more time and > increase the probability of lock contention, see > [here|https://github.com/google/guava/blob/master/guava/src/com/google/common/cache/CacheBuilder.java#L437]. > > So we propose to add some configurations here, the first is the concurrency > of cache. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-12399) Pass eventTypeSkipList with OPEN_TXN in NotificationEventRequest to avoid receiving OPEN_TXN events from HMS
[
https://issues.apache.org/jira/browse/IMPALA-12399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Venugopal Reddy K reassigned IMPALA-12399:
--
Assignee: Venugopal Reddy K
> Pass eventTypeSkipList with OPEN_TXN in NotificationEventRequest to avoid
> receiving OPEN_TXN events from HMS
>
>
> Key: IMPALA-12399
> URL: https://issues.apache.org/jira/browse/IMPALA-12399
> Project: IMPALA
> Issue Type: Improvement
> Components: Catalog
>Reporter: Venugopal Reddy K
>Assignee: Venugopal Reddy K
>Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Notification events like OPEN_TXN are ignored on catalogd
> {{{}MetastoreEventsProcessor{}}}. So, we can pass eventTypeSkipList with
> OPEN_TXN in NotificationEventRequest while invoking get_next_notification()
> to avoid reading such notification messages from HMS and then ignoring on
> catalogd. OPEN_TXN event being more frequent(received even upon describe
> table operation from beeline), we can significantly reduce unwanted
> processing on both HMS and catalogd. Catalogd reads events in batches of
> EVENTS_BATCH_SIZE_PER_RPC, skipping such unnecessary events can help catchup
> the events faster.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-12399) Pass eventTypeSkipList with OPEN_TXN in NotificationEventRequest to avoid receiving OPEN_TXN events from HMS
[
https://issues.apache.org/jira/browse/IMPALA-12399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Venugopal Reddy K reassigned IMPALA-12399:
--
Assignee: (was: Venugopal Reddy K)
> Pass eventTypeSkipList with OPEN_TXN in NotificationEventRequest to avoid
> receiving OPEN_TXN events from HMS
>
>
> Key: IMPALA-12399
> URL: https://issues.apache.org/jira/browse/IMPALA-12399
> Project: IMPALA
> Issue Type: Improvement
> Components: Catalog
>Reporter: Venugopal Reddy K
>Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Notification events like OPEN_TXN are ignored on catalogd
> {{{}MetastoreEventsProcessor{}}}. So, we can pass eventTypeSkipList with
> OPEN_TXN in NotificationEventRequest while invoking get_next_notification()
> to avoid reading such notification messages from HMS and then ignoring on
> catalogd. OPEN_TXN event being more frequent(received even upon describe
> table operation from beeline), we can significantly reduce unwanted
> processing on both HMS and catalogd. Catalogd reads events in batches of
> EVENTS_BATCH_SIZE_PER_RPC, skipping such unnecessary events can help catchup
> the events faster.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-12399) Pass eventTypeSkipList with OPEN_TXN in NotificationEventRequest to avoid receiving OPEN_TXN events from HMS
[
https://issues.apache.org/jira/browse/IMPALA-12399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Venugopal Reddy K reassigned IMPALA-12399:
--
Assignee: Venugopal Reddy K
> Pass eventTypeSkipList with OPEN_TXN in NotificationEventRequest to avoid
> receiving OPEN_TXN events from HMS
>
>
> Key: IMPALA-12399
> URL: https://issues.apache.org/jira/browse/IMPALA-12399
> Project: IMPALA
> Issue Type: Improvement
> Components: Catalog
>Reporter: Venugopal Reddy K
>Assignee: Venugopal Reddy K
>Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Notification events like OPEN_TXN are ignored on catalogd
> {{{}MetastoreEventsProcessor{}}}. So, we can pass eventTypeSkipList with
> OPEN_TXN in NotificationEventRequest while invoking get_next_notification()
> to avoid reading such notification messages from HMS and then ignoring on
> catalogd. OPEN_TXN event being more frequent(received even upon describe
> table operation from beeline), we can significantly reduce unwanted
> processing on both HMS and catalogd. Catalogd reads events in batches of
> EVENTS_BATCH_SIZE_PER_RPC, skipping such unnecessary events can help catchup
> the events faster.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
