zhangbutao commented on code in PR #4700:
URL: https://github.com/apache/hive/pull/4700#discussion_r1331852260
##########
ql/src/java/org/apache/hadoop/hive/ql/parse/UpdateDeleteSemanticAnalyzer.java:
##########
@@ -177,39 +169,41 @@ private void reparseAndSuperAnalyze(ASTNode tree, Table
mTable, ASTNode tabNameN
ASTNode where = null;
int whereIndex = deleting() ? 1 : 2;
+
if (children.size() > whereIndex) {
where = (ASTNode)children.get(whereIndex);
assert where.getToken().getType() == HiveParser.TOK_WHERE :
"Expected where clause, but found " + where.getName();
+
+ if (copyOnWriteMode) {
+ String whereClause = ctx.getTokenRewriteStream().toString(
+ where.getChild(0).getTokenStartIndex(),
where.getChild(0).getTokenStopIndex());
+
+ rewrittenQueryStr.append(" where ");
+ // Add the inverted where clause, since we want to hold the records
which doesn't satisfy the condition.
+ rewrittenQueryStr.append(" not(").append(whereClause).append(")");
+ // Add the file path filter that matches the delete condition.
+ rewrittenQueryStr.append(" and FILE__PATH in (");
+ rewrittenQueryStr.append(" select `FILE__PATH` from
").append(getFullTableNameForSQL(tabNameNode));
+ rewrittenQueryStr.append(" where ").append(whereClause);
+ rewrittenQueryStr.append(" )");
- if (shouldOverwrite) {
- if (where.getChildCount() == 1) {
-
- // Add isNull check for the where clause condition, since null is
treated as false in where condition and
- // not null also resolves to false, so we need to explicitly handle
this case.
- ASTNode isNullFuncNodeExpr = new ASTNode(new
CommonToken(HiveParser.TOK_FUNCTION, "TOK_FUNCTION"));
- isNullFuncNodeExpr.addChild(new ASTNode(new
CommonToken(HiveParser.Identifier, "isNull")));
- isNullFuncNodeExpr.addChild(where.getChild(0));
-
- ASTNode orNodeExpr = new ASTNode(new CommonToken(HiveParser.KW_OR,
"OR"));
- orNodeExpr.addChild(isNullFuncNodeExpr);
-
- // Add the inverted where clause condition, since we want to hold
the records which doesn't satisfy this
- // condition.
- ASTNode notNodeExpr = new ASTNode(new CommonToken(HiveParser.KW_NOT,
"!"));
- notNodeExpr.addChild(where.getChild(0));
- orNodeExpr.addChild(notNodeExpr);
- where.setChild(0, orNodeExpr);
- } else if (where.getChildCount() > 1) {
- throw new SemanticException("Overwrite mode not supported with more
than 1 children in where clause.");
- }
+ rewrittenQueryStr.append(" union all ");
Review Comment:
In my example, no datafiles would be replaced because there is no datafile
which has value `>100 ` and also the affected can be determined in the first
Map `tableScan`. So I think it would good if we can skip the following
unnecessary tasks(union & reduce).
I have not dived into this part of the code. Maybe this just my subjective
guess, and i will give more info if i have a overall understanding in the
future.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
