[
https://issues.apache.org/jira/browse/GROOVY-11983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18077865#comment-18077865
]
ASF GitHub Bot commented on GROOVY-11983:
-----------------------------------------
Copilot commented on code in PR #2509:
URL: https://github.com/apache/groovy/pull/2509#discussion_r3177464384
##########
src/main/java/org/codehaus/groovy/transform/stc/StaticTypeCheckingVisitor.java:
##########
@@ -6645,6 +6653,38 @@ private void putNotInstanceOfTypeInfo(final Object key,
final Collection<ClassNo
tti.addAll(types); // stash negative type(s)
}
+ /**
+ * GROOVY-11983: Whether the temporary type info captured while visiting
{@code expr}
+ * may be soundly inverted for the else branch of {@code if (expr) ...
else ...} (or
+ * the false branch of a ternary, or the surrounding scope after an
early-returning
+ * {@code if (expr) return}). Inversion is only sound when {@code !expr}
pins down
+ * the negation of every individual instanceof / !instanceof check inside
it. A
+ * logical-AND breaks that: the else of {@code A && B} is reached when at
least one
+ * of {@code A}, {@code B} is false, not necessarily both — so an
instanceof inside
+ * either operand cannot be safely inverted. In particular, a {@code
!instanceof}
+ * inside an AND would otherwise be unwrapped into a positive smart-cast,
producing
+ * an unsound {@code checkcast} and a runtime ClassCastException.
+ */
+ private static boolean canInvertNarrowingForElseBranch(final Expression
expr) {
+ if (expr instanceof BinaryExpression be) {
+ int op = be.getOperation().getType();
+ if (op == LOGICAL_AND) return false;
+ if (op == LOGICAL_OR) {
+ return canInvertNarrowingForElseBranch(be.getLeftExpression())
+ &&
canInvertNarrowingForElseBranch(be.getRightExpression());
+ }
+ return true; // instanceof, ==, comparisons, etc.
+ }
+ if (expr instanceof BooleanExpression be) return
canInvertNarrowingForElseBranch(be.getExpression());
+ if (expr instanceof NotExpression ne) return
canInvertNarrowingForElseBranch(ne.getExpression());
+ if (expr instanceof TernaryExpression te) {
+ return canInvertNarrowingForElseBranch(te.getBooleanExpression())
+ && canInvertNarrowingForElseBranch(te.getTrueExpression())
+ && canInvertNarrowingForElseBranch(te.getFalseExpression());
+ }
Review Comment:
canInvertNarrowingForElseBranch only rejects LOGICAL_AND and only recurses
through LOGICAL_OR. In Groovy, boolean expressions can also involve
BITWISE_AND/BITWISE_XOR (and BITWISE_OR containing nested subexpressions), and
the same unsound inversion issue applies when the else branch can be reached
without falsifying every instanceof/!instanceof sub-check. Consider extending
this helper to traverse the full expression tree (including casts) and treat
additional boolean operators (at least BITWISE_AND, likely BITWISE_XOR; and
recurse into BITWISE_OR operands) so inversion is conservatively disabled
whenever an AND-like node appears anywhere in the condition.
> STC: unsound smart-cast in else branch of if (cond && !(x instanceof Y))
> ------------------------------------------------------------------------
>
> Key: GROOVY-11983
> URL: https://issues.apache.org/jira/browse/GROOVY-11983
> Project: Groovy
> Issue Type: Bug
> Affects Versions: 5.0.5
> Reporter: Paul King
> Assignee: Paul King
> Priority: Major
>
> {{@CompileStatic}} unsoundly smart-casts a variable in the {{else}} branch
> when the {{if}} condition has the shape {{cond && !(x instanceof Y)}}. The
> else block is reached whenever {{cond}} is false (regardless of {{x}}'s
> type), but STC narrows {{x}} to {{Y}} and codegen emits {{checkcast Y}},
> throwing {{ClassCastException}} at runtime.
> h3. Reproducer
> {code:groovy}
> import groovy.transform.CompileStatic
> abstract class Animal { String name; boolean tame }
> class Cat extends Animal {}
> @CompileStatic
> class Tester {
> static String describe(Animal a) {
> if (a.tame && !(a instanceof Cat)) {
> return "tame non-cat $a.name"
> } else {
> return "other $a.name" // <-- a smart-cast to Cat here
> }
> }
> }
> Animal mystery = new Animal() {{ name = 'x'; tame = false }}
> Tester.describe(mystery) // ClassCastException: ... cannot be cast to Cat
> {code}
> {{javap}} of {{describe}} shows {{checkcast Cat}} on {{a}} in the else block
> before {{getName()}}.
> h3. Expected
> The else branch should not narrow {{a}} to {{Cat}}: it is reached when
> {{!a.tame || a instanceof Cat}}, so {{a}} is not guaranteed to be a {{Cat}}.
> h3. Workarounds
> * Replace {{!(x instanceof Y)}} with {{!Y.isAssignableFrom(x.getClass())}}
> * Split the conditional so the {{!instanceof}} stands alone
> h3. Root cause
> {{StaticTypeCheckingVisitor.visitIfElse}} inverts captured tti entries with
> {{tti.forEach(this::putNotInstanceOfTypeInfo)}}. The inversion is
> structurally blind: a {{!instanceof}} entry contributed by one operand of
> {{&&}} is unwrapped into the positive smart-cast bucket for the else branch,
> even though the else does not imply that operand was false. Sound for a
> single-operand boolean; unsound for compound conditions mixing {{&&}} with
> {{!instanceof}}.
> Related: GROOVY-6429, GROOVY-8412, GROOVY-8523, GROOVY-9931, GROOVY-11864
> (same {{tti}} plumbing); GROOVY-7971 fixed the symmetric then-branch issue
> for {{||}}.
> h3. Affects
> Groovy 5.0.x and 6.0.0-SNAPSHOT (master). Not present in 4.0.x.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
