[
https://issues.apache.org/jira/browse/GROOVY-12460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paul King updated GROOVY-12460:
-------------------------------
Description:
h2. Summary
Allow {{@Pure}} on parameters and local variables so {{PurityChecker}} can
verify closures passed as arguments, and treat closure-taking GDK methods as
pure when their closure is pure.
h2. Background
{{PurityChecker}} (incubating, 6.0) checks that a {{@Pure}} method has no side
effects. It already descends into closure literals inside the method body, so
an effect inside xs.each \{ println it } is reported. However, {{@Pure}}
targets methods only, so there is no way to state that a parameter must receive
a pure closure, and the checker has no notion of a method being pure *because*
its closure argument is pure. Compiling the following under
{{@TypeChecked(extensions = 'groovy.typecheckers.PurityChecker')}} on
6.0.0-beta-2 gives:
{code:groovy}
@Pure
static int applyTwice(Closure<Integer> f, int x) { f(f(x)) }
// @Pure warning: call to 'f.call()' — purity cannot be verified (consider
adding @Pure)
@Pure
static List<Integer> doubled(List<Integer> xs) { xs.collect { it * 2 } }
// @Pure warning: call to 'xs.collect()' — purity cannot be verified (consider
adding @Pure)
@Pure
static int total(List<Integer> xs) { xs.inject(0) { a, b -> a + b } }
// @Pure warning: call to 'xs.inject()' — purity cannot be verified (consider
adding @Pure)
{code}
The consequences:
* A {{@Pure}} method cannot take a callback at all.
* A {{@Pure}} method cannot use {{collect}}, {{inject}}, {{findAll}}, stream
pipelines or any other closure-taking idiom, which is how most Groovy code is
written. The known-pure method list excludes closure-taking methods
deliberately.
* Library APIs that need a pure callback for correctness (a parallel reduction
combiner, a memoized function, a transformation over a sensitive value) cannot
say so.
Two incidental observations from the same probe:
* {{sink << v}} on a parameter passes silently because the left-shift operator
is not a {{MethodCallExpression}}, while {{sink.add(v)}} is reported. Worth a
separate ticket.
* The "purity cannot be verified" messages are worded as warnings but raised as
errors.
h2. Proposal
h3. 1. {{@Pure}} on parameters and locals
Add {{ElementType.PARAMETER}} and {{ElementType.LOCAL_VARIABLE}} to the targets
of {{groovy.transform.Pure}}. On a parameter it means "the argument passed here
must be pure". On a local it means "the closure assigned here is checked once,
and the variable may then be passed where a pure closure is required". The
existing {{allows}} grades apply unchanged.
h3. 2. Call-site checking
When the checker sees a call whose resolved target has a {{@Pure}} parameter,
the corresponding argument is accepted if it is:
* a closure literal or lambda whose body passes the existing purity visitor,
plus a captured-write rule (below);
* a method pointer or method reference to a {{@Pure}} method;
* a variable, parameter or field itself declared {{@Pure}}.
Otherwise a static type error is reported.
Inside a {{@Pure}} method body, a call through a parameter declared {{@Pure}}
({{f\(x)}}, {{f.call\(x)}}, {{f.apply\(x)}}) is accepted.
h3. 3. Captured-write rule for closure literals
Groovy closures share captured locals by reference, so an assignment, prefix or
postfix operation whose target is a variable declared outside the closure is
reported as a violation, in the same way a field write is today. Reads of
captured variables remain fine.
h3. 4. Closure-polymorphic GDK methods
Add a list to the checker of GDK and {{java.util.stream}} methods that are pure
whenever their closure argument is pure: {{collect}}, {{collectMany}},
{{collectEntries}}, {{inject}}, {{findAll}}, {{find}}, {{any}}, {{every}},
{{count}}, {{sum}}, {{max}}, {{min}}, {{sort}} (non-mutating form),
{{groupBy}}, {{each}}, {{eachWithIndex}}, {{withIndex}}, {{takeWhile}},
{{dropWhile}}, {{map}}, {{filter}}, {{flatMap}}, {{reduce}}, {{mapToInt}} and
similar. When a call to one of these has a closure argument that passes the
body check, the call is accepted.
This list is one more of the lists {{PurityChecker}} already keeps (known-pure
methods, non-deterministic methods, logging, metrics and I/O prefixes). It is
not part of the GEP-25 model library and has no dependency on GEP-25. The
long-term direction is to annotate the GDK methods themselves, for example
{{@Pure collect(Iterable self, @Pure Closure transform)}}, at which point the
checker's existing callee-annotation lookup makes the list unnecessary.
h2. Examples after the change
{code:groovy}
@TypeChecked(extensions = 'groovy.typecheckers.PurityChecker')
class Pipeline {
@Pure
static int applyTwice(@Pure Closure<Integer> f, int x) { f(f(x)) } //
verifies
@Pure
static List<Integer> doubled(List<Integer> xs) { xs.collect { it * 2 } }
// verifies
@Pure
static void shout(List<Integer> xs) { xs.each { println it } } // one
error, for println only
@Pure
static int leak(List<Integer> xs, List<Integer> sink) {
xs.each { sink.add(it) } //
error: mutates parameter 'sink'
xs.size()
}
}
int counter = 0
Pipeline.applyTwice({ it + 1 }, 3) // ok
Pipeline.applyTwice(Maths.&square, 3) // ok if Maths.square is @Pure
Pipeline.applyTwice({ counter++; it }, 3) // error: closure passed to @Pure
parameter 'f' writes to captured variable 'counter'
Pipeline.applyTwice({ println it; it }, 3) // error: 'println()' is a
logging/output call
@Pure Closure<Integer> inc = { it + 1 } // checked once here
Closure<Integer> g = { it + 1 }
Pipeline.applyTwice(inc, 3) // ok
Pipeline.applyTwice(g, 3) // error: 'g' is not declared
@Pure
static void audit(@Pure(allows = 'LOGGING') Closure<Void> report) { ... }
audit { log.info "checked $it" } // ok
audit { new File('x').text = it } // error: I/O is not allowed
{code}
h2. Why it matters on its own
* A {{@Pure}} method that takes a callback can be verified instead of refused.
* {{@Pure}} methods can use ordinary collection idioms.
* {{injectParallel}} and similar can require a combiner that is pure as well as
associative, so a non-deterministic parallel fold is caught by two independent
checkers.
* Memoization and {{groovy.concurrent}} agent updates get a checked purity
premise.
* Any library wanting a side-effect-free transformation (for example a wrapper
for sensitive values whose {{map}} must not leak) can state it and have it
enforced.
h2. Scope and limits
* Static compilation only ({{@TypeChecked}} or {{@CompileStatic}}). Dynamic
dispatch cannot be checked.
* Closure bodies are walked with the existing visitor; no new effect categories.
* Method references to Java methods without annotations remain unverifiable, as
today.
h2. Suggested tests
* Callback parameter: pure literal accepted; captured write, logging, I/O and
non-deterministic call each rejected with a message naming the parameter.
* Each closure-polymorphic GDK method with a pure literal accepted, with an
impure literal rejected for the body's reason only (no second "cannot be
verified" error for the GDK call).
* {{@Pure}} local assigned a literal, passed on; unannotated local rejected.
* Method pointer to a {{@Pure}} method accepted; to an unannotated method
rejected.
* {{allows}} grade on a parameter honoured.
* Lambda syntax behaves the same as closure syntax.
was:
h2. Summary
Allow {{@Pure}} on parameters and local variables so {{PurityChecker}} can
verify closures passed as arguments, and treat closure-taking GDK methods as
pure when their closure is pure.
h2. Background
{{PurityChecker}} (incubating, 6.0) checks that a {{@Pure}} method has no side
effects. It already descends into closure literals inside the method body, so
an effect inside xs.each { println it } is reported. However, {{@Pure}} targets
methods only, so there is no way to state that a parameter must receive a pure
closure, and the checker has no notion of a method being pure *because* its
closure argument is pure. Compiling the following under
{{@TypeChecked(extensions = 'groovy.typecheckers.PurityChecker')}} on
6.0.0-beta-2 gives:
{code:groovy}
@Pure
static int applyTwice(Closure<Integer> f, int x) { f(f(x)) }
// @Pure warning: call to 'f.call()' — purity cannot be verified (consider
adding @Pure)
@Pure
static List<Integer> doubled(List<Integer> xs) { xs.collect { it * 2 } }
// @Pure warning: call to 'xs.collect()' — purity cannot be verified (consider
adding @Pure)
@Pure
static int total(List<Integer> xs) { xs.inject(0) { a, b -> a + b } }
// @Pure warning: call to 'xs.inject()' — purity cannot be verified (consider
adding @Pure)
{code}
The consequences:
* A {{@Pure}} method cannot take a callback at all.
* A {{@Pure}} method cannot use {{collect}}, {{inject}}, {{findAll}}, stream
pipelines or any other closure-taking idiom, which is how most Groovy code is
written. The known-pure method list excludes closure-taking methods
deliberately.
* Library APIs that need a pure callback for correctness (a parallel reduction
combiner, a memoized function, a transformation over a sensitive value) cannot
say so.
Two incidental observations from the same probe:
* {{sink << v}} on a parameter passes silently because the left-shift operator
is not a {{MethodCallExpression}}, while {{sink.add(v)}} is reported. Worth a
separate ticket.
* The "purity cannot be verified" messages are worded as warnings but raised as
errors.
h2. Proposal
h3. 1. {{@Pure}} on parameters and locals
Add {{ElementType.PARAMETER}} and {{ElementType.LOCAL_VARIABLE}} to the targets
of {{groovy.transform.Pure}}. On a parameter it means "the argument passed here
must be pure". On a local it means "the closure assigned here is checked once,
and the variable may then be passed where a pure closure is required". The
existing {{allows}} grades apply unchanged.
h3. 2. Call-site checking
When the checker sees a call whose resolved target has a {{@Pure}} parameter,
the corresponding argument is accepted if it is:
* a closure literal or lambda whose body passes the existing purity visitor,
plus a captured-write rule (below);
* a method pointer or method reference to a {{@Pure}} method;
* a variable, parameter or field itself declared {{@Pure}}.
Otherwise a static type error is reported.
Inside a {{@Pure}} method body, a call through a parameter declared {{@Pure}}
({{f\(x)}}, {{f.call\(x)}}, {{f.apply\(x)}}) is accepted.
h3. 3. Captured-write rule for closure literals
Groovy closures share captured locals by reference, so an assignment, prefix or
postfix operation whose target is a variable declared outside the closure is
reported as a violation, in the same way a field write is today. Reads of
captured variables remain fine.
h3. 4. Closure-polymorphic GDK methods
Add a list to the checker of GDK and {{java.util.stream}} methods that are pure
whenever their closure argument is pure: {{collect}}, {{collectMany}},
{{collectEntries}}, {{inject}}, {{findAll}}, {{find}}, {{any}}, {{every}},
{{count}}, {{sum}}, {{max}}, {{min}}, {{sort}} (non-mutating form),
{{groupBy}}, {{each}}, {{eachWithIndex}}, {{withIndex}}, {{takeWhile}},
{{dropWhile}}, {{map}}, {{filter}}, {{flatMap}}, {{reduce}}, {{mapToInt}} and
similar. When a call to one of these has a closure argument that passes the
body check, the call is accepted.
This list is one more of the lists {{PurityChecker}} already keeps (known-pure
methods, non-deterministic methods, logging, metrics and I/O prefixes). It is
not part of the GEP-25 model library and has no dependency on GEP-25. The
long-term direction is to annotate the GDK methods themselves, for example
{{@Pure collect(Iterable self, @Pure Closure transform)}}, at which point the
checker's existing callee-annotation lookup makes the list unnecessary.
h2. Examples after the change
{code:groovy}
@TypeChecked(extensions = 'groovy.typecheckers.PurityChecker')
class Pipeline {
@Pure
static int applyTwice(@Pure Closure<Integer> f, int x) { f(f(x)) } //
verifies
@Pure
static List<Integer> doubled(List<Integer> xs) { xs.collect { it * 2 } }
// verifies
@Pure
static void shout(List<Integer> xs) { xs.each { println it } } // one
error, for println only
@Pure
static int leak(List<Integer> xs, List<Integer> sink) {
xs.each { sink.add(it) } //
error: mutates parameter 'sink'
xs.size()
}
}
int counter = 0
Pipeline.applyTwice({ it + 1 }, 3) // ok
Pipeline.applyTwice(Maths.&square, 3) // ok if Maths.square is @Pure
Pipeline.applyTwice({ counter++; it }, 3) // error: closure passed to @Pure
parameter 'f' writes to captured variable 'counter'
Pipeline.applyTwice({ println it; it }, 3) // error: 'println()' is a
logging/output call
@Pure Closure<Integer> inc = { it + 1 } // checked once here
Closure<Integer> g = { it + 1 }
Pipeline.applyTwice(inc, 3) // ok
Pipeline.applyTwice(g, 3) // error: 'g' is not declared
@Pure
static void audit(@Pure(allows = 'LOGGING') Closure<Void> report) { ... }
audit { log.info "checked $it" } // ok
audit { new File('x').text = it } // error: I/O is not allowed
{code}
h2. Why it matters on its own
* A {{@Pure}} method that takes a callback can be verified instead of refused.
* {{@Pure}} methods can use ordinary collection idioms.
* {{injectParallel}} and similar can require a combiner that is pure as well as
associative, so a non-deterministic parallel fold is caught by two independent
checkers.
* Memoization and {{groovy.concurrent}} agent updates get a checked purity
premise.
* Any library wanting a side-effect-free transformation (for example a wrapper
for sensitive values whose {{map}} must not leak) can state it and have it
enforced.
h2. Scope and limits
* Static compilation only ({{@TypeChecked}} or {{@CompileStatic}}). Dynamic
dispatch cannot be checked.
* Closure bodies are walked with the existing visitor; no new effect categories.
* Method references to Java methods without annotations remain unverifiable, as
today.
h2. Suggested tests
* Callback parameter: pure literal accepted; captured write, logging, I/O and
non-deterministic call each rejected with a message naming the parameter.
* Each closure-polymorphic GDK method with a pure literal accepted, with an
impure literal rejected for the body's reason only (no second "cannot be
verified" error for the GDK call).
* {{@Pure}} local assigned a literal, passed on; unannotated local rejected.
* Method pointer to a {{@Pure}} method accepted; to an unannotated method
rejected.
* {{allows}} grade on a parameter honoured.
* Lambda syntax behaves the same as closure syntax.
> Allow @Pure on parameters and local variables
> ---------------------------------------------
>
> Key: GROOVY-12460
> URL: https://issues.apache.org/jira/browse/GROOVY-12460
> Project: Groovy
> Issue Type: Improvement
> Components: groovy-typecheckers
> Reporter: Paul King
> Assignee: Paul King
> Priority: Major
> Fix For: 7.x
>
>
> h2. Summary
> Allow {{@Pure}} on parameters and local variables so {{PurityChecker}} can
> verify closures passed as arguments, and treat closure-taking GDK methods as
> pure when their closure is pure.
> h2. Background
> {{PurityChecker}} (incubating, 6.0) checks that a {{@Pure}} method has no
> side effects. It already descends into closure literals inside the method
> body, so an effect inside xs.each \{ println it } is reported. However,
> {{@Pure}} targets methods only, so there is no way to state that a parameter
> must receive a pure closure, and the checker has no notion of a method being
> pure *because* its closure argument is pure. Compiling the following under
> {{@TypeChecked(extensions = 'groovy.typecheckers.PurityChecker')}} on
> 6.0.0-beta-2 gives:
> {code:groovy}
> @Pure
> static int applyTwice(Closure<Integer> f, int x) { f(f(x)) }
> // @Pure warning: call to 'f.call()' — purity cannot be verified (consider
> adding @Pure)
> @Pure
> static List<Integer> doubled(List<Integer> xs) { xs.collect { it * 2 } }
> // @Pure warning: call to 'xs.collect()' — purity cannot be verified
> (consider adding @Pure)
> @Pure
> static int total(List<Integer> xs) { xs.inject(0) { a, b -> a + b } }
> // @Pure warning: call to 'xs.inject()' — purity cannot be verified (consider
> adding @Pure)
> {code}
> The consequences:
> * A {{@Pure}} method cannot take a callback at all.
> * A {{@Pure}} method cannot use {{collect}}, {{inject}}, {{findAll}}, stream
> pipelines or any other closure-taking idiom, which is how most Groovy code is
> written. The known-pure method list excludes closure-taking methods
> deliberately.
> * Library APIs that need a pure callback for correctness (a parallel
> reduction combiner, a memoized function, a transformation over a sensitive
> value) cannot say so.
> Two incidental observations from the same probe:
> * {{sink << v}} on a parameter passes silently because the left-shift
> operator is not a {{MethodCallExpression}}, while {{sink.add(v)}} is
> reported. Worth a separate ticket.
> * The "purity cannot be verified" messages are worded as warnings but raised
> as errors.
> h2. Proposal
> h3. 1. {{@Pure}} on parameters and locals
> Add {{ElementType.PARAMETER}} and {{ElementType.LOCAL_VARIABLE}} to the
> targets of {{groovy.transform.Pure}}. On a parameter it means "the argument
> passed here must be pure". On a local it means "the closure assigned here is
> checked once, and the variable may then be passed where a pure closure is
> required". The existing {{allows}} grades apply unchanged.
> h3. 2. Call-site checking
> When the checker sees a call whose resolved target has a {{@Pure}} parameter,
> the corresponding argument is accepted if it is:
> * a closure literal or lambda whose body passes the existing purity visitor,
> plus a captured-write rule (below);
> * a method pointer or method reference to a {{@Pure}} method;
> * a variable, parameter or field itself declared {{@Pure}}.
> Otherwise a static type error is reported.
> Inside a {{@Pure}} method body, a call through a parameter declared {{@Pure}}
> ({{f\(x)}}, {{f.call\(x)}}, {{f.apply\(x)}}) is accepted.
> h3. 3. Captured-write rule for closure literals
> Groovy closures share captured locals by reference, so an assignment, prefix
> or postfix operation whose target is a variable declared outside the closure
> is reported as a violation, in the same way a field write is today. Reads of
> captured variables remain fine.
> h3. 4. Closure-polymorphic GDK methods
> Add a list to the checker of GDK and {{java.util.stream}} methods that are
> pure whenever their closure argument is pure: {{collect}}, {{collectMany}},
> {{collectEntries}}, {{inject}}, {{findAll}}, {{find}}, {{any}}, {{every}},
> {{count}}, {{sum}}, {{max}}, {{min}}, {{sort}} (non-mutating form),
> {{groupBy}}, {{each}}, {{eachWithIndex}}, {{withIndex}}, {{takeWhile}},
> {{dropWhile}}, {{map}}, {{filter}}, {{flatMap}}, {{reduce}}, {{mapToInt}} and
> similar. When a call to one of these has a closure argument that passes the
> body check, the call is accepted.
> This list is one more of the lists {{PurityChecker}} already keeps
> (known-pure methods, non-deterministic methods, logging, metrics and I/O
> prefixes). It is not part of the GEP-25 model library and has no dependency
> on GEP-25. The long-term direction is to annotate the GDK methods themselves,
> for example {{@Pure collect(Iterable self, @Pure Closure transform)}}, at
> which point the checker's existing callee-annotation lookup makes the list
> unnecessary.
> h2. Examples after the change
> {code:groovy}
> @TypeChecked(extensions = 'groovy.typecheckers.PurityChecker')
> class Pipeline {
> @Pure
> static int applyTwice(@Pure Closure<Integer> f, int x) { f(f(x)) } //
> verifies
> @Pure
> static List<Integer> doubled(List<Integer> xs) { xs.collect { it * 2 } }
> // verifies
> @Pure
> static void shout(List<Integer> xs) { xs.each { println it } } // one
> error, for println only
> @Pure
> static int leak(List<Integer> xs, List<Integer> sink) {
> xs.each { sink.add(it) } //
> error: mutates parameter 'sink'
> xs.size()
> }
> }
> int counter = 0
> Pipeline.applyTwice({ it + 1 }, 3) // ok
> Pipeline.applyTwice(Maths.&square, 3) // ok if Maths.square is @Pure
> Pipeline.applyTwice({ counter++; it }, 3) // error: closure passed to
> @Pure parameter 'f' writes to captured variable 'counter'
> Pipeline.applyTwice({ println it; it }, 3) // error: 'println()' is a
> logging/output call
> @Pure Closure<Integer> inc = { it + 1 } // checked once here
> Closure<Integer> g = { it + 1 }
> Pipeline.applyTwice(inc, 3) // ok
> Pipeline.applyTwice(g, 3) // error: 'g' is not declared
> @Pure
> static void audit(@Pure(allows = 'LOGGING') Closure<Void> report) { ... }
> audit { log.info "checked $it" } // ok
> audit { new File('x').text = it } // error: I/O is not allowed
> {code}
> h2. Why it matters on its own
> * A {{@Pure}} method that takes a callback can be verified instead of refused.
> * {{@Pure}} methods can use ordinary collection idioms.
> * {{injectParallel}} and similar can require a combiner that is pure as well
> as associative, so a non-deterministic parallel fold is caught by two
> independent checkers.
> * Memoization and {{groovy.concurrent}} agent updates get a checked purity
> premise.
> * Any library wanting a side-effect-free transformation (for example a
> wrapper for sensitive values whose {{map}} must not leak) can state it and
> have it enforced.
> h2. Scope and limits
> * Static compilation only ({{@TypeChecked}} or {{@CompileStatic}}). Dynamic
> dispatch cannot be checked.
> * Closure bodies are walked with the existing visitor; no new effect
> categories.
> * Method references to Java methods without annotations remain unverifiable,
> as today.
> h2. Suggested tests
> * Callback parameter: pure literal accepted; captured write, logging, I/O and
> non-deterministic call each rejected with a message naming the parameter.
> * Each closure-polymorphic GDK method with a pure literal accepted, with an
> impure literal rejected for the body's reason only (no second "cannot be
> verified" error for the GDK call).
> * {{@Pure}} local assigned a literal, passed on; unannotated local rejected.
> * Method pointer to a {{@Pure}} method accepted; to an unannotated method
> rejected.
> * {{allows}} grade on a parameter honoured.
> * Lambda syntax behaves the same as closure syntax.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
