FrankYang0529 commented on code in PR #71189:
URL: https://github.com/apache/airflow/pull/71189#discussion_r4181105698
##########
java-sdk/processor/src/main/kotlin/org/apache/airflow/sdk/BuilderProcessor.kt:
##########
@@ -190,46 +231,206 @@ class BuilderProcessor : AbstractProcessor() {
explicitConfig(el, DAG_ANNOTATION, DAG_STRUCTURAL_ATTRIBUTES,
SchemaFields.DAG).forEach { (key, value) ->
buildMethod.addStatement($$"dag.config($S, $L)", key, value)
}
-
- for (inner in el.enclosedElements) {
- if (inner !is ExecutableElement) continue
- if (inner.isVarArgs) throw IllegalArgumentException("Cannot create task
from vararg function ${inner.simpleName}")
-
- val taskAnn = inner.getAnnotation(Builder.Task::class.java) ?: continue
- val innerName =
inner.simpleName.toString().replaceFirstChar(Char::uppercase)
-
- builderClass.addType(buildTask(innerName, inner, el))
-
+ if (deps != null) {
buildMethod.addStatement(
- $$"dag.addTask($L)",
- taskDefCode(inner, taskAnn.id.ifBlank { inner.simpleName.toString() },
innerName),
+ $$"return $T.record(dag, $T.of($L), new $T()::depends)",
+ REFS_TYPE,
+ ClassName.get(List::class.java),
+ CodeBlock.join(declarations.map { CodeBlock.of($$"$S", it.id) }, ", "),
+ ClassName.get(deps),
)
+ } else {
+ // No wiring class: register every task with no Java-side edges, which
+ // is the task-handler shape rather than a Dag Java owns.
+ declarations.forEach { decl ->
+ buildMethod.addStatement($$"dag.addTask($L)", taskDefCode(decl,
CodeBlock.of($$"$L", decl.className)))
+ }
+ buildMethod.addStatement("return dag")
Review Comment:
The `// No wiring class` comment here and java.rst both call a
`@Builder.Dag` class with no `@Builder.Deps` class "the task-handler shape
rather than a Dag Java owns". But ADR-0011 says a task handler for a Python Dag
is not a Dag registration. It also removes an old pattern: a Java artifact
declaring a Dag under a dag_id that a Python file already owns. The `else`
branch still turns such a class into a Java-owned Dag. `Bundle.register(Class)`
puts the resulting Dag in `bundle.dags`, and #71190 serializes every Dag in
`bundle.dags`.
I compiled a `@Builder.Dag` class that has `extract()` and `transform(long
extracted, double factor)` but no wiring class. `Bundle.register(Class)`
accepted the class and registered a Dag with no edges. `transform` then failed
only at run time, with an error about a stub call that this Dag does not have.
Should the processor reject a `@Builder.Dag` class that has no `@Builder.Deps`
class?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
