bobbai00 commented on code in PR #5024:
URL: https://github.com/apache/texera/pull/5024#discussion_r3216386221
##########
common/pybuilder/src/main/scala/org/apache/texera/amber/pybuilder/BoundaryValidator.scala:
##########
@@ -22,21 +22,21 @@ package org.apache.texera.amber.pybuilder
import scala.reflect.macros.blackbox
/**
- * Macro-only helper: validates boundaries for Encodable insertions.
- *
- * Compile-time: abort with good messages for direct Encodable args.
- * Runtime: for nested builders (unknown content at compile time), generate a
check that throws if the builder contains Encodable chunks.
- */
+ * Macro-only helper: validates boundaries for Encodable insertions.
Review Comment:
revert this change ?
##########
common/pybuilder/src/main/scala/org/apache/texera/amber/pybuilder/BoundaryValidator.scala:
##########
@@ -22,21 +22,21 @@ package org.apache.texera.amber.pybuilder
import scala.reflect.macros.blackbox
/**
- * Macro-only helper: validates boundaries for Encodable insertions.
- *
- * Compile-time: abort with good messages for direct Encodable args.
- * Runtime: for nested builders (unknown content at compile time), generate a
check that throws if the builder contains Encodable chunks.
- */
+ * Macro-only helper: validates boundaries for Encodable insertions.
+ *
+ * Compile-time: abort with good messages for direct Encodable args.
+ * Runtime: for nested builders (unknown content at compile time), generate a
check that throws if the builder contains Encodable chunks.
+ */
final class BoundaryValidator[C <: blackbox.Context](val c: C) {
import PythonLexerUtils._
import c.universe._
/**
- * Centralized, templatized error messages (Option A).
- *
- * NOTE: This object lives inside the class so it can freely use string
templates
- * without any macro-context type gymnastics.
- */
+ * Centralized, templatized error messages (Option A).
Review Comment:
ditto
##########
common/pybuilder/src/main/scala/org/apache/texera/amber/pybuilder/EncodableInspector.scala:
##########
@@ -145,18 +145,18 @@ final class EncodableInspector[C <: blackbox.Context](val
c: C) {
// - treat already-wrapped EncodableStringRenderer as encodable
// - OR detect @EncodableStringAnnotation on symbol/type
(tpe != null && (tpe.dealias.widen <:< encodableStringRendererTpe)) ||
- treeHasEncodableString(argExpr.tree)
+ treeHasEncodableString(argExpr.tree)
}
}
/**
- * Wrap an argument expression as a [[PythonTemplateBuilder.StringRenderer]]
AST node.
- *
- * Priority:
- * 1) If it's already a StringRenderer, keep it (cast).
- * 2) Else if Encodable-marked, wrap as EncodableStringRenderer.
- * 3) Else wrap as PyLiteralStringRenderer.
- */
+ * Wrap an argument expression as a
[[PythonTemplateBuilder.StringRenderer]] AST node.
+ *
+ * Priority:
+ * 1) If it's already a StringRenderer, keep it (cast).
+ * 2) Else if Encodable-marked, wrap as EncodableStringRenderer.
+ * 3) Else wrap as PyLiteralStringRenderer.
+ */
def wrapArg(argExpr: c.Expr[Any]): Tree = {
Review Comment:
ditto
##########
common/pybuilder/src/main/scala/org/apache/texera/amber/pybuilder/PythonTemplateBuilder.scala:
##########
@@ -92,117 +92,117 @@ object PyStringTypes {
}
/**
- * =PythonTemplateBuilder: ergonomic Python codegen via `pyb"..."`=
- *
- * This module provides a tiny DSL for assembling Python source code from
Scala while preserving two competing goals:
- * (1) developers want to write templates that look like normal Python, and
(2) user-provided text must not be injected
- * into the emitted Python as raw literals that can break syntax or create
ambiguous token boundaries.
- *
- * The core idea is that every value spliced into a `pyb"..."` template is
first classified into one of two buckets:
- *
- * - '''Python literals''' (ordinary Scala strings or already-safe fragments)
are inserted as-is.
- * - '''Encodable strings''' (typically UI-provided text) are base64-encoded
at build time and rendered as a *Python
- * expression* that decodes at runtime, rather than being embedded as a
Python string literal.
- *
- * This classification is driven by a TYPE_USE annotation: `String
@EncodableStringAnnotation`. The annotation is defined
- * with a runtime retention and is allowed on fields, parameters, local
variables, and type uses, so it survives many
- * common Scala typing patterns (e.g., inferred vals, constructor params, or
aliases). Users normally do not construct the
- * annotation directly; instead, they use helper type aliases/factories in
`PyStringTypes` for readability.
- *
- * ==Render modes==
- *
- * A `PythonTemplateBuilder` can be rendered in two modes:
- *
- * - `plain`: emit everything as raw text (useful for debugging or when you
know all content is safe).
- * - `encode`: emit encodable chunks as Python decode expressions (the
default `toString` behavior).
- *
- * Internally this is represented as a small sealed trait enum
(`RenderMode.Plain` / `RenderMode.Encode`) rather than an
- * integer flag, to keep call sites self-documenting and avoid “magic numbers”.
- *
- * ==Chunk model (immutable, composable)==
- *
- * A builder is an immutable list of chunks:
- *
- * - `Text(value)` for literal template parts
- * - `Value(renderer)` for interpolated arguments that know how to render in
each mode
- *
- * Two concrete renderers are provided:
- *
- * - `EncodableStringRenderer`: pre-encodes `stringValue` as base64 (UTF-8)
once, and in `Encode` mode produces a Python
- * expression like `self.decode_python_template('<b64>')` given by
[[wrapWithPythonDecoderExpr]].
- * - `PyLiteralStringRenderer`: always emits the raw string value unchanged.
- *
- * Builders can be concatenated with `+` (builder + builder), which merges
adjacent `Text` chunks for compactness.
- * Direct concatenation with a plain `String` is intentionally unsupported to
prevent bypassing the macro’s safety checks.
- *
- * ==How the `pyb"..."` macro works==
- *
- * The `pyb` interpolator is implemented as a Scala macro. At compile time it
receives:
- *
- * - the literal parts from the `StringContext` (the “gaps” around `$args`)
- * - the argument trees corresponding to each `$arg`
- *
- * The macro’s pipeline is:
- *
- * 1. '''Extract literal parts''' from the `StringContext` AST and ensure
they are *string literals*. If any part is not
- * a literal, compilation aborts. This prevents “template text” from being
computed dynamically where correctness and
- * boundary analysis would become unreliable.
- *
- * 2. '''Classify direct encodable arguments''' using `EncodableInspector`:
- * it inspects both the argument symbol and the argument type to determine
whether the encodable annotation is present.
- * This includes a small “accessor hop” so that annotations placed on
fields/constructor params are still visible when
- * call sites reference getters.
- *
- * 3. '''Compile-time boundary validation for direct encodables''':
- * if an argument is directly encodable (and not a nested builder),
`BoundaryValidator.validateCompileTime` is run on
- * its surrounding literal context. The validator performs quick lexical
checks on the current line:
- *
- * - the splice must not occur inside an unclosed single/double-quoted
string
- * - the splice must not occur after a `#` comment marker
- * - the splice must not be immediately adjacent to identifier
characters or quote characters on either side
- *
- * These restrictions exist because an Encodable string renders as a Python
*expression*, not a Python string literal.
- * Putting an expression inside quotes, inside a comment, or glued to an
identifier would either be invalid Python or
- * silently change tokenization in surprising ways.
- *
- * 4. '''Lower each argument into a builder''':
- * every `$arg` becomes a `PythonTemplateBuilder`.
- *
- * - If the argument is already a `PythonTemplateBuilder`, it is used
directly.
- * - Otherwise, it is wrapped into a `StringRenderer`
(`EncodableStringRenderer` or `PyLiteralStringRenderer`) and
- * turned into a minimal builder containing a single `Value(...)`
chunk.
- *
- * Each argument is evaluated once and stored in a fresh local `val
__pyb_argN` so that expensive expressions or
- * side-effects are not duplicated by expansion.
- *
- * 5. '''Runtime safety for nested builders''':
- * for arguments that are themselves `PythonTemplateBuilder`s, the macro
cannot always know at compile time whether they
- * contain Encodable chunks (they may be computed, returned, or composed
elsewhere). For these nested builders, the macro
- * conditionally emits runtime guards *only when the surrounding context is
unsafe* (inside quotes, after comments, or
- * adjacent to “bad neighbor” characters). The guard pattern is:
- *
- * {{{
- * if (__pyb_argN.containsEncodableString) throw new
IllegalArgumentException("...")
- * }}}
- *
- * This preserves the ergonomics of composing builders while keeping the same
safety contract as direct splices.
- *
- * 6. '''Assemble the final builder''':
- * the macro concatenates `text0 + arg0 + text1 + arg1 + ... + textN` into one
`PythonTemplateBuilder`.
- *
- * ==Lexical checks (best-effort, intentionally small)==
- *
- * The boundary rules rely on `PythonLexerUtils`, a tiny state machine that
scans only the “current line tail” to decide
- * whether quotes are unbalanced and whether a `#` begins a comment outside
quotes. This is not a full Python parser.
- * It is deliberately lightweight so the macro stays fast and so the helpers
can be unit-tested independently.
- *
- * ==Extensibility notes==
- *
- * The design keeps all rendering behavior behind `StringRenderer`, and keeps
boundary policy in `BoundaryValidator`.
- * If new encoding schemes, alternate runtime decode helpers, or additional
safety rules are needed, they can be introduced
- * without rewriting the template-building API. In particular, swapping
`wrapWithPythonDecoderExpr` or adding new renderers
- * is a contained change: the macro only needs to decide *which renderer* to
use, not *how it renders*.
- */
+ * =PythonTemplateBuilder: ergonomic Python codegen via `pyb"..."`=
Review Comment:
ditto
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
