KushShriv commented on code in PR #13278:
URL: https://github.com/apache/gluten/pull/13278#discussion_r4398437662
##########
gluten-substrait/src/main/scala/org/apache/gluten/expression/ExpressionConverter.scala:
##########
@@ -949,6 +979,33 @@ object ExpressionConverter extends SQLConfHelper with
Logging {
}
}
+ /**
+ * Translates the pattern argument of a regex expression to RE2 syntax
before it is written into
+ * the Substrait literal. If the pattern is a string literal, Java Unicode
escapes (backslash-u
+ * followed by 4 hex digits) are converted to RE2's `\x{XXXX}` form.
Patterns that contain
+ * constructs with no RE2 equivalent (lookaheads, lookbehinds,
backreferences) throw
+ * [[GlutenNotSupportException]] to trigger graceful fallback. Non-literal
(column-referenced)
+ * patterns, and all patterns on backends that do not use RE2, are passed
through unchanged.
+ */
+ private def withRe2PatternTranslation(
+ patternChild: Expression,
+ attributeSeq: Seq[Attribute],
+ expressionsMap: Map[Class[_], String]): ExpressionTransformer = {
+ patternChild match {
+ case Literal(patStr: UTF8String, StringType)
+ if
BackendsApiManager.getSparkPlanExecApiInstance.needsRe2PatternTranslation =>
+ val translated = Re2PatternUtils
+ .translateJavaPatternToRe2(patStr.toString)
+ .getOrElse(throw new GlutenNotSupportException(
+ s"Regex pattern
${Re2PatternUtils.describePattern(patStr.toString)} uses " +
+ s"constructs (lookahead/lookbehind/backreference) or unpaired
surrogates " +
+ s"that RE2 cannot handle. Falling back to Spark."))
+ LiteralTransformer(Literal(UTF8String.fromString(translated),
StringType))
+ case other =>
+ replaceWithExpressionTransformer0(other, attributeSeq, expressionsMap)
Review Comment:
Thanks @philo-he, done in aeb75ef2e. `needsRe2PatternTranslation` is gone
and `SparkPlanExecApi` now has `rewriteRegexPattern(pattern: Expression):
Expression`, which returns the pattern unchanged by default. Velox and Bolt
override it. Since the two bodies would be identical, the override delegates to
a shared `Re2PatternUtils.rewriteRegexPattern` with your snippet (I kept the
existing fallback wording so the `FallbackSuite` assertion still matches).
`ExpressionConverter` now just calls the hook and wraps a literal result in
`LiteralTransformer`; ClickHouse is unaffected because the default is a no-op.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
