szehon-ho commented on code in PR #58002:
URL: https://github.com/apache/spark/pull/58002#discussion_r3817382701
##########
sql/catalyst/src/main/java/org/apache/spark/sql/connector/catalog/functions/BoundFunction.java:
##########
@@ -90,6 +90,14 @@ default boolean isDeterministic() {
* functions in other catalogs. For example, many catalogs may define a
"bucket" function with a
* different implementation. Adding context, like
"com.mycompany.bucket(string)", is recommended
* to avoid unintentional collisions.
+ * <p>
+ * Two functions that partition data differently must not return the same
name; Spark may
+ * otherwise treat unrelated data as co-partitioned. An override should
return a stable name
Review Comment:
this is a bit out of nowhere (per report).
Maybe like:
```For example, Spark binds a function afresh when ...```
##########
sql/catalyst/src/main/java/org/apache/spark/sql/connector/catalog/functions/BoundFunction.java:
##########
@@ -100,4 +108,38 @@ default String canonicalName() {
// bugs if not replaced before release.
return UUID.randomUUID().toString();
}
+
+ /**
+ * Implementations SHOULD override {@link Object#equals(Object)} and {@link
Object#hashCode()}.
+ * <p>
+ * Spark binds a function afresh every time it converts a partitioning or
ordering reported by a
Review Comment:
can we generalize it and not be so tied to the current implementation
detail?
Like 'Spark may bind a function multiple times'. And then hopefully the
bullet point covers it?
Or if we really need the detail: ```Spark may bind a function multiple
times, such as ...```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
