Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/6413#discussion_r31199761
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/physical/partitioning.scala
---
@@ -49,8 +49,19 @@ case object AllTuples extends Distribution
* [[Expression Expressions]] will be co-located. Based on the context,
this
* can mean such tuples are either co-located in the same partition or
they will be contiguous
* within a single partition.
+ * There is also another constraint, the `clustering` value contains null
will be considered
+ * as a valid value if `nullKeysSensitive` == true.
+ *
+ * For examples:
+ * JOIN KEYS: values contains null will be considered as invalid values,
which means
--- End diff --
Should be the input value. (Either the original data from table or the
intermediate result(e.g. join outputs)).
Validity of the null in the original table, depends on the semantics, in
`Join`, it's should also be invalid, but it's valid for `Group BY`.
It would an other optimization for repartition, contains `null` in the join
keys.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
