[
https://issues.apache.org/jira/browse/SPARK-37288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon updated SPARK-37288:
---------------------------------
Description:
Annotation for {{pyspark.since}} has been changed in SPARK-36906, from
{code:python}
def since(version: str) -> Callable[[T], T]: ...
{code}
to
{code:python}
def since(version: Union[str, float]) -> Callable[[T], T]: ...
{code}
We should backport this change to 3.1 and 3.2 (see for example
https://github.com/delta-io/delta/pull/305)
This is technically a bug in the downstream projects that runs mypy checks
against pyspark.since. When they use it, for example, with
{{pyspark.since(3.2)}}, mypy checks fails; however, this case is legitimate.
After this change, the mypy change can pass.
was:
Annotation for {{pyspark.since}} has been changed in SPARK-36906, from
{code:python}
def since(version: str) -> Callable[[T], T]: ...
{code}
to
{code:python}
def since(version: Union[str, float]) -> Callable[[T], T]: ...
{code}
We should backport this change to 3.1 and 3.2 (see for example
https://github.com/delta-io/delta/pull/305)
> Backport update pyspark.since annotation to 3.1 and 3.2
> -------------------------------------------------------
>
> Key: SPARK-37288
> URL: https://issues.apache.org/jira/browse/SPARK-37288
> Project: Spark
> Issue Type: Bug
> Components: PySpark
> Affects Versions: 3.1.0, 3.2.0
> Reporter: Maciej Szymkiewicz
> Priority: Major
>
> Annotation for {{pyspark.since}} has been changed in SPARK-36906, from
> {code:python}
> def since(version: str) -> Callable[[T], T]: ...
> {code}
> to
> {code:python}
> def since(version: Union[str, float]) -> Callable[[T], T]: ...
> {code}
> We should backport this change to 3.1 and 3.2 (see for example
> https://github.com/delta-io/delta/pull/305)
> This is technically a bug in the downstream projects that runs mypy checks
> against pyspark.since. When they use it, for example, with
> {{pyspark.since(3.2)}}, mypy checks fails; however, this case is legitimate.
> After this change, the mypy change can pass.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
