I think the annotations are compatible with Python 2 since Maciej implemented them via stub files <https://www.python.org/dev/peps/pep-0484/#stub-files>, which Python 2 simply ignores. Folks using mypy <http://www.mypy-lang.org> to check types will get the benefit whether they're on Python 2 or 3, since mypy works with both.
On Fri, Jan 25, 2019 at 12:27 PM Reynold Xin <r...@databricks.com> wrote: > If we can make the annotation compatible with Python 2, why don’t we add > type annotation to make life easier for users of Python 3 (with type)? > > On Fri, Jan 25, 2019 at 7:53 AM Maciej Szymkiewicz <mszymkiew...@gmail.com> > wrote: > >> >> Hello everyone, >> >> I'd like to revisit the topic of adding PySpark type annotations in 3.0. >> It has been discussed before ( >> http://apache-spark-developers-list.1001551.n3.nabble.com/Python-friendly-API-for-Spark-3-0-td25016.html >> and >> http://apache-spark-developers-list.1001551.n3.nabble.com/PYTHON-PySpark-typing-hints-td21560.html) >> and is tracked by SPARK-17333 ( >> https://issues.apache.org/jira/browse/SPARK-17333). Is there any >> consensus here? >> >> In the spirit of full disclosure I am trying to decide if, and if yes to >> what extent, migrate my stub package ( >> https://github.com/zero323/pyspark-stubs) to 3.0 and beyond. Maintaining >> such package is relatively time consuming (not being active PySpark user >> anymore, it is the least priority for me at the moment) and if there any >> official plans to make it obsolete, it would be a valuable information for >> me. >> >> If there are no plans to add native annotations to PySpark, I'd like to >> use this opportunity to ask PySpark commiters, to drop by and open issue ( >> https://github.com/zero323/pyspark-stubs/issues) when new methods are >> introduced, or there are changes in the existing API (PR's are of course >> welcomed as well). Thanks in advance. >> >> -- >> Best, >> Maciej >> >> >>
