Hello everyone, I'd like to revisit the topic of adding PySpark type annotations in 3.0. It has been discussed before ( http://apache-spark-developers-list.1001551.n3.nabble.com/Python-friendly-API-for-Spark-3-0-td25016.html and http://apache-spark-developers-list.1001551.n3.nabble.com/PYTHON-PySpark-typing-hints-td21560.html) and is tracked by SPARK-17333 ( https://issues.apache.org/jira/browse/SPARK-17333). Is there any consensus here?
In the spirit of full disclosure I am trying to decide if, and if yes to what extent, migrate my stub package ( https://github.com/zero323/pyspark-stubs) to 3.0 and beyond. Maintaining such package is relatively time consuming (not being active PySpark user anymore, it is the least priority for me at the moment) and if there any official plans to make it obsolete, it would be a valuable information for me. If there are no plans to add native annotations to PySpark, I'd like to use this opportunity to ask PySpark commiters, to drop by and open issue ( https://github.com/zero323/pyspark-stubs/issues) when new methods are introduced, or there are changes in the existing API (PR's are of course welcomed as well). Thanks in advance. -- Best, Maciej
