The doc says that the minimum supported pandas version is 0.23.2 which is only supported in python2. IIRC, python2 is not supported in pyspark a long time ago. Can any one confirm whether the doc is wrong and what is the right version of pandas and pyarrow ?
https://spark.apache.org/docs/latest/api/python/user_guide/arrow_pandas.html#recommended-pandas-and-pyarrow-versions -- Best Regards Jeff Zhang
