Yikun commented on PR #36353: URL: https://github.com/apache/spark/pull/36353#issuecomment-1178956945
Let me do a brief conclusion to help review: - **Change `setitem` to make a copy**https://github.com/pandas-dev/pandas/commit/03dd698bc1e84c35aba8b51bdd45c472860b9ec3 dataframe.setitem make a copy and never write into the existing array, so we also follow this. - **Skip update/fillna for 1.4.0-1.4.2**: https://github.com/pandas-dev/pandas/issues/47188 this issue fix the `update` and `fillna` behaviors, I confirm it had been resolved by pandas 1.4.3, so we skip the validate copy check. - **Skip eval for 1.4.0-1.4.3**: https://github.com/pandas-dev/pandas/issues/47449 `eval` still have a regression, so we skip it in 1.4.0~1.4.3. I also test 1.4.2 CI with this PR: https://github.com/Yikun/spark/pull/113, all tests passed, so we can upgrade pandas to `1.4.2` in CI after this PR merged. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
