Github user opme commented on the issue: https://github.com/apache/spark/pull/14995 @witgo I have a Pyspark application that was failing in 3 different places but is able to run without errors now. I'm glad for this patch as I am not sure how I would have explained to my professors why the big data application I chose to do my analysis has 32 bit limitations. This is my final project for a Georgia Tech Big data class and I will write about the these limitations of Spark in my paper. My app is called the Surgeon Scorecard and it computes surgical complication rate for surgeons on the Medicare synthetic cms dataset which is about 1.6 billion records. https://github.com/opme/SurgeonScorecard.
--- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org