[jira] [Commented] (SPARK-34510) .foreachPartition command hangs when ran inside Python package but works when ran from Python file outside the package on EMR

2021-03-30 Thread Yuriy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17311978#comment-17311978 ] Yuriy commented on SPARK-34510: --- Sean, the code is not actually performing any S3 operatio

[jira] [Updated] (SPARK-34510) .foreachPartition command hangs when ran inside Python package but works when ran from Python file outside the package on EMR

2021-02-23 Thread Yuriy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuriy updated SPARK-34510: -- Description: I'm running on EMR Pyspark 3.0.0. with project structure below, process.py is what controls the

[jira] [Updated] (SPARK-34510) .foreachPartition command hangs when ran inside Python package but works when ran from Python file outside the package on EMR

2021-02-23 Thread Yuriy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuriy updated SPARK-34510: -- Description: I'm running on EMR Pyspark 3.0.0. with a project structure below, process.py is what controls th

[jira] [Updated] (SPARK-34510) .foreachPartition command hangs when ran inside Python package but works when ran from Python file outside the package on EMR

2021-02-23 Thread Yuriy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuriy updated SPARK-34510: -- Description: I'm running on EMR Pyspark 3.0.0. with a project structure below, process.py is what controls th

[jira] [Updated] (SPARK-34510) .foreachPartition command hangs when ran inside Python package but works when ran from Python file outside the package on EMR

2021-02-23 Thread Yuriy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuriy updated SPARK-34510: -- Description: I'm running on EMR Pyspark 3.0.0. with a project structure below, process.py is what controls th

[jira] [Updated] (SPARK-34510) .foreachPartition command hangs when ran inside Python package but works when ran from Python file outside the package on EMR

2021-02-23 Thread Yuriy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuriy updated SPARK-34510: -- Description: I'm running on EMR Pyspark 3.0.0. with a project structure below, process.py is what controls th

[jira] [Updated] (SPARK-34510) .foreachPartition command hangs when ran inside Python package but works when ran from Python file outside the package on EMR

2021-02-23 Thread Yuriy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuriy updated SPARK-34510: -- Summary: .foreachPartition command hangs when ran inside Python package but works when ran from Python file ou

[jira] [Updated] (SPARK-34510) .foreachPartition command hangs when ran inside Python package but works when ran from Python file outside the package

2021-02-23 Thread Yuriy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuriy updated SPARK-34510: -- Attachment: Code.zip > .foreachPartition command hangs when ran inside Python package but works when > ran fr

[jira] [Created] (SPARK-34510) .foreachPartition command hangs when ran inside Python package but works when ran from Python file outside the package

2021-02-23 Thread Yuriy (Jira)
Yuriy created SPARK-34510: - Summary: .foreachPartition command hangs when ran inside Python package but works when ran from Python file outside the package Key: SPARK-34510 URL: https://issues.apache.org/jira/browse/SPARK