[
https://issues.apache.org/jira/browse/SPARK-14240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15216655#comment-15216655
]
Sayak Ghosh edited comment on SPARK-14240 at 3/29/16 7:08 PM:
--------------------------------------------------------------
Now it is giving this. Is it memory issue? Please find the attached file.
was (Author: sayak.ghosh):
Now it is giving this. Is it memory issue?
> PySpark Standalone Application hangs without any Error message
> --------------------------------------------------------------
>
> Key: SPARK-14240
> URL: https://issues.apache.org/jira/browse/SPARK-14240
> Project: Spark
> Issue Type: Bug
> Components: Deploy, PySpark
> Affects Versions: 1.6.0
> Reporter: Sayak Ghosh
> Attachments: error_terminal.png
>
>
> I am relatively new to Spark and wrote a simple script using python and spark
> SQL. My problem is that it is perfectly allright at the starting phase of the
> execution but gradually it slowed down and at the end of the last phase the
> whole application hangs
> Here is my code snippet -
> hivectx.registerDataFrameAsTable(aggregatedDataV1,"aggregatedDataV1")
> q1 = "SELECT *, (Total_Sale/Sale_Weeks) as Average_Sale_Per_SaleWeek,
> (Total_Weeks/Sale_Weeks) as Velocity FROM aggregatedDataV1"
> aggregatedData = hivectx.sql(q1)
> aggregatedData.show(100)
> ========== Terminal Hanging with the following =========
> 16/03/29 09:05:50 INFO TaskSetManager: Finished task 96.0 in stage 416.0 (TID
> 19992) in 41924 ms on 10.9.0.7 (104/200)
> 16/03/29 09:05:50 INFO TaskSetManager: Finished task 108.0 in stage 416.0
> (TID 20004) in 24608 ms on 10.9.0.10 (105/200)
> 16/03/29 09:05:50 INFO TaskSetManager: Finished task 105.0 in stage 416.0
> (TID 20001) in 24610 ms on 10.9.0.10 (106/200)
> 16/03/29 09:05:55 INFO TaskSetManager: Starting task 116.0 in stage 416.0
> (TID 20012, 10.9.0.10, partition 116,NODE_LOCAL, 2240 bytes)
> 16/03/29 09:06:31 INFO TaskSetManager: Finished task 99.0 in stage 416.0 (TID
> 19995) in 78435 ms on 10.9.0.7 (110/200)
> 16/03/29 09:06:40 INFO TaskSetManager: Starting task 119.0 in stage 416.0
> (TID 20015, 10.9.0.10, partition 119,NODE_LOCAL, 2240 bytes)
> 16/03/29 09:07:12 INFO TaskSetManager: Starting task 122.0 in stage 416.0
> (TID 20018, 10.9.0.7, partition 122,NODE_LOCAL, 2240 bytes)
> 16/03/29 09:07:16 INFO TaskSetManager: Starting task 123.0 in stage 416.0
> (TID 20019, 10.9.0.7, partition 123,NODE_LOCAL, 2240 bytes)
> 16/03/29 09:07:28 INFO TaskSetManager: Finished task 111.0 in stage 416.0
> (TID 20007) in 110198 ms on 10.9.0.7 (114/200)
> 16/03/29 09:07:52 INFO TaskSetManager: Starting task 124.0 in stage 416.0
> (TID 20020, 10.9.0.10, partition 124,NODE_LOCAL, 2240 bytes)
> 16/03/29 09:08:08 INFO TaskSetManager: Finished task 110.0 in stage 416.0
> (TID 20006) in 150023 ms on 10.9.0.7 (115/200)
> 16/03/29 09:08:12 INFO TaskSetManager: Finished task 113.0 in stage 416.0
> (TID 20009) in 154120 ms on 10.9.0.7 (116/200)
> 16/03/29 09:08:16 INFO TaskSetManager: Finished task 116.0 in stage 416.0
> (TID 20012) in 145691 ms on 10.9.0.10 (117/200)
> There is no sign of error.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
