[
https://issues.apache.org/jira/browse/SPARK-27019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16781604#comment-16781604
]
peay commented on SPARK-27019:
------------------------------
OK, I can actually reproduce it pretty easily with pyspark:
{code:java}
df_test = spark.range(1024 * 1024 * 1024 * 10).toPandas(){code}
This makes the tasks fail because my executors don't have enough memory, which
seems to be key to hitting the issue. Using only 1000 elements, the job
succeeds and it does not trigger the issue.
!query-1-list.png!
!query-1-details.png!
!query-job-1.png!
> Spark UI's SQL tab shows inconsistent values
> --------------------------------------------
>
> Key: SPARK-27019
> URL: https://issues.apache.org/jira/browse/SPARK-27019
> Project: Spark
> Issue Type: Bug
> Components: SQL, Web UI
> Affects Versions: 2.4.0
> Reporter: peay
> Priority: Major
> Attachments: query-1-details.png, query-1-list.png, query-job-1.png,
> screenshot-spark-ui-details.png, screenshot-spark-ui-list.png
>
>
> Since 2.4.0, I am frequently seeing broken outputs in the SQL tab of the
> Spark UI, where submitted/duration make no sense, description has the ID
> instead of the actual description.
> Clicking on the link to open a query, the SQL plan is missing as well.
> I have tried to increase `spark.scheduler.listenerbus.eventqueue.capacity` to
> very large values like 30k out of paranoia that we may have too many events,
> but to no avail. I have not identified anything particular that leads to
> that: it doesn't occur in all my jobs, but it does occur in a lot of them
> still.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
