[ 
https://issues.apache.org/jira/browse/SPARK-41365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mars updated SPARK-41365:
-------------------------
    Description: 
My environment CDH 5.8 , click to enter the spark UI from the yarn interface
when visit the stage URI, it fails to load,  URI is
{code:java}
http://<yarn-url>:8088/proxy/application_1669877165233_0021/stages/stage/?id=0&attempt=0
 {code}
!image-2022-12-02-17-53-03-003.png|width=430,height=697!

Server error stack trace:
{code:java}
Caused by: java.lang.NullPointerException
        at 
org.apache.spark.status.api.v1.StagesResource.$anonfun$doPagination$1(StagesResource.scala:207)
        at 
org.apache.spark.status.api.v1.BaseAppResource.$anonfun$withUI$1(ApiRootResource.scala:142)
        at org.apache.spark.ui.SparkUI.withSparkUI(SparkUI.scala:147)
        at 
org.apache.spark.status.api.v1.BaseAppResource.withUI(ApiRootResource.scala:137)
        at 
org.apache.spark.status.api.v1.BaseAppResource.withUI$(ApiRootResource.scala:135)
        at 
org.apache.spark.status.api.v1.StagesResource.withUI(StagesResource.scala:31)
        at 
org.apache.spark.status.api.v1.StagesResource.doPagination(StagesResource.scala:206)
        at 
org.apache.spark.status.api.v1.StagesResource.$anonfun$taskTable$1(StagesResource.scala:161)
        at 
org.apache.spark.status.api.v1.BaseAppResource.$anonfun$withUI$1(ApiRootResource.scala:142)
        at org.apache.spark.ui.SparkUI.withSparkUI(SparkUI.scala:147)
        at 
org.apache.spark.status.api.v1.BaseAppResource.withUI(ApiRootResource.scala:137)
        at 
org.apache.spark.status.api.v1.BaseAppResource.withUI$(ApiRootResource.scala:135)
        at 
org.apache.spark.status.api.v1.StagesResource.withUI(StagesResource.scala:31)
        at 
org.apache.spark.status.api.v1.StagesResource.taskTable(StagesResource.scala:145)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62){code}
 

The issue is similar to, the final phenomenon of the issue is the same, because 
the parameter encode twice
https://issues.apache.org/jira/browse/SPARK-32467
https://issues.apache.org/jira/browse/SPARK-33611

The two issues solve two scenarios to avoid encode twice:
1. https redirect proxy
2. set reverse proxy enabled (spark.ui.reverseProxy)  in Nginx 

But if encode twice due to other reasons, such as this issue (yarn proxy), it 
will also fail

  was:
My environment CDH 5.8 , click to enter the spark UI from the yarn interface
when visit the stage URI, it fails to load,  URI is
{code:java}
http://<yarn-url>:8088/proxy/application_1669877165233_0021/stages/stage/?id=0&attempt=0
 {code}
!image-2022-12-02-17-53-03-003.png|width=430,height=697!

Server error stack trace:
{code}
Caused by: java.lang.NullPointerException
        at 
org.apache.spark.status.api.v1.StagesResource.$anonfun$doPagination$1(StagesResource.scala:207)
        at 
org.apache.spark.status.api.v1.BaseAppResource.$anonfun$withUI$1(ApiRootResource.scala:142)
        at org.apache.spark.ui.SparkUI.withSparkUI(SparkUI.scala:147)
        at 
org.apache.spark.status.api.v1.BaseAppResource.withUI(ApiRootResource.scala:137)
        at 
org.apache.spark.status.api.v1.BaseAppResource.withUI$(ApiRootResource.scala:135)
        at 
org.apache.spark.status.api.v1.StagesResource.withUI(StagesResource.scala:31)
        at 
org.apache.spark.status.api.v1.StagesResource.doPagination(StagesResource.scala:206)
        at 
org.apache.spark.status.api.v1.StagesResource.$anonfun$taskTable$1(StagesResource.scala:161)
        at 
org.apache.spark.status.api.v1.BaseAppResource.$anonfun$withUI$1(ApiRootResource.scala:142)
        at org.apache.spark.ui.SparkUI.withSparkUI(SparkUI.scala:147)
        at 
org.apache.spark.status.api.v1.BaseAppResource.withUI(ApiRootResource.scala:137)
        at 
org.apache.spark.status.api.v1.BaseAppResource.withUI$(ApiRootResource.scala:135)
        at 
org.apache.spark.status.api.v1.StagesResource.withUI(StagesResource.scala:31)
        at 
org.apache.spark.status.api.v1.StagesResource.taskTable(StagesResource.scala:145)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62){code}


> Stages UI page fails to load for proxy in some yarn versions 
> -------------------------------------------------------------
>
>                 Key: SPARK-41365
>                 URL: https://issues.apache.org/jira/browse/SPARK-41365
>             Project: Spark
>          Issue Type: Bug
>          Components: Web UI
>    Affects Versions: 3.3.1
>         Environment: as above
>            Reporter: Mars
>            Priority: Major
>         Attachments: image-2022-12-02-17-53-03-003.png
>
>
> My environment CDH 5.8 , click to enter the spark UI from the yarn interface
> when visit the stage URI, it fails to load,  URI is
> {code:java}
> http://<yarn-url>:8088/proxy/application_1669877165233_0021/stages/stage/?id=0&attempt=0
>  {code}
> !image-2022-12-02-17-53-03-003.png|width=430,height=697!
> Server error stack trace:
> {code:java}
> Caused by: java.lang.NullPointerException
>       at 
> org.apache.spark.status.api.v1.StagesResource.$anonfun$doPagination$1(StagesResource.scala:207)
>       at 
> org.apache.spark.status.api.v1.BaseAppResource.$anonfun$withUI$1(ApiRootResource.scala:142)
>       at org.apache.spark.ui.SparkUI.withSparkUI(SparkUI.scala:147)
>       at 
> org.apache.spark.status.api.v1.BaseAppResource.withUI(ApiRootResource.scala:137)
>       at 
> org.apache.spark.status.api.v1.BaseAppResource.withUI$(ApiRootResource.scala:135)
>       at 
> org.apache.spark.status.api.v1.StagesResource.withUI(StagesResource.scala:31)
>       at 
> org.apache.spark.status.api.v1.StagesResource.doPagination(StagesResource.scala:206)
>       at 
> org.apache.spark.status.api.v1.StagesResource.$anonfun$taskTable$1(StagesResource.scala:161)
>       at 
> org.apache.spark.status.api.v1.BaseAppResource.$anonfun$withUI$1(ApiRootResource.scala:142)
>       at org.apache.spark.ui.SparkUI.withSparkUI(SparkUI.scala:147)
>       at 
> org.apache.spark.status.api.v1.BaseAppResource.withUI(ApiRootResource.scala:137)
>       at 
> org.apache.spark.status.api.v1.BaseAppResource.withUI$(ApiRootResource.scala:135)
>       at 
> org.apache.spark.status.api.v1.StagesResource.withUI(StagesResource.scala:31)
>       at 
> org.apache.spark.status.api.v1.StagesResource.taskTable(StagesResource.scala:145)
>       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>       at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62){code}
>  
> The issue is similar to, the final phenomenon of the issue is the same, 
> because the parameter encode twice
> https://issues.apache.org/jira/browse/SPARK-32467
> https://issues.apache.org/jira/browse/SPARK-33611
> The two issues solve two scenarios to avoid encode twice:
> 1. https redirect proxy
> 2. set reverse proxy enabled (spark.ui.reverseProxy)  in Nginx 
> But if encode twice due to other reasons, such as this issue (yarn proxy), it 
> will also fail



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

Reply via email to