[jira] [Commented] (SPARK-31967) Loading jobs UI page takes 40 seconds
[ https://issues.apache.org/jira/browse/SPARK-31967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17135271#comment-17135271 ] Xiao Li commented on SPARK-31967: - [~Gengliang.Wang] Please also change the fix versions when you merged the PRs. > Loading jobs UI page takes 40 seconds > - > > Key: SPARK-31967 > URL: https://issues.apache.org/jira/browse/SPARK-31967 > Project: Spark > Issue Type: Bug > Components: Web UI >Affects Versions: 2.4.6, 3.0.1 >Reporter: Gengliang Wang >Assignee: Gengliang Wang >Priority: Blocker > Fix For: 3.0.1, 3.1.0, 2.4.7 > > Attachments: load_time.jpeg, profile.png > > > In the latest master branch, I find that the job list page becomes very slow. > To reproduce in local setup: > {code:java} > spark.read.parquet("/tmp/p1").createOrReplaceTempView("t1") > spark.read.parquet("/tmp/p2").createOrReplaceTempView("t2") > (1 to 1000).map(_ => spark.sql("select * from t1, t2 where > t1.value=t2.value").show()) > {code} > And that, open live UI: http://localhost:4040/ > The loading time is about 40 seconds. > If we comment out the function call for `drawApplicationTimeline`, then the > loading time is around 1 second. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-31967) Loading jobs UI page takes 40 seconds
[ https://issues.apache.org/jira/browse/SPARK-31967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17135270#comment-17135270 ] Xiao Li commented on SPARK-31967: - nvm. This has been merged to 2.4 branch. > Loading jobs UI page takes 40 seconds > - > > Key: SPARK-31967 > URL: https://issues.apache.org/jira/browse/SPARK-31967 > Project: Spark > Issue Type: Bug > Components: Web UI >Affects Versions: 2.4.6, 3.0.1 >Reporter: Gengliang Wang >Assignee: Gengliang Wang >Priority: Blocker > Fix For: 3.0.1, 3.1.0 > > Attachments: load_time.jpeg, profile.png > > > In the latest master branch, I find that the job list page becomes very slow. > To reproduce in local setup: > {code:java} > spark.read.parquet("/tmp/p1").createOrReplaceTempView("t1") > spark.read.parquet("/tmp/p2").createOrReplaceTempView("t2") > (1 to 1000).map(_ => spark.sql("select * from t1, t2 where > t1.value=t2.value").show()) > {code} > And that, open live UI: http://localhost:4040/ > The loading time is about 40 seconds. > If we comment out the function call for `drawApplicationTimeline`, then the > loading time is around 1 second. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-31967) Loading jobs UI page takes 40 seconds
[ https://issues.apache.org/jira/browse/SPARK-31967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17135269#comment-17135269 ] Xiao Li commented on SPARK-31967: - Do we need to resolve the same issue in 2.4.6? > Loading jobs UI page takes 40 seconds > - > > Key: SPARK-31967 > URL: https://issues.apache.org/jira/browse/SPARK-31967 > Project: Spark > Issue Type: Bug > Components: Web UI >Affects Versions: 3.0.1 >Reporter: Gengliang Wang >Assignee: Gengliang Wang >Priority: Blocker > Fix For: 3.0.1, 3.1.0 > > Attachments: load_time.jpeg, profile.png > > > In the latest master branch, I find that the job list page becomes very slow. > To reproduce in local setup: > {code:java} > spark.read.parquet("/tmp/p1").createOrReplaceTempView("t1") > spark.read.parquet("/tmp/p2").createOrReplaceTempView("t2") > (1 to 1000).map(_ => spark.sql("select * from t1, t2 where > t1.value=t2.value").show()) > {code} > And that, open live UI: http://localhost:4040/ > The loading time is about 40 seconds. > If we comment out the function call for `drawApplicationTimeline`, then the > loading time is around 1 second. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-31967) Loading jobs UI page takes 40 seconds
[ https://issues.apache.org/jira/browse/SPARK-31967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17134608#comment-17134608 ] Gengliang Wang commented on SPARK-31967: The issue is resoveld in https://github.com/apache/spark/pull/28806 > Loading jobs UI page takes 40 seconds > - > > Key: SPARK-31967 > URL: https://issues.apache.org/jira/browse/SPARK-31967 > Project: Spark > Issue Type: Bug > Components: Web UI >Affects Versions: 3.0.1 >Reporter: Gengliang Wang >Priority: Blocker > Attachments: load_time.jpeg, profile.png > > > In the latest master branch, I find that the job list page becomes very slow. > To reproduce in local setup: > {code:java} > spark.read.parquet("/tmp/p1").createOrReplaceTempView("t1") > spark.read.parquet("/tmp/p2").createOrReplaceTempView("t2") > (1 to 1000).map(_ => spark.sql("select * from t1, t2 where > t1.value=t2.value").show()) > {code} > And that, open live UI: http://localhost:4040/ > The loading time is about 40 seconds. > If we comment out the function call for `drawApplicationTimeline`, then the > loading time is around 1 second. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-31967) Loading jobs UI page takes 40 seconds
[ https://issues.apache.org/jira/browse/SPARK-31967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17134050#comment-17134050 ] Apache Spark commented on SPARK-31967: -- User 'gengliangwang' has created a pull request for this issue: https://github.com/apache/spark/pull/28813 > Loading jobs UI page takes 40 seconds > - > > Key: SPARK-31967 > URL: https://issues.apache.org/jira/browse/SPARK-31967 > Project: Spark > Issue Type: Bug > Components: Web UI >Affects Versions: 3.0.1 >Reporter: Gengliang Wang >Priority: Blocker > Attachments: load_time.jpeg, profile.png > > > In the latest master branch, I find that the job list page becomes very slow. > To reproduce in local setup: > {code:java} > spark.read.parquet("/tmp/p1").createOrReplaceTempView("t1") > spark.read.parquet("/tmp/p2").createOrReplaceTempView("t2") > (1 to 1000).map(_ => spark.sql("select * from t1, t2 where > t1.value=t2.value").show()) > {code} > And that, open live UI: http://localhost:4040/ > The loading time is about 40 seconds. > If we comment out the function call for `drawApplicationTimeline`, then the > loading time is around 1 second. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-31967) Loading jobs UI page takes 40 seconds
[ https://issues.apache.org/jira/browse/SPARK-31967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17134046#comment-17134046 ] Apache Spark commented on SPARK-31967: -- User 'gengliangwang' has created a pull request for this issue: https://github.com/apache/spark/pull/28811 > Loading jobs UI page takes 40 seconds > - > > Key: SPARK-31967 > URL: https://issues.apache.org/jira/browse/SPARK-31967 > Project: Spark > Issue Type: Bug > Components: Web UI >Affects Versions: 3.0.1 >Reporter: Gengliang Wang >Priority: Blocker > Attachments: load_time.jpeg, profile.png > > > In the latest master branch, I find that the job list page becomes very slow. > To reproduce in local setup: > {code:java} > spark.read.parquet("/tmp/p1").createOrReplaceTempView("t1") > spark.read.parquet("/tmp/p2").createOrReplaceTempView("t2") > (1 to 1000).map(_ => spark.sql("select * from t1, t2 where > t1.value=t2.value").show()) > {code} > And that, open live UI: http://localhost:4040/ > The loading time is about 40 seconds. > If we comment out the function call for `drawApplicationTimeline`, then the > loading time is around 1 second. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-31967) Loading jobs UI page takes 40 seconds
[ https://issues.apache.org/jira/browse/SPARK-31967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17134043#comment-17134043 ] Apache Spark commented on SPARK-31967: -- User 'gengliangwang' has created a pull request for this issue: https://github.com/apache/spark/pull/28811 > Loading jobs UI page takes 40 seconds > - > > Key: SPARK-31967 > URL: https://issues.apache.org/jira/browse/SPARK-31967 > Project: Spark > Issue Type: Bug > Components: Web UI >Affects Versions: 3.0.1 >Reporter: Gengliang Wang >Priority: Blocker > Attachments: load_time.jpeg, profile.png > > > In the latest master branch, I find that the job list page becomes very slow. > To reproduce in local setup: > {code:java} > spark.read.parquet("/tmp/p1").createOrReplaceTempView("t1") > spark.read.parquet("/tmp/p2").createOrReplaceTempView("t2") > (1 to 1000).map(_ => spark.sql("select * from t1, t2 where > t1.value=t2.value").show()) > {code} > And that, open live UI: http://localhost:4040/ > The loading time is about 40 seconds. > If we comment out the function call for `drawApplicationTimeline`, then the > loading time is around 1 second. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-31967) Loading jobs UI page takes 40 seconds
[ https://issues.apache.org/jira/browse/SPARK-31967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17133931#comment-17133931 ] Apache Spark commented on SPARK-31967: -- User 'gengliangwang' has created a pull request for this issue: https://github.com/apache/spark/pull/28806 > Loading jobs UI page takes 40 seconds > - > > Key: SPARK-31967 > URL: https://issues.apache.org/jira/browse/SPARK-31967 > Project: Spark > Issue Type: Bug > Components: Web UI >Affects Versions: 3.0.1 >Reporter: Gengliang Wang >Priority: Blocker > Attachments: load_time.jpeg, profile.png > > > In the latest master branch, I find that the job list page becomes very slow. > To reproduce in local setup: > {code:java} > spark.read.parquet("/tmp/p1").createOrReplaceTempView("t1") > spark.read.parquet("/tmp/p2").createOrReplaceTempView("t2") > (1 to 1000).map(_ => spark.sql("select * from t1, t2 where > t1.value=t2.value").show()) > {code} > And that, open live UI: http://localhost:4040/ > The loading time is about 40 seconds. > If we comment out the function call for `drawApplicationTimeline`, then the > loading time is around 1 second. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-31967) Loading jobs UI page takes 40 seconds
[ https://issues.apache.org/jira/browse/SPARK-31967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17133930#comment-17133930 ] Apache Spark commented on SPARK-31967: -- User 'gengliangwang' has created a pull request for this issue: https://github.com/apache/spark/pull/28806 > Loading jobs UI page takes 40 seconds > - > > Key: SPARK-31967 > URL: https://issues.apache.org/jira/browse/SPARK-31967 > Project: Spark > Issue Type: Bug > Components: Web UI >Affects Versions: 3.0.1 >Reporter: Gengliang Wang >Priority: Blocker > Attachments: load_time.jpeg, profile.png > > > In the latest master branch, I find that the job list page becomes very slow. > To reproduce in local setup: > {code:java} > spark.read.parquet("/tmp/p1").createOrReplaceTempView("t1") > spark.read.parquet("/tmp/p2").createOrReplaceTempView("t2") > (1 to 1000).map(_ => spark.sql("select * from t1, t2 where > t1.value=t2.value").show()) > {code} > And that, open live UI: http://localhost:4040/ > The loading time is about 40 seconds. > If we comment out the function call for `drawApplicationTimeline`, then the > loading time is around 1 second. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-31967) Loading jobs UI page takes 40 seconds
[ https://issues.apache.org/jira/browse/SPARK-31967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17133109#comment-17133109 ] Kousuke Saruta commented on SPARK-31967: For now, I'm considering a solution like StagePage does. > Loading jobs UI page takes 40 seconds > - > > Key: SPARK-31967 > URL: https://issues.apache.org/jira/browse/SPARK-31967 > Project: Spark > Issue Type: Bug > Components: Web UI >Affects Versions: 3.0.1 >Reporter: Gengliang Wang >Priority: Blocker > Attachments: load_time.jpeg, profile.png > > > In the latest master branch, I find that the job list page becomes very slow. > To reproduce in local setup: > {code:java} > spark.read.parquet("/tmp/p1").createOrReplaceTempView("t1") > spark.read.parquet("/tmp/p2").createOrReplaceTempView("t2") > (1 to 1000).map(_ => spark.sql("select * from t1, t2 where > t1.value=t2.value").show()) > {code} > And that, open live UI: http://localhost:4040/ > The loading time is about 40 seconds. > If we comment out the function call for `drawApplicationTimeline`, then the > loading time is around 1 second. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-31967) Loading jobs UI page takes 40 seconds
[ https://issues.apache.org/jira/browse/SPARK-31967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17133058#comment-17133058 ] Gengliang Wang commented on SPARK-31967: Thanks. Also, not directly related to this topic, I think we don't need to render the timeline in the page unless the "event timeline" link is clicked..we can have follow-up improvements. > Loading jobs UI page takes 40 seconds > - > > Key: SPARK-31967 > URL: https://issues.apache.org/jira/browse/SPARK-31967 > Project: Spark > Issue Type: Bug > Components: Web UI >Affects Versions: 3.0.1 >Reporter: Gengliang Wang >Priority: Blocker > Attachments: load_time.jpeg, profile.png > > > In the latest master branch, I find that the job list page becomes very slow. > To reproduce in local setup: > {code:java} > spark.read.parquet("/tmp/p1").createOrReplaceTempView("t1") > spark.read.parquet("/tmp/p2").createOrReplaceTempView("t2") > (1 to 1000).map(_ => spark.sql("select * from t1, t2 where > t1.value=t2.value").show()) > {code} > And that, open live UI: http://localhost:4040/ > The loading time is about 40 seconds. > If we comment out the function call for `drawApplicationTimeline`, then the > loading time is around 1 second. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-31967) Loading jobs UI page takes 40 seconds
[ https://issues.apache.org/jira/browse/SPARK-31967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17133049#comment-17133049 ] Kousuke Saruta commented on SPARK-31967: This issue can be related. [https://github.com/visjs/vis-timeline/issues/379] > Loading jobs UI page takes 40 seconds > - > > Key: SPARK-31967 > URL: https://issues.apache.org/jira/browse/SPARK-31967 > Project: Spark > Issue Type: Bug > Components: Web UI >Affects Versions: 3.0.1 >Reporter: Gengliang Wang >Priority: Blocker > Attachments: load_time.jpeg, profile.png > > > In the latest master branch, I find that the job list page becomes very slow. > To reproduce in local setup: > {code:java} > spark.read.parquet("/tmp/p1").createOrReplaceTempView("t1") > spark.read.parquet("/tmp/p2").createOrReplaceTempView("t2") > (1 to 1000).map(_ => spark.sql("select * from t1, t2 where > t1.value=t2.value").show()) > {code} > And that, open live UI: http://localhost:4040/ > The loading time is about 40 seconds. > If we comment out the function call for `drawApplicationTimeline`, then the > loading time is around 1 second. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-31967) Loading jobs UI page takes 40 seconds
[ https://issues.apache.org/jira/browse/SPARK-31967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17133039#comment-17133039 ] Kousuke Saruta commented on SPARK-31967: I'll investigate. > Loading jobs UI page takes 40 seconds > - > > Key: SPARK-31967 > URL: https://issues.apache.org/jira/browse/SPARK-31967 > Project: Spark > Issue Type: Bug > Components: Web UI >Affects Versions: 3.0.1 >Reporter: Gengliang Wang >Priority: Blocker > Attachments: load_time.jpeg, profile.png > > > In the latest master branch, I find that the job list page becomes very slow. > To reproduce in local setup: > {code:java} > spark.read.parquet("/tmp/p1").createOrReplaceTempView("t1") > spark.read.parquet("/tmp/p2").createOrReplaceTempView("t2") > (1 to 1000).map(_ => spark.sql("select * from t1, t2 where > t1.value=t2.value").show()) > {code} > And that, open live UI: http://localhost:4040/ > The loading time is about 40 seconds. > If we comment out the function call for `drawApplicationTimeline`, then the > loading time is around 1 second. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org