[jira] [Commented] (SPARK-13298) DAG visualization does not render correctly for jobs
[
https://issues.apache.org/jira/browse/SPARK-13298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15192324#comment-15192324
]
Todd Leo commented on SPARK-13298:
--
Pls also see SPARK-13645: DAG Diagram not shown properly in Chrome
> DAG visualization does not render correctly for jobs
>
>
> Key: SPARK-13298
> URL: https://issues.apache.org/jira/browse/SPARK-13298
> Project: Spark
> Issue Type: Bug
> Components: Web UI
>Affects Versions: 1.6.0
>Reporter: Lucas Woltmann
>Assignee: Shixiong Zhu
> Fix For: 1.6.1, 2.0.0
>
> Attachments: dag_full.png, dag_viz.png, no-dag.png
>
>
> Whenever I try to open the DAG for a job, I get something like this:
> !dag_viz.png!
> Obviously the svg doesn't get resized, but if I resize it manually, only the
> first of four stages in the DAG is shown.
> The js console says (variable v is null in peg$c34):
> {code:javascript}
> Uncaught TypeError: Cannot read property '3' of null
> peg$c34 @ graphlib-dot.min.js:1
> peg$parseidDef @ graphlib-dot.min.js:1
> peg$parseaList @ graphlib-dot.min.js:1
> peg$parseattrListBlock @ graphlib-dot.min.js:1
> peg$parseattrList @ graphlib-dot.min.js:1
> peg$parsenodeStmt @ graphlib-dot.min.js:1
> peg$parsestmt @ graphlib-dot.min.js:1
> peg$parsestmtList @ graphlib-dot.min.js:1
> peg$parsesubgraphStmt @ graphlib-dot.min.js:1
> peg$parsenodeIdOrSubgraph @ graphlib-dot.min.js:1
> peg$parseedgeStmt @ graphlib-dot.min.js:1
> peg$parsestmt @ graphlib-dot.min.js:1
> peg$parsestmtList @ graphlib-dot.min.js:1
> peg$parsesubgraphStmt @ graphlib-dot.min.js:1
> peg$parsenodeIdOrSubgraph @ graphlib-dot.min.js:1
> peg$parseedgeStmt @ graphlib-dot.min.js:1
> peg$parsestmt @ graphlib-dot.min.js:1
> peg$parsestmtList @ graphlib-dot.min.js:1
> peg$parsegraphStmt @ graphlib-dot.min.js:1
> parse @ graphlib-dot.min.js:2
> readOne @ graphlib-dot.min.js:2
> renderDot @ spark-dag-viz.js:281
> (anonymous function) @ spark-dag-viz.js:248
> (anonymous function) @ d3.min.js:
> 3Y @ d3.min.js:1
> _a.each @ d3.min.js:3
> renderDagVizForJob @ spark-dag-viz.js:207
> renderDagViz @ spark-dag-viz.js:163
> toggleDagViz @ spark-dag-viz.js:100
> onclick @ ?id=2:153
> {code}
> (tested in FIrefox 44.0.1 and Chromium 48.0.2564.103)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-13298) DAG visualization does not render correctly for jobs
[
https://issues.apache.org/jira/browse/SPARK-13298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15157633#comment-15157633
]
Shixiong Zhu commented on SPARK-13298:
--
Thanks for the reproducer. Just submit a PR to fix it.
> DAG visualization does not render correctly for jobs
>
>
> Key: SPARK-13298
> URL: https://issues.apache.org/jira/browse/SPARK-13298
> Project: Spark
> Issue Type: Bug
> Components: Web UI
>Affects Versions: 1.6.0
>Reporter: Lucas Woltmann
> Attachments: dag_full.png, dag_viz.png, no-dag.png
>
>
> Whenever I try to open the DAG for a job, I get something like this:
> !dag_viz.png!
> Obviously the svg doesn't get resized, but if I resize it manually, only the
> first of four stages in the DAG is shown.
> The js console says (variable v is null in peg$c34):
> {code:javascript}
> Uncaught TypeError: Cannot read property '3' of null
> peg$c34 @ graphlib-dot.min.js:1
> peg$parseidDef @ graphlib-dot.min.js:1
> peg$parseaList @ graphlib-dot.min.js:1
> peg$parseattrListBlock @ graphlib-dot.min.js:1
> peg$parseattrList @ graphlib-dot.min.js:1
> peg$parsenodeStmt @ graphlib-dot.min.js:1
> peg$parsestmt @ graphlib-dot.min.js:1
> peg$parsestmtList @ graphlib-dot.min.js:1
> peg$parsesubgraphStmt @ graphlib-dot.min.js:1
> peg$parsenodeIdOrSubgraph @ graphlib-dot.min.js:1
> peg$parseedgeStmt @ graphlib-dot.min.js:1
> peg$parsestmt @ graphlib-dot.min.js:1
> peg$parsestmtList @ graphlib-dot.min.js:1
> peg$parsesubgraphStmt @ graphlib-dot.min.js:1
> peg$parsenodeIdOrSubgraph @ graphlib-dot.min.js:1
> peg$parseedgeStmt @ graphlib-dot.min.js:1
> peg$parsestmt @ graphlib-dot.min.js:1
> peg$parsestmtList @ graphlib-dot.min.js:1
> peg$parsegraphStmt @ graphlib-dot.min.js:1
> parse @ graphlib-dot.min.js:2
> readOne @ graphlib-dot.min.js:2
> renderDot @ spark-dag-viz.js:281
> (anonymous function) @ spark-dag-viz.js:248
> (anonymous function) @ d3.min.js:
> 3Y @ d3.min.js:1
> _a.each @ d3.min.js:3
> renderDagVizForJob @ spark-dag-viz.js:207
> renderDagViz @ spark-dag-viz.js:163
> toggleDagViz @ spark-dag-viz.js:100
> onclick @ ?id=2:153
> {code}
> (tested in FIrefox 44.0.1 and Chromium 48.0.2564.103)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-13298) DAG visualization does not render correctly for jobs
[
https://issues.apache.org/jira/browse/SPARK-13298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15157637#comment-15157637
]
Apache Spark commented on SPARK-13298:
--
User 'zsxwing' has created a pull request for this issue:
https://github.com/apache/spark/pull/11309
> DAG visualization does not render correctly for jobs
>
>
> Key: SPARK-13298
> URL: https://issues.apache.org/jira/browse/SPARK-13298
> Project: Spark
> Issue Type: Bug
> Components: Web UI
>Affects Versions: 1.6.0
>Reporter: Lucas Woltmann
>Assignee: Shixiong Zhu
> Attachments: dag_full.png, dag_viz.png, no-dag.png
>
>
> Whenever I try to open the DAG for a job, I get something like this:
> !dag_viz.png!
> Obviously the svg doesn't get resized, but if I resize it manually, only the
> first of four stages in the DAG is shown.
> The js console says (variable v is null in peg$c34):
> {code:javascript}
> Uncaught TypeError: Cannot read property '3' of null
> peg$c34 @ graphlib-dot.min.js:1
> peg$parseidDef @ graphlib-dot.min.js:1
> peg$parseaList @ graphlib-dot.min.js:1
> peg$parseattrListBlock @ graphlib-dot.min.js:1
> peg$parseattrList @ graphlib-dot.min.js:1
> peg$parsenodeStmt @ graphlib-dot.min.js:1
> peg$parsestmt @ graphlib-dot.min.js:1
> peg$parsestmtList @ graphlib-dot.min.js:1
> peg$parsesubgraphStmt @ graphlib-dot.min.js:1
> peg$parsenodeIdOrSubgraph @ graphlib-dot.min.js:1
> peg$parseedgeStmt @ graphlib-dot.min.js:1
> peg$parsestmt @ graphlib-dot.min.js:1
> peg$parsestmtList @ graphlib-dot.min.js:1
> peg$parsesubgraphStmt @ graphlib-dot.min.js:1
> peg$parsenodeIdOrSubgraph @ graphlib-dot.min.js:1
> peg$parseedgeStmt @ graphlib-dot.min.js:1
> peg$parsestmt @ graphlib-dot.min.js:1
> peg$parsestmtList @ graphlib-dot.min.js:1
> peg$parsegraphStmt @ graphlib-dot.min.js:1
> parse @ graphlib-dot.min.js:2
> readOne @ graphlib-dot.min.js:2
> renderDot @ spark-dag-viz.js:281
> (anonymous function) @ spark-dag-viz.js:248
> (anonymous function) @ d3.min.js:
> 3Y @ d3.min.js:1
> _a.each @ d3.min.js:3
> renderDagVizForJob @ spark-dag-viz.js:207
> renderDagViz @ spark-dag-viz.js:163
> toggleDagViz @ spark-dag-viz.js:100
> onclick @ ?id=2:153
> {code}
> (tested in FIrefox 44.0.1 and Chromium 48.0.2564.103)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-13298) DAG visualization does not render correctly for jobs
[
https://issues.apache.org/jira/browse/SPARK-13298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15152057#comment-15152057
]
Lucas Woltmann commented on SPARK-13298:
Yes, of course:
{code}
from pyspark import SparkContext
from pyspark.sql import SQLContext
sc = SparkContext(appName="HierarchyGroup")
sqlContext = SQLContext(sc)
df = sqlContext.read.format('com.databricks.spark.csv').options(header='true',
inferschema='true').load('cars.csv')
stats = df \
.groupby("year", "make", "model") \
.pivot("model") \
.count() \
.cache()
stats_elem1 = stats.groupby("make").sum()
stats_elem1 = stats_elem1.unionAll(stats.groupby("year").sum())
stats_elem1.show()
{code}
I use {{--packages com.databricks:spark-csv_2.10:1.3.0}} with {{spark-submit}}.
The example csv can be found here (it's the standard example from the csv
package documentation):
{code}
wget
https://github.com/databricks/spark-csv/raw/master/src/test/resources/cars.csv
{code}
> DAG visualization does not render correctly for jobs
>
>
> Key: SPARK-13298
> URL: https://issues.apache.org/jira/browse/SPARK-13298
> Project: Spark
> Issue Type: Bug
> Components: Web UI
>Affects Versions: 1.6.0
>Reporter: Lucas Woltmann
> Attachments: dag_full.png, dag_viz.png
>
>
> Whenever I try to open the DAG for a job, I get something like this:
> !dag_viz.png!
> Obviously the svg doesn't get resized, but if I resize it manually, only the
> first of four stages in the DAG is shown.
> The js console says (variable v is null in peg$c34):
> {code:javascript}
> Uncaught TypeError: Cannot read property '3' of null
> peg$c34 @ graphlib-dot.min.js:1
> peg$parseidDef @ graphlib-dot.min.js:1
> peg$parseaList @ graphlib-dot.min.js:1
> peg$parseattrListBlock @ graphlib-dot.min.js:1
> peg$parseattrList @ graphlib-dot.min.js:1
> peg$parsenodeStmt @ graphlib-dot.min.js:1
> peg$parsestmt @ graphlib-dot.min.js:1
> peg$parsestmtList @ graphlib-dot.min.js:1
> peg$parsesubgraphStmt @ graphlib-dot.min.js:1
> peg$parsenodeIdOrSubgraph @ graphlib-dot.min.js:1
> peg$parseedgeStmt @ graphlib-dot.min.js:1
> peg$parsestmt @ graphlib-dot.min.js:1
> peg$parsestmtList @ graphlib-dot.min.js:1
> peg$parsesubgraphStmt @ graphlib-dot.min.js:1
> peg$parsenodeIdOrSubgraph @ graphlib-dot.min.js:1
> peg$parseedgeStmt @ graphlib-dot.min.js:1
> peg$parsestmt @ graphlib-dot.min.js:1
> peg$parsestmtList @ graphlib-dot.min.js:1
> peg$parsegraphStmt @ graphlib-dot.min.js:1
> parse @ graphlib-dot.min.js:2
> readOne @ graphlib-dot.min.js:2
> renderDot @ spark-dag-viz.js:281
> (anonymous function) @ spark-dag-viz.js:248
> (anonymous function) @ d3.min.js:
> 3Y @ d3.min.js:1
> _a.each @ d3.min.js:3
> renderDagVizForJob @ spark-dag-viz.js:207
> renderDagViz @ spark-dag-viz.js:163
> toggleDagViz @ spark-dag-viz.js:100
> onclick @ ?id=2:153
> {code}
> (tested in FIrefox 44.0.1 and Chromium 48.0.2564.103)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-13298) DAG visualization does not render correctly for jobs
[
https://issues.apache.org/jira/browse/SPARK-13298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15149457#comment-15149457
]
Shixiong Zhu commented on SPARK-13298:
--
Do you have a reproducer?
> DAG visualization does not render correctly for jobs
>
>
> Key: SPARK-13298
> URL: https://issues.apache.org/jira/browse/SPARK-13298
> Project: Spark
> Issue Type: Bug
> Components: Web UI
>Affects Versions: 1.6.0
>Reporter: Lucas Woltmann
> Attachments: dag_full.png, dag_viz.png
>
>
> Whenever I try to open the DAG for a job, I get something like this:
> !dag_viz.png!
> Obviously the svg doesn't get resized, but if I resize it manually, only the
> first of four stages in the DAG is shown.
> The js console says (variable v is null in peg$c34):
> {code:javascript}
> Uncaught TypeError: Cannot read property '3' of null
> peg$c34 @ graphlib-dot.min.js:1
> peg$parseidDef @ graphlib-dot.min.js:1
> peg$parseaList @ graphlib-dot.min.js:1
> peg$parseattrListBlock @ graphlib-dot.min.js:1
> peg$parseattrList @ graphlib-dot.min.js:1
> peg$parsenodeStmt @ graphlib-dot.min.js:1
> peg$parsestmt @ graphlib-dot.min.js:1
> peg$parsestmtList @ graphlib-dot.min.js:1
> peg$parsesubgraphStmt @ graphlib-dot.min.js:1
> peg$parsenodeIdOrSubgraph @ graphlib-dot.min.js:1
> peg$parseedgeStmt @ graphlib-dot.min.js:1
> peg$parsestmt @ graphlib-dot.min.js:1
> peg$parsestmtList @ graphlib-dot.min.js:1
> peg$parsesubgraphStmt @ graphlib-dot.min.js:1
> peg$parsenodeIdOrSubgraph @ graphlib-dot.min.js:1
> peg$parseedgeStmt @ graphlib-dot.min.js:1
> peg$parsestmt @ graphlib-dot.min.js:1
> peg$parsestmtList @ graphlib-dot.min.js:1
> peg$parsegraphStmt @ graphlib-dot.min.js:1
> parse @ graphlib-dot.min.js:2
> readOne @ graphlib-dot.min.js:2
> renderDot @ spark-dag-viz.js:281
> (anonymous function) @ spark-dag-viz.js:248
> (anonymous function) @ d3.min.js:
> 3Y @ d3.min.js:1
> _a.each @ d3.min.js:3
> renderDagVizForJob @ spark-dag-viz.js:207
> renderDagViz @ spark-dag-viz.js:163
> toggleDagViz @ spark-dag-viz.js:100
> onclick @ ?id=2:153
> {code}
> (tested in FIrefox 44.0.1 and Chromium 48.0.2564.103)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-13298) DAG visualization does not render correctly for jobs
[
https://issues.apache.org/jira/browse/SPARK-13298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15148232#comment-15148232
]
Lucas Woltmann commented on SPARK-13298:
Looks like .cache() breaks it.
DAG without cache(): !dag_full-png!
> DAG visualization does not render correctly for jobs
>
>
> Key: SPARK-13298
> URL: https://issues.apache.org/jira/browse/SPARK-13298
> Project: Spark
> Issue Type: Bug
> Components: Web UI
>Affects Versions: 1.6.0
>Reporter: Lucas Woltmann
> Attachments: dag_viz.png
>
>
> Whenever I try to open the DAG for a job, I get something like this:
> !dag_viz.png!
> Obviously the svg doesn't get resized, but if I resize it manually, only the
> first of four stages in the DAG is shown.
> The js console says (variable v is null in peg$c34):
> {code:javascript}
> Uncaught TypeError: Cannot read property '3' of null
> peg$c34 @ graphlib-dot.min.js:1
> peg$parseidDef @ graphlib-dot.min.js:1
> peg$parseaList @ graphlib-dot.min.js:1
> peg$parseattrListBlock @ graphlib-dot.min.js:1
> peg$parseattrList @ graphlib-dot.min.js:1
> peg$parsenodeStmt @ graphlib-dot.min.js:1
> peg$parsestmt @ graphlib-dot.min.js:1
> peg$parsestmtList @ graphlib-dot.min.js:1
> peg$parsesubgraphStmt @ graphlib-dot.min.js:1
> peg$parsenodeIdOrSubgraph @ graphlib-dot.min.js:1
> peg$parseedgeStmt @ graphlib-dot.min.js:1
> peg$parsestmt @ graphlib-dot.min.js:1
> peg$parsestmtList @ graphlib-dot.min.js:1
> peg$parsesubgraphStmt @ graphlib-dot.min.js:1
> peg$parsenodeIdOrSubgraph @ graphlib-dot.min.js:1
> peg$parseedgeStmt @ graphlib-dot.min.js:1
> peg$parsestmt @ graphlib-dot.min.js:1
> peg$parsestmtList @ graphlib-dot.min.js:1
> peg$parsegraphStmt @ graphlib-dot.min.js:1
> parse @ graphlib-dot.min.js:2
> readOne @ graphlib-dot.min.js:2
> renderDot @ spark-dag-viz.js:281
> (anonymous function) @ spark-dag-viz.js:248
> (anonymous function) @ d3.min.js:
> 3Y @ d3.min.js:1
> _a.each @ d3.min.js:3
> renderDagVizForJob @ spark-dag-viz.js:207
> renderDagViz @ spark-dag-viz.js:163
> toggleDagViz @ spark-dag-viz.js:100
> onclick @ ?id=2:153
> {code}
> (tested in FIrefox 44.0.1 and Chromium 48.0.2564.103)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
