[
https://issues.apache.org/jira/browse/LIVY-507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16611362#comment-16611362
]
Glenn McCall edited comment on LIVY-507 at 9/11/18 11:28 PM:
-------------------------------------------------------------
Hi Saiso,
I understand what the change is and how it faithfully reflects the output given
in Scala REPL.
However, IMHO, I do not see Zeppelin as a Scala REPL. Zeppelin is more of a
notebook that has a similar look and feel to a WIki - the big feature is the
ability to have live code that produces nice output for perusal by end users
and in our case, and I'm sure many others presentation to non-technical users
such as management. When looking at a Wiki page you are not exposed to the Gory
details of HTML, CSS and so on - you see the final formatted output. I feel
that Zeppelin should be the same - you should be able to see the final
formatted output without explicitly having to hide the gory intermediate REPL
outputs via a series of manual efforts that must be performed for each and
every paragraph in each and every notebook.
Faithfully capturing the entire REPL output is clever, but it messes up the
notebook.
IMHO, if you want to see REPL output, then use a REPL such as SBT,
Spark-Shell, IntelliJ etc. There are other advantages to using a real REPL as
you get context sensitive "code completion" in the REPL.
While I can understand the need, maybe, for having debugging output which shows
the entire REPL output so you can see what is going on, there should at least
be a method (under control of the notebook code) that *manually turns this on*.
There should, IMHO, be backward comparability, so, show the output of a
paragraph as it was in 0.4 (i.e. only the last statement output is shown) and
an *option*, which can manually be turned on to *dump* the entire REPL output
if required. Just as with a Wiki page, if you really need to see the HTML, CSS,
Javascript and so on, you can manually show the page source within the browser.
The proposed workarounds to hide the REPL output such as:
* splitting a paragraph into two being a) preparatory piece of code - in which
code and output are hidden and b) a single println statement (or as per the
justification in the original Jira) two print statements in a single paragraph
* Enclosing the entire code segment in to a block (i.e. putting braces around
it)
* Others
Create their own problems IMHO
For example:
Splitting the paragraph:
* If you wish to refresh your output, you have to remember to run two
paragraphs - the preparatory code and the display code (and you have to train
your users to do that as well - previously you could say if you want to refresh
this chart, just click it's run button - now you have to tell them to click
it's run button, but first click this seemingly unrelated other run button(s)).
* The extra paragraphs containing preparatory code consume some screen real
estate. If, like me, you want to modularise and have paragraphs for specific
purposes, you will end up with several paragraphs consuming screen real estate
just for the setup code.
* Whether you modularise or not, it is possible that you will end up with
large paragraphs (e.g. 40-50 lines of scala code). If you have a runtime error
it can often be difficult to see because:
** Previously, when only the last output was displayed, the exception will
appear *at the top* of the output area of the paragraph.
** Now, when all REPL output is displayed it can be difficult to tell if your
code has a runtime error as it will often be off the bottom of the output area
(or off the bottom of the browser). And if the little "Status indicator" next
to the run paragraph button is not visible (i.e. you run the paragraph via the
keyboard shortcuts) it can be difficult to just see whether there is a problem
or not.
* Password management - we need to secure passwords from casual observation.
Part of our process involves decryption password for use with external systems.
When all of the REPL output is captured and displayed an embedded line of code
faithfully shows our passwords in cleartext - this is highly undesirable. Sure,
we can hide the output, but if for some reason we need to expose it (e.g. to
view the runtime error) then the password is their for everyone to see - or we
need more paragraphs (e.g. modularise the password out into its own paragraph)
which occupy more screen real estates (there are other reasons why this is not
a good idea for the password management use case)
Enclosing the code in a block:
* This is a nice workaround, but it requires refactoring all of our code
* Variables defined in the code block are not visible to subsequent paragraphs
as they become out of scope at the end of the code block/paragraph defining
them.
Thus this has some usage, but has limited use cases (I have several notebooks
that already work this way)
Concluding:
* The supporting use case is, IMHO, an extreme edge case. The supporting use
case was something like "Print(1)\n println(2)" results in only 2 being
displayed when the desired output is 12.
However, consider the following more realistic use case:
{code:java}
println("result:")
val x = 1
val y = 2
println(s"x + y = ${x + y}")
{code}
Which will result in:
{code:java}
result:
x: Int = 1
y: Int = 2
x + y = 3
{code}
So you still do not get the "nice output" proposed in the documentation
supporting the use case.
So, you might argue split the paragraphs and just have:
{code:java}
val x = 1
val y = 2
{code}
followed by:
{code:java}
println("result:")
println(s"x + y = ${x + y}")
{code}
To which I would reply just merge the two print statements into one using the
plus operator e.g. println("result:\n" + s"x + y = ${x + y}") or more simply
println(s"result\nx + y = ${x + y}")
No matter how you re-arrange the initial example into a single paragraph, you
will still get messy output:
e.g.
{code:java}
val x = 1
val y = 2
print("result")
println(s"x + y = ${x + y}")
{code}
Produces:
{code:java}
x: Int = 1
y: Int = 2
result:
x + y = 3
{code}
which IMHO, is not much better
was (Author: gm310509):
Hi Saiso,
I understand what the change is and how it faithfully reflects the output given
in Scala REPL.
However, IMHO, I do not see Zeppelin as a Scala REPL. Zeppelin is more of a
notebook that has a similar look and feel to a WIki - the big feature is the
ability to have live code that produces nice output for perusal by end users
and in our case, and I'm sure many others presentation to non-technical users
such as management.
Faithfully capturing the entire REPL output is clever, but it messes up the
notebook.
IMHO, if you want to see REPL output, use a REPL such as SBT, Spark-Shell,
IntelliJ etc. There are advantages to using a real REPL as you get context
sensitive "code completion" in the REPL.
The proposed workarounds such as:
* splitting a paragraph into two being a) preparatory piece of code - in which
code and output are hidden and b) a single println statement (or as per the
justification in the original Jira) two print statements in a single paragraph
* Enclosing the entire code segment in to a block (i.e. putting braces around
it)
* Others
Create their own problems IMHO
For example:
Splitting the paragraph:
* If you wish to refresh your output, you have to remember to run two
paragraphs - the preparatory code and the display code.
* The extra paragraphs containing preparatory code consume some screen real
estate. If, like me, you want to modularise and have paragraphs for specific
purposes, you will end up with several paragraphs consuming screen real estate
just for the setup code.
* Whether you modularise or not, it is possible that you will end up with
large paragraphs (e.g. 40-50 lines of scala code). If you have a runtime error
it can often be difficult to see because:
** Previously, when only the last output was displayed, the exception will
appear *at the top* of the output area of the paragraph.
** Now, when all REPL output is displayed it can be difficult to tell if your
code has a runtime error as it will often be off the bottom of the output area
(or off the bottom of the browser). And if the little "Status indicator" next
to the run paragraph button is not visible (i.e. you run the paragraph via the
keyboard shortcuts) it can be difficult to just see whether there is a problem
or not.
* Password management - we need to secure passwords from casual observation.
Part of our process involves decryption password for use with external systems.
When all of the REPL output is captured and displayed an embedded line of code
faithfully shows our passwords in cleartext - this is highly undesirable. Sure,
we can hide the output, but if for some reason we need to expose it (e.g. to
view the runtime error) then the password is their for everyone to see - or we
need more paragraphs (e.g. modularise the password out into its own paragraph)
which occupy more screen real estates (there are other reasons why this is not
a good idea for the password management use case)
Enclosing the code in a block:
* This is a nice workaround, but it requires refactoring all of our code
* Variables defined in the code block are not visible to subsequent paragraphs
as they become out of scope at the end of the code block/paragraph defining
them.
Thus this has some usage, but has limited use cases (I have several notebooks
that already work this way)
Concluding:
* The supporting use case is, IMHO, an extreme edge case. The supporting use
case was something like "Print(1)\n println(2)" results in only 2 being
displayed when the desired output is 12.
However, consider the following more realistic use case:
*
{code:java}
println("1")
{code}
*
*
> Livy 0.5 incubating. Dumps output of command run in the browser.
> ----------------------------------------------------------------
>
> Key: LIVY-507
> URL: https://issues.apache.org/jira/browse/LIVY-507
> Project: Livy
> Issue Type: Bug
> Components: Interpreter
> Affects Versions: 0.5.0
> Reporter: Bicky
> Priority: Blocker
> Fix For: 0.5.0
>
> Attachments: Screen Shot 2018-09-10 at 4.57.18 pm.png
>
>
> Team,
> We upgraded to livy 0.5 incubating post which we are seeing that livy
> interpreter on zeppelin is displaying the code of the paragraph in the
> browser itself .
>
> !Screen Shot 2018-09-10 at 4.57.18 pm.png!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
