[jira] [Commented] (TOREE-427) Unable to use display() or magics in Toree-PySpark
[
https://issues.apache.org/jira/browse/TOREE-427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16306065#comment-16306065
]
Huadong Liu commented on TOREE-427:
---
Yes, magic is available in the scala kernel, but not the pyspark kernel. Looks
like it used to work
https://www.mail-archive.com/dev@toree.incubator.apache.org/msg00823.html.
> Unable to use display() or magics in Toree-PySpark
> --
>
> Key: TOREE-427
> URL: https://issues.apache.org/jira/browse/TOREE-427
> Project: TOREE
> Issue Type: Bug
> Components: Kernel
>Affects Versions: 0.2.0
> Environment: Linux
>Reporter: Mark Gritter
>Priority: Minor
>
> I am trying to use PySpark in a Jupyter notebook using PySpark. Plain-text
> interaction is working fine. When I tried to include HTML or graphical
> output, though I get the following:
> display(sc.getConf().getAll())
> Name: org.apache.toree.interpreter.broker.BrokerException
> Message: Traceback (most recent call last):
> File
> "/tmp/kernel-PySpark-7b121ed7-79a8-4222-bf15-733286d125d2/pyspark_runner.py",
> line 189, in
> eval(compiled_code)
> File "", line 1, in
> File "/usr/local/lib/python2.7/dist-packages/IPython/core/display.py", line
> 300, in display
> format = InteractiveShell.instance().display_formatter.format
> File
> "/usr/local/lib/python2.7/dist-packages/traitlets/config/configurable.py",
> line 412, in instance
> inst = cls(*args, **kwargs)
> File
> "/usr/local/lib/python2.7/dist-packages/IPython/core/interactiveshell.py",
> line 500, in __init__
> self.init_io()
> File
> "/usr/local/lib/python2.7/dist-packages/IPython/core/interactiveshell.py",
> line 660, in init_io
> io.stdout = io.IOStream(sys.stdout)
> File "/usr/local/lib/python2.7/dist-packages/IPython/utils/io.py", line 34,
> in __init__
> raise ValueError("fallback required, but not specified")
> ValueError: fallback required, but not specified
> StackTrace:
> org.apache.toree.interpreter.broker.BrokerState$$anonfun$markFailure$1.apply(BrokerState.scala:163)
> org.apache.toree.interpreter.broker.BrokerState$$anonfun$markFailure$1.apply(BrokerState.scala:163)
> scala.Option.foreach(Option.scala:257)
> org.apache.toree.interpreter.broker.BrokerState.markFailure(BrokerState.scala:162)
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> java.lang.reflect.Method.invoke(Method.java:498)
> py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
> py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
> py4j.Gateway.invoke(Gateway.java:280)
> py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
> py4j.commands.CallCommand.execute(CallCommand.java:79)
> py4j.GatewayConnection.run(GatewayConnection.java:214)
> java.lang.Thread.run(Thread.java:748)
> I tried using magics to display raw HTML (following the example here:
> https://github.com/apache/incubator-toree/blob/master/etc/examples/notebooks/magic-tutorial.ipynb)
> and get the following backtrace:
> %%dataframe
> Name: org.apache.toree.interpreter.broker.BrokerException
> Message: Traceback (most recent call last):
> File
> "/tmp/kernel-PySpark-7b121ed7-79a8-4222-bf15-733286d125d2/pyspark_runner.py",
> line 189, in
> eval(compiled_code)
> File "", line 1, in
> File
> "/tmp/kernel-PySpark-7b121ed7-79a8-4222-bf15-733286d125d2/pyspark_runner.py",
> line 107, in __getattr__
> return self._jvm_kernel.__getattribute__(name)
> AttributeError: 'JavaObject' object has no attribute 'magics'
> StackTrace:
> org.apache.toree.interpreter.broker.BrokerState$$anonfun$markFailure$1.apply(BrokerState.scala:163)
> org.apache.toree.interpreter.broker.BrokerState$$anonfun$markFailure$1.apply(BrokerState.scala:163)
> scala.Option.foreach(Option.scala:257)
> org.apache.toree.interpreter.broker.BrokerState.markFailure(BrokerState.scala:162)
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> java.lang.reflect.Method.invoke(Method.java:498)
> py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
> py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
> py4j.Gateway.invoke(Gateway.java:280)
> py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
> py4j.commands.CallCommand.execute(CallCommand.java:79)
> py4j.GatewayConnection.run(GatewayConnection.java:214)
> java.lang.Thread.run(Thread.java:748)
> VERSION: 0.2.0.dev1-incubating
> COMMIT: 9b577f19df83
--
This message was sent
[jira] [Comment Edited] (TOREE-427) Unable to use display() or magics in Toree-PySpark
[
https://issues.apache.org/jira/browse/TOREE-427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16306065#comment-16306065
]
Huadong Liu edited comment on TOREE-427 at 12/29/17 8:09 AM:
-
Yes, magic is available in the scala kernel, but not the pyspark kernel. Looks
like it used to work
https://www.mail-archive.com/dev@toree.incubator.apache.org/msg00823.html.
[~luciano resende] can we bump up priority?
was (Author: huadongliu):
Yes, magic is available in the scala kernel, but not the pyspark kernel. Looks
like it used to work
https://www.mail-archive.com/dev@toree.incubator.apache.org/msg00823.html.
> Unable to use display() or magics in Toree-PySpark
> --
>
> Key: TOREE-427
> URL: https://issues.apache.org/jira/browse/TOREE-427
> Project: TOREE
> Issue Type: Bug
> Components: Kernel
>Affects Versions: 0.2.0
> Environment: Linux
>Reporter: Mark Gritter
>Priority: Minor
>
> I am trying to use PySpark in a Jupyter notebook using PySpark. Plain-text
> interaction is working fine. When I tried to include HTML or graphical
> output, though I get the following:
> display(sc.getConf().getAll())
> Name: org.apache.toree.interpreter.broker.BrokerException
> Message: Traceback (most recent call last):
> File
> "/tmp/kernel-PySpark-7b121ed7-79a8-4222-bf15-733286d125d2/pyspark_runner.py",
> line 189, in
> eval(compiled_code)
> File "", line 1, in
> File "/usr/local/lib/python2.7/dist-packages/IPython/core/display.py", line
> 300, in display
> format = InteractiveShell.instance().display_formatter.format
> File
> "/usr/local/lib/python2.7/dist-packages/traitlets/config/configurable.py",
> line 412, in instance
> inst = cls(*args, **kwargs)
> File
> "/usr/local/lib/python2.7/dist-packages/IPython/core/interactiveshell.py",
> line 500, in __init__
> self.init_io()
> File
> "/usr/local/lib/python2.7/dist-packages/IPython/core/interactiveshell.py",
> line 660, in init_io
> io.stdout = io.IOStream(sys.stdout)
> File "/usr/local/lib/python2.7/dist-packages/IPython/utils/io.py", line 34,
> in __init__
> raise ValueError("fallback required, but not specified")
> ValueError: fallback required, but not specified
> StackTrace:
> org.apache.toree.interpreter.broker.BrokerState$$anonfun$markFailure$1.apply(BrokerState.scala:163)
> org.apache.toree.interpreter.broker.BrokerState$$anonfun$markFailure$1.apply(BrokerState.scala:163)
> scala.Option.foreach(Option.scala:257)
> org.apache.toree.interpreter.broker.BrokerState.markFailure(BrokerState.scala:162)
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> java.lang.reflect.Method.invoke(Method.java:498)
> py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
> py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
> py4j.Gateway.invoke(Gateway.java:280)
> py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
> py4j.commands.CallCommand.execute(CallCommand.java:79)
> py4j.GatewayConnection.run(GatewayConnection.java:214)
> java.lang.Thread.run(Thread.java:748)
> I tried using magics to display raw HTML (following the example here:
> https://github.com/apache/incubator-toree/blob/master/etc/examples/notebooks/magic-tutorial.ipynb)
> and get the following backtrace:
> %%dataframe
> Name: org.apache.toree.interpreter.broker.BrokerException
> Message: Traceback (most recent call last):
> File
> "/tmp/kernel-PySpark-7b121ed7-79a8-4222-bf15-733286d125d2/pyspark_runner.py",
> line 189, in
> eval(compiled_code)
> File "", line 1, in
> File
> "/tmp/kernel-PySpark-7b121ed7-79a8-4222-bf15-733286d125d2/pyspark_runner.py",
> line 107, in __getattr__
> return self._jvm_kernel.__getattribute__(name)
> AttributeError: 'JavaObject' object has no attribute 'magics'
> StackTrace:
> org.apache.toree.interpreter.broker.BrokerState$$anonfun$markFailure$1.apply(BrokerState.scala:163)
> org.apache.toree.interpreter.broker.BrokerState$$anonfun$markFailure$1.apply(BrokerState.scala:163)
> scala.Option.foreach(Option.scala:257)
> org.apache.toree.interpreter.broker.BrokerState.markFailure(BrokerState.scala:162)
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> java.lang.reflect.Method.invoke(Method.java:498)
> py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
> py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
> py4j.Gateway.invoke(Gateway.java:280)
> py4j.c
[jira] [Commented] (TOREE-413) Add support for session timeout and exit
[ https://issues.apache.org/jira/browse/TOREE-413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16432958#comment-16432958 ] Huadong Liu commented on TOREE-413: --- Hi [~rdblue], was there any reason why this pull request hasn't been merged? > Add support for session timeout and exit > > > Key: TOREE-413 > URL: https://issues.apache.org/jira/browse/TOREE-413 > Project: TOREE > Issue Type: Improvement >Reporter: Ryan Blue >Priority: Major > > When using Toree with Jupyter console, the only way to exit is to use CTRL+D > to send EOF and then confirm. Toree should include support for "exit", > "quit", and ":q" in any interpreter. > In addition, because Toree runs in a separate process and can start up > slowly, the default Jupyter console timeout (60s) will sometimes cause the > console to exit but leave a Toree kernel running. Similarly, losing a SSH > session running Jupyter console with Toree leaves the kernel running. Toree > should have a 24h idle timeout to clean up these zombie kernels. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
