[
https://issues.apache.org/jira/browse/SPARK-13973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15249656#comment-15249656
]
Paul Shearer edited comment on SPARK-13973 at 4/20/16 11:36 AM:
----------------------------------------------------------------
In my development experience and those I know, the notebook is not the only or
even the main use case for IPython. It's just the most visible because it's
easy to make slides and such. IPython, without the notebook, is a productive
general-purpose programming environment for data scientists.
Just to get us on the same page - pyspark is a python shell with SparkContext
pre-loaded. The python shell can be either default python shell or the enhanced
IPython shell. The latter offers tab completion of object attributes, nicely
formatted tracebacks, "magic" macros for executing pasted code snippets,
scripting, debugging, and accessing help/docstrings. It is a separate entity
from the notebook and long predates the notebook.
Most working scientists and analysts I know, including myself, use the IPython
shell much more than the notebook. The notebook is more of a presentation and
exploratory analysis tool, while the IPython shell is better for general
programming / power users.
And yes, not overriding `PYSPARK_DRIVER_PYTHON` would avoid breaking my setup.
I found it easy enough just to comment out `IPYTHON=1` once I realized that was
the problem, so alternatively you could print some alert that this is
deprecated and tell people how to change it, so you don't have to keep
supporting it.
was (Author: pshearer):
Just to get us on the same page - pyspark is a python shell with SparkContext
pre-loaded. The python shell can be either default python shell or the enhanced
IPython shell. The latter offers tab completion of object attributes, nicely
formatted tracebacks, "magic" macros for executing pasted code snippets,
scripting, debugging, and accessing help/docstrings. It is a separate entity
from the notebook and long predates the notebook.
Most working scientists and analysts I know, including myself, use the IPython
shell much more than the notebook. The notebook is more of a presentation and
exploratory analysis tool, while the IPython shell is better for general
programming / power users.
And yes, not overriding `PYSPARK_DRIVER_PYTHON` would avoid breaking my setup.
I found it easy enough just to comment out `IPYTHON=1` once I realized that was
the problem, so alternatively you could print some alert that this is
deprecated and tell people how to change it, so you don't have to keep
supporting it.
> `ipython notebook` is going away...
> -----------------------------------
>
> Key: SPARK-13973
> URL: https://issues.apache.org/jira/browse/SPARK-13973
> Project: Spark
> Issue Type: Improvement
> Components: PySpark
> Environment: spark-1.6.1-bin-hadoop2.6
> Anaconda2-2.5.0-Linux-x86_64
> Reporter: Bogdan Pirvu
> Assignee: Rekha Joshi
> Priority: Trivial
> Fix For: 2.0.0
>
>
> Starting {{pyspark}} with following environment variables:
> {code:none}
> export IPYTHON=1
> export IPYTHON_OPTS="notebook --no-browser"
> {code}
> yields this warning
> {code:none}
> [TerminalIPythonApp] WARNING | Subcommand `ipython notebook` is deprecated
> and will be removed in future versions.
> [TerminalIPythonApp] WARNING | You likely want to use `jupyter notebook`...
> continue in 5 sec. Press Ctrl-C to quit now.
> {code}
> Changing line 52 from
> {code:none}
> PYSPARK_DRIVER_PYTHON="ipython"
> {code}
> to
> {code:none}
> PYSPARK_DRIVER_PYTHON="jupyter"
> {code}
> in https://github.com/apache/spark/blob/master/bin/pyspark works for me to
> solve this issue, but I'm not sure if it's sustainable as I'm not familiar
> with the rest of the code...
> This is the relevant part of my Python environment:
> {code:none}
> ipython 4.1.2 py27_0
> ipython-genutils 0.1.0 <pip>
> ipython_genutils 0.1.0 py27_0
> ipywidgets 4.1.1 py27_0
> ...
> jupyter 1.0.0 py27_1
> jupyter-client 4.2.1 <pip>
> jupyter-console 4.1.1 <pip>
> jupyter-core 4.1.0 <pip>
> jupyter_client 4.2.1 py27_0
> jupyter_console 4.1.1 py27_0
> jupyter_core 4.1.0 py27_0
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
