[jira] [Updated] (TOREE-402) Installer should support parameterized kernel names

2017-04-03 Thread Christian Kadner (JIRA) 

 [ 
https://issues.apache.org/jira/browse/TOREE-402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christian Kadner updated TOREE-402:
---
Description: 
For enterprise deployments of Apache Toree it would be nice to have more 
flexibility when specifying kernel name(s) when installing multiple 
interpreters at the same time.

Currently the Apache Toree installer allows to specify {{kernel_name}} and 
{{interpreters}}, i.e. running {code}jupyter toree install --kernel_name='Spark 
2.1' --interpresters=Scala,PySpark,SparkR,SQL{code} would result in kernels 
with these names:
{code}
Spark 2.1 - Scala
Spark 2.1 - PySpark
Spark 2.1 - SparkR
Spark 2.1 - SQL
{code}

For enterprise deployments that support other languages and Spark versions 
however this naming scheme is not flexible enough. Suppose this is the desired 
list of kernels (kernel display names):

{code}
Python 2.7 with Spark 1.6
Python 2.7 with Spark 2.0
Python 3.5 with Spark 1.6
Python 3.5 with Spark 2.0
R with Spark 1.6
R with Spark 2.0
Scala 2.10 with Spark 1.6
Scala 2.11 with Spark 2.0
{code}

In order to achieve the above names, one would have to write a custom script to 
replace the {{display_name}} in the {{kernel.json}} files that get created by 
the Toree installer.

It would be nice to enrich the Toree install options to allow for some kind of 
pattern instead of a fixed string, i.e.:

{code}
jupyter toree install --kernel_name='{interpreter.name} {interpreter.version} 
with Spark {spark.version}' ...
{code}

The install documentation might read:
{noformat}
--kernel_name= (ToreeInstall.kernel_name)
Examples:  '{interpreter.name} {interpreter.version} with Spark 
{spark.version}'
Default:   'Apache Toree - {interpreter.name}'
Install the kernel spec with this name. This is also used as the display 
name for kernels in the Jupyter UI.
{noformat}

Of course the placeholders would then have to be replaced by the Toree install 
code and actual list of available variables may be different from the above 
suggestion.

  was:
For enterprise deployments of Apache Toree it would be nice to have more 
flexibility when specifying kernel name(s) when installing multiple 
interpreters at the same time.

Currently the Apache Toree installer allows to specify {{kernel_name}} and 
{{interpreters}}, i.e. running {code}jupyter toree install --kernel_name='Spark 
2.1' --interpresters=Scala,PySpark,SparkR,SQL{code} would result in kernels 
with these names:
{code}
Spark 2.1 - Scala
Spark 2.1 - PySpark
Spark 2.1 - SparkR
Spark 2.1 - SQL
{code}

For enterprise deployments that support other languages and Spark versions 
however this naming scheme is not flexible enough. Suppose this is the desired 
list of kernels (kernel display names):

{code}
Python 2.7 with Spark 1.6
Python 2.7 with Spark 2.0
Python 3.5 with Spark 1.6
Python 3.5 with Spark 2.0
R with Spark 1.6
R with Spark 2.0
Scala 2.10 with Spark 1.6
Scala 2.11 with Spark 2.0
{code}

In order to achieve the above names, one would have to write a custom script to 
replace the {{display_name}} in the {{kernel.json}} files that get created by 
the Toree installer.

It would be nice to enrich the Toree install options to allow for some kind of 
pattern instead of a fixed string, i.e.:

{code}
jupyter install --kernel_name='{interpreter.name} {interpreter.version} with 
Spark {spark.version}' ...
{code}

The install documentation might read:
{noformat}
--kernel_name= (ToreeInstall.kernel_name)
Examples:  '{interpreter.name} {interpreter.version} with Spark 
{spark.version}'
Default:   'Apache Toree - {interpreter.name}'
Install the kernel spec with this name. This is also used as the display 
name for kernels in the Jupyter UI.
{noformat}

Of course the placeholders would then have to be replaced by the Toree install 
code and actual list of available variables may be different from the above 
suggestion.


> Installer should support parameterized kernel names
> ---
>
> Key: TOREE-402
> URL: https://issues.apache.org/jira/browse/TOREE-402
> Project: TOREE
>  Issue Type: Wish
>  Components: Kernel
>Affects Versions: 0.2.0
>Reporter: Christian Kadner
>Priority: Minor
>
> For enterprise deployments of Apache Toree it would be nice to have more 
> flexibility when specifying kernel name(s) when installing multiple 
> interpreters at the same time.
> Currently the Apache Toree installer allows to specify {{kernel_name}} and 
> {{interpreters}}, i.e. running {code}jupyter toree install 
> --kernel_name='Spark 2.1' --interpresters=Scala,PySpark,SparkR,SQL{code} 
> would result in kernels with these names:
> {code}
> Spark 2.1 - Scala
> Spark 2.1 - PySpark
> Spark 2.1 - SparkR
> Spark 2.1 - SQL
> {code}
> For enterprise deployments that support other languages and Spark versions 
> however this

[jira] [Updated] (TOREE-402) Installer should support parameterized kernel names

2017-04-03 Thread Christian Kadner (JIRA) 

 [ 
https://issues.apache.org/jira/browse/TOREE-402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christian Kadner updated TOREE-402:
---
Description: 
For enterprise deployments of Apache Toree it would be nice to have more 
flexibility when specifying kernel name(s) when installing multiple 
interpreters at the same time.

Currently the Apache Toree installer allows to specify {{kernel_name}} and 
{{interpreters}}, i.e. running {code}jupyter toree install --kernel_name='Spark 
2.1' --interpresters=Scala,PySpark,SparkR,SQL{code} would result in kernels 
with these names:
{code}
Spark 2.1 - Scala
Spark 2.1 - PySpark
Spark 2.1 - SparkR
Spark 2.1 - SQL
{code}

For enterprise deployments that support other languages and Spark versions 
however this naming scheme is not flexible enough. Suppose this is the desired 
list of kernels (kernel display names):

{code}
Python 2.7 with Spark 1.6
Python 2.7 with Spark 2.0
Python 3.5 with Spark 1.6
Python 3.5 with Spark 2.0
R with Spark 1.6
R with Spark 2.0
Scala 2.10 with Spark 1.6
Scala 2.11 with Spark 2.0
{code}

In order to achieve the above names, one would have to write a custom script to 
replace the {{display_name}} in the {{kernel.json}} files that get created by 
the Toree installer.

It would be nice to enrich the Toree install options to allow for some kind of 
pattern instead of a fixed string, i.e.:

{code}
jupyter install --kernel_name='{interpreter.name} {interpreter.version} with 
Spark {spark.version}' ...
{code}

The install documentation might read:
{noformat}
--kernel_name= (ToreeInstall.kernel_name)
Examples:  '{interpreter.name} {interpreter.version} with Spark 
{spark.version}'
Default:   'Apache Toree - {interpreter.name}'
Install the kernel spec with this name. This is also used as the display 
name for kernels in the Jupyter UI.
{noformat}

Of course the placeholders would then have to be replaced by the Toree install 
code and actual list of available variables may be different from the above 
suggestion.

  was:
For enterprise deployments of Apache Toree it would be nice to have more 
flexibility when specifying kernel name(s) when installing multiple 
interpreters at the same time.

Currently the Apache Toree installer allows to specify {{kernel_name}} and 
{{interpreters}}, i.e. running {code}toree install --kernel_name='Spark 2.1' 
--interpresters=Scala,PySpark,SparkR,SQL{code} would result in kernels with 
these names:
{code}
Spark 2.1 - Scala
Spark 2.1 - PySpark
Spark 2.1 - SparkR
Spark 2.1 - SQL
{code}

For enterprise deployments that support other languages and Spark versions 
however this naming scheme is not flexible enough. Suppose this is the desired 
list of kernels (kernel display names):

{code}
Python 2.7 with Spark 1.6
Python 2.7 with Spark 2.0
Python 3.5 with Spark 1.6
Python 3.5 with Spark 2.0
R with Spark 1.6
R with Spark 2.0
Scala 2.10 with Spark 1.6
Scala 2.11 with Spark 2.0
{code}

In order to achieve the above names, one would have to write a custom script to 
replace the {{display_name}} in the {{kernel.json}} files that get created by 
the Toree installer.

It would be nice to enrich the Toree install options to allow for some kind of 
pattern instead of a fixed string, i.e.:

{code}
jupyter install --kernel_name='{interpreter.name} {interpreter.version} with 
Spark {spark.version}' ...
{code}

The install documentation might read:
{noformat}
--kernel_name= (ToreeInstall.kernel_name)
Examples:  '{interpreter.name} {interpreter.version} with Spark 
{spark.version}'
Default:   'Apache Toree - {interpreter.name}'
Install the kernel spec with this name. This is also used as the display 
name for kernels in the Jupyter UI.
{noformat}

Of course the placeholders would then have to be replaced by the Toree install 
code and actual list of available variables may be different from the above 
suggestion.


> Installer should support parameterized kernel names
> ---
>
> Key: TOREE-402
> URL: https://issues.apache.org/jira/browse/TOREE-402
> Project: TOREE
>  Issue Type: Wish
>  Components: Kernel
>Affects Versions: 0.2.0
>Reporter: Christian Kadner
>Priority: Minor
>
> For enterprise deployments of Apache Toree it would be nice to have more 
> flexibility when specifying kernel name(s) when installing multiple 
> interpreters at the same time.
> Currently the Apache Toree installer allows to specify {{kernel_name}} and 
> {{interpreters}}, i.e. running {code}jupyter toree install 
> --kernel_name='Spark 2.1' --interpresters=Scala,PySpark,SparkR,SQL{code} 
> would result in kernels with these names:
> {code}
> Spark 2.1 - Scala
> Spark 2.1 - PySpark
> Spark 2.1 - SparkR
> Spark 2.1 - SQL
> {code}
> For enterprise deployments that support other languages and Spark versions 
> however this naming scheme is

[jira] [Created] (TOREE-402) Installer should support parameterized kernel names

2017-04-03 Thread Christian Kadner (JIRA) 
Christian Kadner created TOREE-402:
--

 Summary: Installer should support parameterized kernel names
 Key: TOREE-402
 URL: https://issues.apache.org/jira/browse/TOREE-402
 Project: TOREE
  Issue Type: Wish
  Components: Kernel
Affects Versions: 0.2.0
Reporter: Christian Kadner
Priority: Minor


For enterprise deployments of Apache Toree it would be nice to have more 
flexibility when specifying kernel name(s) when installing multiple 
interpreters at the same time.

Currently the Apache Toree installer allows to specify {{kernel_name}} and 
{{interpreters}}, i.e. running {code}toree install --kernel_name='Spark 2.1' 
--interpresters=Scala,PySpark,SparkR,SQL{code} would result in kernels with 
these names:
{code}
Spark 2.1 - Scala
Spark 2.1 - PySpark
Spark 2.1 - SparkR
Spark 2.1 - SQL
{code}

For enterprise deployments that support other languages and Spark versions 
however this naming scheme is not flexible enough. Suppose this is the desired 
list of kernels (kernel display names):

{code}
Python 2.7 with Spark 1.6
Python 2.7 with Spark 2.0
Python 3.5 with Spark 1.6
Python 3.5 with Spark 2.0
R with Spark 1.6
R with Spark 2.0
Scala 2.10 with Spark 1.6
Scala 2.11 with Spark 2.0
{code}

In order to achieve the above names, one would have to write a custom script to 
replace the {{display_name}} in the {{kernel.json}} files that get created by 
the Toree installer.

It would be nice to enrich the Toree install options to allow for some kind of 
pattern instead of a fixed string, i.e.:

{code}
jupyter install --kernel_name='{interpreter.name} {interpreter.version} with 
Spark {spark.version}' ...
{code}

The install documentation might read:
{noformat}
--kernel_name= (ToreeInstall.kernel_name)
Examples:  '{interpreter.name} {interpreter.version} with Spark 
{spark.version}'
Default:   'Apache Toree - {interpreter.name}'
Install the kernel spec with this name. This is also used as the display 
name for kernels in the Jupyter UI.
{noformat}

Of course the placeholders would then have to be replaced by the Toree install 
code and actual list of available variables may be different from the above 
suggestion.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (TOREE-401) Update website with download info for Toree 0.1.0

2017-04-03 Thread Luciano Resende (JIRA) 
Luciano Resende created TOREE-401:
-

 Summary: Update website with download info for Toree 0.1.0
 Key: TOREE-401
 URL: https://issues.apache.org/jira/browse/TOREE-401
 Project: TOREE
  Issue Type: Task
  Components: Website
Reporter: Luciano Resende
Priority: Blocker


The release vote has been approved by the IPMC and we need to update the 
website with release info and download in order to send announce e-mail. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (TOREE-400) Make website master branch up to date

2017-04-03 Thread Luciano Resende (JIRA) 

 [ 
https://issues.apache.org/jira/browse/TOREE-400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Luciano Resende updated TOREE-400:
--
Issue Type: Task  (was: Bug)

> Make website master branch up to date
> -
>
> Key: TOREE-400
> URL: https://issues.apache.org/jira/browse/TOREE-400
> Project: TOREE
>  Issue Type: Task
>Reporter: Luciano Resende
>Priority: Blocker
>
> Please see details on this dev list thread:
> https://www.mail-archive.com/dev@toree.incubator.apache.org/msg01194.html



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (TOREE-400) Make website master branch up to date

2017-04-03 Thread Luciano Resende (JIRA) 
Luciano Resende created TOREE-400:
-

 Summary: Make website master branch up to date
 Key: TOREE-400
 URL: https://issues.apache.org/jira/browse/TOREE-400
 Project: TOREE
  Issue Type: Bug
Reporter: Luciano Resende
Priority: Blocker


Please see details on this dev list thread:
https://www.mail-archive.com/dev@toree.incubator.apache.org/msg01194.html



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: SQL interpreter

2017-04-03 Thread Chip Senkbeil 
When refactoring our original interpreter codebase to support new languages
such as Python and R, I wrote the broker service to abstract the process of
communicating with another language's interpreter. When SQL was added, it
was easier to add a stubbed-out broker service such that the interpreter
followed the same structure and interfaces as the others.

The other intention was that we might fork the broker service to run in a
separate process along with the interpreter itself.

Go ahead and put your code as a WIP PR and I can take a look at it later.

On Mon, Apr 3, 2017 at 12:05 PM Ryan Blue  wrote:

> I've been working on improvements to the SQL interpreter, but I'm not quite
> sure why it works the way it currently does. Right now, the interpreter is
> associated with a SQL service that is a BrokerService. BrokerService says
> that it is a high-level interface between the JVM and another process, like
> R or Python, so it isn't clear why SQL uses it.
>
> The version I have working is considerably simpler and uses the lazy
> `kernel.sparkSession` accessor. Is there a reason why the current one
> doesn't use this approach?
>
> rb
>
> --
> Ryan Blue
> Software Engineer
> Netflix
>

SQL interpreter

2017-04-03 Thread Ryan Blue 
I've been working on improvements to the SQL interpreter, but I'm not quite
sure why it works the way it currently does. Right now, the interpreter is
associated with a SQL service that is a BrokerService. BrokerService says
that it is a high-level interface between the JVM and another process, like
R or Python, so it isn't clear why SQL uses it.

The version I have working is considerably simpler and uses the lazy
`kernel.sparkSession` accessor. Is there a reason why the current one
doesn't use this approach?

rb

-- 
Ryan Blue
Software Engineer
Netflix