[jira] [Comment Edited] (TOREE-375) Incorrect fully qualified name for spark context

2017-02-21 Thread Jakob Odersky (JIRA) 

[ 
https://issues.apache.org/jira/browse/TOREE-375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15876892#comment-15876892
 ] 

Jakob Odersky edited comment on TOREE-375 at 2/21/17 10:37 PM:
---

[~fschueler] so I'm actually not entirely sure what's the cause of this bug, 
however I can confirm it is not specific to Toree.

It can be reproduces in a standard spark shell as well:

{code}scala> :power
Power mode enabled. :phase is at typer.
import scala.tools.nsc._, intp.global._, definitions._
Try :help or completions for vals._ and power._

scala> power.intp.addUrlsToClassPath(jar)

scala> import org.apache.sysml.api.MLContext
import org.apache.sysml.api.MLContext

scala> val ml = new MLContext(sc)
:50: error: overloaded method constructor MLContext with alternatives:
  (x$1: 
org.apache.spark.api.java.JavaSparkContext)org.apache.sysml.api.MLContext 
  (x$1: 
org.apache.spark.org.apache.spark.org.apache.spark.org.apache.spark.org.apache.spark.SparkContext)org.apache.sysml.api.MLContext
 cannot be applied to 
(org.apache.spark.org.apache.spark.org.apache.spark.org.apache.spark.org.apache.spark.SparkContext)
   val ml = new MLContext(sc)
{code}


was (Author: jodersky):
[~fschueler] so I'm actually not entirely sure what's the cause of this bug, 
however I can confirm it is not specific to Toree.

It can be reproduces in a standard spark shell as well:

```
scala> :power
Power mode enabled. :phase is at typer.
import scala.tools.nsc._, intp.global._, definitions._
Try :help or completions for vals._ and power._

scala> power.intp.addUrlsToClassPath(jar)

scala> import org.apache.sysml.api.MLContext
import org.apache.sysml.api.MLContext

scala> val ml = new MLContext(sc)
:50: error: overloaded method constructor MLContext with alternatives:
  (x$1: 
org.apache.spark.api.java.JavaSparkContext)org.apache.sysml.api.MLContext 
  (x$1: 
org.apache.spark.org.apache.spark.org.apache.spark.org.apache.spark.org.apache.spark.SparkContext)org.apache.sysml.api.MLContext
 cannot be applied to 
(org.apache.spark.org.apache.spark.org.apache.spark.org.apache.spark.org.apache.spark.SparkContext)
   val ml = new MLContext(sc)
```

> Incorrect fully qualified name for spark context
> 
>
> Key: TOREE-375
> URL: https://issues.apache.org/jira/browse/TOREE-375
> Project: TOREE
>  Issue Type: Bug
> Environment: Jupyter Notebook with Toree latest master 
> (1a9c11f5f1381c15b691a716acd0e1f0432a9a35) and Spark 2.0.2, Scala 2.11
>Reporter: Felix Schüler
>Priority: Critical
>
> When running below snippet in a cell I get a compile error for the MLContext 
> Constructor. Somehow the fully qualified name of the SparkContext gets messed 
> up. 
> The same does not happen when I start a Spark shell with the --jars command 
> and create the MLContext there.
> Snippet (the systemml jar is build with the latest master of SystemML):
> {code}
> %addjar 
> file:///home/felix/repos/incubator-systemml/target/systemml-0.13.0-incubating-SNAPSHOT.jar
>  -f
> import org.apache.sysml.api.mlcontext._
> import org.apache.sysml.api.mlcontext.ScriptFactory._
> val ml = new MLContext(sc)
> Starting download from 
> file:///home/felix/repos/incubator-systemml/target/systemml-0.13.0-incubating-SNAPSHOT.jar
> Finished download of systemml-0.13.0-incubating-SNAPSHOT.jar
> Name: Compile Error
> Message: :25: error: overloaded method constructor MLContext with 
> alternatives:
>   (x$1: 
> org.apache.spark.api.java.JavaSparkContext)org.apache.sysml.api.mlcontext.MLContext
>  
>   (x$1: 
> org.apache.spark.org.apache.spark.org.apache.spark.org.apache.spark.org.apache.spark.SparkContext)org.apache.sysml.api.mlcontext.MLContext
>  cannot be applied to 
> (org.apache.spark.org.apache.spark.org.apache.spark.org.apache.spark.org.apache.spark.SparkContext)
>val ml = new MLContext(sc)
> ^
> StackTrace: 
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: [RESULT] [VOTE] Apache Toree 0.1.0 RC6

2017-02-21 Thread Chip Senkbeil 
Vote thread on the incubator @general channel can be found here:

https://lists.apache.org/thread.html/ecb5e40078f01c488ada52e776d57cd80e130164c1e31d8ded564ba2@


On Tue, Feb 21, 2017 at 2:05 PM Chip Senkbeil 
wrote:

> Voting is now closed. This vote has PASSED with 4 +1 votes and no 0 or -1
> votes. This vote will now go to the IPMC list for a second 72-hour minimum
> vote. Toree developers are encouraged to comment on the IPMC vote as well.
>
> +1:
> Chip Senkbeil
> Marius van Niekerk
> Gino Bustelo
> Corey Stubbs
>
> 0:
>
> -1:
>

[RESULT] [VOTE] Apache Toree 0.1.0 RC6

2017-02-21 Thread Chip Senkbeil 
Voting is now closed. This vote has PASSED with 4 +1 votes and no 0 or -1
votes. This vote will now go to the IPMC list for a second 72-hour minimum
vote. Toree developers are encouraged to comment on the IPMC vote as well.

+1:
Chip Senkbeil
Marius van Niekerk
Gino Bustelo
Corey Stubbs

0:

-1:

Re: [VOTE] Apache Toree 0.1.0 RC6

2017-02-21 Thread Chip Senkbeil 
It's been over 72 hours (not including weekend), so going to close this and
start the vote in incubutor general.

On Mon, Feb 20, 2017 at 9:33 AM Gino Bustelo  wrote:

> +1
>
> I installed it locally on my machine and it works
>
> On Mon, Feb 20, 2017 at 9:32 AM Corey Stubbs  wrote:
>
> > +1 Ran and installed the pip dist with Apache Spark 1.6.3.
> >
> > On Thu, Feb 16, 2017 at 11:24 AM Marius van Niekerk <
> > marius.v.niek...@gmail.com> wrote:
> >
> > > +1
> > >
> > > Going to +1 this since I assisted with getting rid of some of the
> bundled
> > > pieces that were blocking for RC5.
> > >
> > > On Thu, Feb 16, 2017, 12:14 Chip Senkbeil 
> > wrote:
> > >
> > > I'll go ahead and give a +1 since I've done a bit of testing using
> > > Scala/SparkR to get this back up to speed.
> > >
> > > Also resolved the remaining issues we had from the incubator general
> > vote.
> > > We're still bundling additional license information in the binary
> > release,
> > > but the source release no longer has the extra license info tacked onto
> > the
> > > LICENSE file. I've also fixed the source release such that it is no
> > longer
> > > dependent on being in a Git repository to be built nor does it require
> > you
> > > to have sbt installed.
> > >
> > > On Thu, Feb 16, 2017 at 11:09 AM Chip Senkbeil <
> chip.senkb...@gmail.com>
> > > wrote:
> > >
> > > > Please vote to approve the release of the following candidate as
> Apache
> > > > Toree version 0.1.0. Pay special attention to the LICENSE and NOTICE
> > > files
> > > > since this is our first release.
> > > >
> > > > ## Information and Artifacts
> > > >
> > > > The tag to be voted on is v0.1.0-rc6
> > > > (51fa49cb5898e0c5b7824f986382436b969cabc7), located here:
> > > >
> > > >
> > > >
> > >
> > >
> >
> https://github.com/apache/incubator-toree/commit/51fa49cb5898e0c5b7824f986382436b969cabc7
> > > >
> > > > All distribution packages, including signatures, digests, etc. can be
> > > found
> > > > at:
> > > >
> > > > https://dist.apache.org/repos/dist/dev/incubator/toree/0.1.0/rc6/
> > > >
> > > > Staging artifacts can be found at:
> > > >
> > > >
> https://repository.apache.org/content/repositories/orgapachetoree-1006
> > > >
> > > > ## Testing Instructions
> > > >
> > > > The fastest way to get up and running is to using Jupyter.
> > > >
> > > > 1. Install Jupyter if you haven't already (
> > > http://jupyter.org/install.html
> > > > )
> > > >
> > > > 2. Grab the Apache Toree archive from
> > > >
> > > >
> > >
> > >
> >
> https://dist.apache.org/repos/dist/dev/incubator/toree/0.1.0/rc6/toree-pip/apache-toree-0.1.0.tar.gz
> > > >
> > > > 3. Install Apache Toree via `pip install apache-toree-0.1.0.tar.gz`
> > > > followed by `jupyter toree install`
> > > >
> > > > - You need to set a valid Apache Spark 1.6.x home, which can be done
> > via
> > > > `jupyter
> > > > toree install --spark_home=/usr/local/spark`
> > > >
> > > > - You may need to run with `sudo` for installation permission
> > > >
> > > > - For all installation options, run `jupyter toree install
> --help-all`
> > > >
> > > > 4. Run a Jupyter notebook server via `jupyter notebook`
> > > >
> > > > - If the notebook portion of Jupyter is not installed but Jupyter is,
> > > > you can install via `pip install notebook`
> > > >
> > > > 5. Create a notebook using "Apache Toree - Scala" from the dropdown
> > under
> > > > new
> > > >
> > > > - If you want other interpreter profiles than Scala, you can change
> the
> > > > interpreters via `jupyter toree install --interpreters=PySpark,SQL`
> > > >
> > > > 6. Run Scala/Spark commands such as `sc.parallelize(1 to 100).sum()`
> in
> > > the
> > > > notebook
> > > >
> > > > ## Voting Instructions
> > > >
> > > > The vote is open for at least 72 hours and passes if a majority of at
> > > least
> > > > 3 +1 PMC votes are cast.
> > > >
> > > > [ ] +1 Release this package as Apache Toree 0.1.0
> > > > [ ] -1 Do not release this package because ...
> > > >
> > >
> > > --
> > > regards
> > > Marius van Niekerk
> > >
> >
>