Re: Spark 2.0.0-preview artifacts still not available in Maven

On Wed, Jun 1, 2016 at 5:58 PM, Reynold Xin wrote: > The preview release is available here: > http://spark.apache.org/downloads.html (there is an entire section dedicated > to it and also there is a news link to it on the right). Oops, it is indeed down there at the bottom,

OOM while unrolling in Spark 1.5.2

Hello, I came across a weird non-easily replicable OOM in executor during unrolling. The standalone cluster uses default memory settings on Spark 1.5.2. What strikes me is that the OOM happens when Spark tries to allocate a bytebuffer for the FileChannel when dropping blocks from memory and

Re: Spark 2.0.0-preview artifacts still not available in Maven

Hi Sean, (writing this email with my Apache hat on only and not Databricks hat) The preview release is available here: http://spark.apache.org/downloads.html (there is an entire section dedicated to it and also there is a news link to it on the right). Again, I think this is a good opportunity

Re: Spark 2.0.0-preview artifacts still not available in Maven

I'll be more specific about the issue that I think trumps all this, which I realize maybe not everyone was aware of. There was a long and contentious discussion on the PMC about, among other things, advertising a "Spark 2.0 preview" from Databricks, such as at

Re: Spark 2.0.0-preview artifacts still not available in Maven

> > I'd think we want less effort, not more, to let people test it? for > example, right now I can't easily try my product build against > 2.0.0-preview. I don't feel super strongly one way or the other, so if we need to publish it permanently we can. However, either way you can still test

Re: Spark 2.0.0-preview artifacts still not available in Maven

On Wed, Jun 1, 2016 at 2:51 PM, Sean Owen wrote: > I'd think we want less effort, not more, to let people test it? for > example, right now I can't easily try my product build against > 2.0.0-preview. While I understand your point of view, I like the extra effort to get to

Re: Spark 2.0.0-preview artifacts still not available in Maven

An RC is something that gets voted on, and the final one is turned into a blessed release. I agree that RCs don't get published to Maven Central, but releases do of course. This was certainly to be an official release, right? A beta or alpha can still be an official, published release. The

Re: Spark 2.0.0-preview artifacts still not available in Maven

I think what Reynold probably means is that previews are releases for which a vote *passed*. ~ Jonathan On Wed, Jun 1, 2016 at 1:53 PM Marcelo Vanzin wrote: > So are RCs, aren't they? > > Personally I'm fine with not releasing to maven central. Any extra > effort needed by

Re: Spark 2.0.0-preview artifacts still not available in Maven

So are RCs, aren't they? Personally I'm fine with not releasing to maven central. Any extra effort needed by regular users to use a preview / RC is good with me. On Wed, Jun 1, 2016 at 1:50 PM, Reynold Xin wrote: > To play devil's advocate, previews are technically not RCs.

Re: Spark 2.0.0-preview artifacts still not available in Maven

To play devil's advocate, previews are technically not RCs. They are actually voted releases. On Wed, Jun 1, 2016 at 1:46 PM, Michael Armbrust wrote: > Yeah, we don't usually publish RCs to central, right? > > On Wed, Jun 1, 2016 at 1:06 PM, Reynold Xin

Re: Spark 2.0.0-preview artifacts still not available in Maven

Yeah, we don't usually publish RCs to central, right? On Wed, Jun 1, 2016 at 1:06 PM, Reynold Xin wrote: > They are here ain't they? > > https://repository.apache.org/content/repositories/orgapachespark-1182/ > > Did you mean publishing them to maven central? My

Re: Spark 2.0.0-preview artifacts still not available in Maven

They are here ain't they? https://repository.apache.org/content/repositories/orgapachespark-1182/ Did you mean publishing them to maven central? My understanding is that publishing to maven central isn't a required step of doing theses. This might be a good opportunity to discuss that. My

Spark 2.0.0-preview artifacts still not available in Maven

Just checked and they are still not published this week. Can these be published ASAP to complete the 2.0.0-preview release? - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail:

Re: ImportError: No module named numpy

Try adding to spark-env.sh (renaming if you still have it with .template at the end): PYSPARK_PYTHON=/path/to/your/bin/python Where your bin/python is your actual Python environment with Numpy installed. > El 1 jun 2016, a las 20:16, Bhupendra Mishra > escribió:

Re: ImportError: No module named numpy

I have numpy installed but where I should setup PYTHONPATH? On Wed, Jun 1, 2016 at 11:39 PM, Sergio Fernández wrote: > sudo pip install numpy > > On Wed, Jun 1, 2016 at 5:56 PM, Bhupendra Mishra < > bhupendra.mis...@gmail.com> wrote: > >> Thanks . >> How can this be

Re: ImportError: No module named numpy

sudo pip install numpy On Wed, Jun 1, 2016 at 5:56 PM, Bhupendra Mishra wrote: > Thanks . > How can this be resolved? > > On Wed, Jun 1, 2016 at 9:02 PM, Holden Karau wrote: > >> Generally this means numpy isn't installed on the system or your

Re: [DISCUSS] Removing or changing maintainer process

I just heard about mention-bot at PyCon 2016 : https://github.com/facebook/mention-bot Do you have a GitHub project that is too big for people to subscribe to all the notifications? The mention bot will automatically mention potential reviewers on

Re: ImportError: No module named numpy

Thanks . How can this be resolved? On Wed, Jun 1, 2016 at 9:02 PM, Holden Karau wrote: > Generally this means numpy isn't installed on the system or your > PYTHONPATH has somehow gotten pointed somewhere odd, > > On Wed, Jun 1, 2016 at 8:31 AM, Bhupendra Mishra < >

Re: ImportError: No module named numpy

Generally this means numpy isn't installed on the system or your PYTHONPATH has somehow gotten pointed somewhere odd, On Wed, Jun 1, 2016 at 8:31 AM, Bhupendra Mishra wrote: > If any one please can help me with following error. > > File >

ImportError: No module named numpy

If any one please can help me with following error. File "/opt/mapr/spark/spark-1.6.1/python/lib/pyspark.zip/pyspark/mllib/__init__.py", line 25, in ImportError: No module named numpy Thanks in advance!

Re: Windows Rstudio to Linux spakR

Selvam, First, deploy the Spark distribution on your Windows machine, which is of the same version of Spark in your Linux cluster Second, follow the instructions at https://github.com/apache/spark/tree/master/R#using-sparkr-from-rstudio. Specify the Spark master URL for your Linux Spark