Re: IMPORTANT: Spark mailing lists moving to Apache by September 1st

2013-12-22 Thread Mike Potts
Haha, no: I’m afraid there my reach exceeds my grasp :( On Dec 22, 2013, at 7:25 PM, Andy Konwinski wrote: > It would be a good thing, I'm just not sure how to achieve it, or if it's > possible. AFAIK, you cannot simply subscribe the lists to each other. Have > you heard of a setup like this

Re: IMPORTANT: Spark mailing lists moving to Apache by September 1st

2013-12-22 Thread Andy Konwinski
It would be a good thing, I'm just not sure how to achieve it, or if it's possible. AFAIK, you cannot simply subscribe the lists to each other. Have you heard of a setup like this being used before? On Dec 22, 2013 7:12 PM, "Mike Potts" wrote: > Just a thought: if the mirroring was bidirectional

Re: IMPORTANT: Spark mailing lists moving to Apache by September 1st

2013-12-22 Thread Mike Potts
Just a thought: if the mirroring was bidirectional (between apache lists and google groups) -- meaning that you could read/write via either channel and everything would be mirrored in both groups -- would that be a bad thing? The apache lists would be comprehensive and available as a first-cla

Re: Installing PySpark on a local machine

2013-12-22 Thread Josh Rosen
I've thought about creating a setup.py file for PySpark; there are a couple of subtleties involved: - PySpark's uses Py4J to create a regular Java Spark driver, so it's subject to the same limitations that Scala / Java Spark have when connecting from a local machine to a remote cluster; a

Installing PySpark on a local machine

2013-12-22 Thread Uri Laserson
Is there a documented/preferred method for installing PySpark on a local machine? I want to be able to run a Python interpreter on my local machine, point it to my Spark cluster and go. There doesn't appear to be a setup.py file anywhere, nor is pyspark registered with PyPI. I'm happy to contrib

Re: IMPORTANT: Spark mailing lists moving to Apache by September 1st

2013-12-22 Thread Andy Konwinski
Unfortunately, I don't see that as an option for either these nabble lists or the google groups. For the google groups, we we could subscribe the old spark google groups to the apache lists (instead of using the new mirror groups I created). However the tradeoff is that as far as I can tell, we w

Re: IMPORTANT: Spark mailing lists moving to Apache by September 1st

2013-12-22 Thread Mike Potts
Thanks!! Is is straightforward to pre-populate it with all the existing threads from apache?

Re: IMPORTANT: Spark mailing lists moving to Apache by September 1st

2013-12-22 Thread Andy Konwinski
Per Matei's suggestion, I've set up two nabble archive lists, one to archive the apache dev list and one to archive the apache user list. user list archive: http://apache-spark-user-list.1001560.n3.nabble.com dev list archive: http://apache-spark-developers-list.1001551.n3.nabble.com Between thes