Re: Using Spark with a SOCKS proxy

Did you try ssh tunneling instead of SOCKS? Thanks Best Regards On Wed, Mar 18, 2015 at 5:45 AM, Kelly, Jonathan jonat...@amazon.com wrote: I'm trying to figure out how I might be able to use Spark with a SOCKS proxy. That is, my dream is to be able to write code in my IDE then run it

Re: Fixed worker ports in the spark worker

You can fix the ports in the configuration - http://spark.apache.org/docs/1.2.0/configuration.html#networking On Wed, Mar 18, 2015 at 11:10 AM, Niranda Perera niranda.per...@gmail.com wrote: Hi all, I see that spark server opens up random ports, especially in the workers. is there any way

Re: Fixed worker ports in the spark worker

Thanks Arush. this is governed by the conf/spark-defaults.conf config, is it? On Wed, Mar 18, 2015 at 1:30 PM, Arush Kharbanda ar...@sigmoidanalytics.com wrote: You can fix the ports in the configuration - http://spark.apache.org/docs/1.2.0/configuration.html#networking On Wed, Mar 18,

Re: [mllib] Is there any bugs to divide a Breeze sparse vectors at Spark v1.3.0-rc3?

sure. On Wed, Mar 18, 2015 at 12:19 AM, Debasish Das debasish.da...@gmail.com wrote: Hi David, We are stress testing breeze.optimize.proximal and nnls...if you are cutting a release now, we will need another release soon once we get the runtime optimizations in place and merged to breeze.

Re: [mllib] Is there any bugs to divide a Breeze sparse vectors at Spark v1.3.0-rc3?

Sorry for the delay in replying. I moved from Tokyo to New York in order to attend Spark Summit East. I verified the snapshot and the difference. https://github.com/scalanlp/breeze/commit/f61d2f61137807651fc860404a244640e213f6d3 Thank you for your great work! Yu Ishikawa - -- Yu Ishikawa

parquet support - some questions about code

Hi, I am trying to better understand the code for Parquet support. In particular i got lost trying to understand ParquetRelation and ParquetRelation2. Does ParquetRelation2 is the new code that should completely remove ParquetRelation? ( I think there is some remark in the code notifying this

Re: [mllib] Is there any bugs to divide a Breeze sparse vectors at Spark v1.3.0-rc3?

Hi David, We are stress testing breeze.optimize.proximal and nnls...if you are cutting a release now, we will need another release soon once we get the runtime optimizations in place and merged to breeze. Thanks. Deb On Mar 15, 2015 9:39 PM, David Hall david.lw.h...@gmail.com wrote: snapshot

Which linear algebra interface to use within Spark MLlib?

Hi, Currently I am using Breeze within Spark MLlib for linear algebra. I would like to reuse previously allocated matrices for storing the result of matrices multiplication, i.e. I need to use gemm function C:=q*A*B+p*C, which is missing in Breeze (Breeze automatically allocates a new matrix

Re: Block Transfer Service encryption support

Still looking for feedback... I opened the ticket as a minor, thinking of changing it to a major? Anyone object. I did see that a related ticket (https://issues.apache.org/jira/browse/SPARK-6229) is marked as a major... Cheers! Jeff -- View this message in context: