RE: Specifying Scala types when calling methods from SparkR

2015-12-10 Thread Chris Freeman
--- From: Shivaram Venkataraman [mailto:shiva...@eecs.berkeley.edu] Sent: Thursday, December 10, 2015 8:21 AM To: Chris Freeman Cc: dev@spark.apache.org Subject: Re: Specifying Scala types when calling methods from SparkR The SparkR callJMethod can only invoke methods as they show up in the Java byte co

RE: Specifying Scala types when calling methods from SparkR

2015-12-10 Thread Sun, Rui
...@alteryx.com] Sent: Friday, December 11, 2015 2:47 AM To: Sun, Rui; shiva...@eecs.berkeley.edu Cc: dev@spark.apache.org Subject: RE: Specifying Scala types when calling methods from SparkR Hi Sun Rui, I’ve had some luck simply using “objectFile” when saving from SparkR directly. The problem is that if you

RE: Specifying Scala types when calling methods from SparkR

2015-12-09 Thread Sun, Rui
riginal Message- From: Shivaram Venkataraman [mailto:shiva...@eecs.berkeley.edu] Sent: Thursday, December 10, 2015 8:21 AM To: Chris Freeman Cc: dev@spark.apache.org Subject: Re: Specifying Scala types when calling methods from SparkR The SparkR callJMethod can only invoke methods as they show

Re: Specifying Scala types when calling methods from SparkR

2015-12-09 Thread Shivaram Venkataraman
The SparkR callJMethod can only invoke methods as they show up in the Java byte code. So in this case you'll need to check the SparkContext byte code (with javap or something like that) to see how that method looks. My guess is the type is passed in as a class tag argument, so you'll need to do