Re: Is spark not good for ingesting into updatable databases?

2018-10-30 Thread ravidspark
Hi Jorn, Just want to check if you got a chance to look at this problem. I couldn't figure out any reason on why this is happening. Any help would be appreciated. -- Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/

Re: unsubsribe

2018-10-30 Thread Anu B Nair
I have already send minimum 10 times! Today also I have send one! On Tue, Oct 30, 2018 at 3:51 PM Biplob Biswas wrote: > You need to send the email to user-unsubscr...@spark.apache.org and not > to the usergroup. > > Thanks & Regards > Biplob Biswas > > > On Tue, Oct 30, 2018 at 10:59 AM Anu B

Re: unsubsribe

2018-10-30 Thread Biplob Biswas
You need to send the email to user-unsubscr...@spark.apache.org and not to the usergroup. Thanks & Regards Biplob Biswas On Tue, Oct 30, 2018 at 10:59 AM Anu B Nair wrote: > I am sending this Unsubscribe mail for last few months! It never happens! > If anyone can help us to unsubscribe it wil

Re: unsubsribe

2018-10-30 Thread Anu B Nair
I am sending this Unsubscribe mail for last few months! It never happens! If anyone can help us to unsubscribe it wil be really helpful! On Tue, Oct 30, 2018 at 3:27 PM Mohan Palavancha wrote: > >

unsubsribe

2018-10-30 Thread Mohan Palavancha

Java Spark to Python spark integration

2018-10-30 Thread Manohar Rao
I would like to know if its possible to invoke python spark code from java. I have a java based framework where a sparksession is created and a some dataframes are passed as argument to an api . Transformation.java interface Transformation { Dataset transform(Set inputDatasets ,

Re: java vs scala for Apache Spark - is there a performance difference ?

2018-10-30 Thread Jörn Franke
Older versions of Spark had indeed a lower performance on Python and R due to a conversion need between JVM datatypes and python/r datatypes. This changed in Spark 2.2, I think, with the integration of Apache Arrow. However, what you do after the conversion in those languages can be still

Re: dremel paper example schema

2018-10-30 Thread Gourav Sengupta
Super, Now it makes sense, I am copying Holden in this email. Regards, Gourav On Tue, 30 Oct 2018, 06:34 lchorbadjiev, wrote: > Hi Gourav, > > the question in fact is are there any the limitations of Apache Spark > support for Parquet file format. > > The example schema from the dremel paper

Re: dremel paper example schema

2018-10-30 Thread Jörn Franke
Are you using the same parquet version as Spark uses? Are you using a recent version of Spark? Why don’t you create the file in Spark? > Am 30.10.2018 um 07:34 schrieb lchorbadjiev : > > Hi Gourav, > > the question in fact is are there any the limitations of Apache Spark > support for Parquet

Re: dremel paper example schema

2018-10-30 Thread lchorbadjiev
Hi Gourav, the question in fact is are there any the limitations of Apache Spark support for Parquet file format. The example schema from the dremel paper is something that is supported in Apache Parquet (using Apache Parquet Java API). Now I am trying to implement the same schema using Apache

Re: java vs scala for Apache Spark - is there a performance difference ?

2018-10-30 Thread akshay naidu
how about Python. java vs scala vs python vs R which is better. On Sat, Oct 27, 2018 at 3:34 AM karan alang wrote: > Hello > - is there a "performance" difference when using Java or Scala for Apache > Spark ? > > I understand, there are other obvious differences (less code with scala, > easier