Re: How to validate orc vectorization is working within spark application?

2018-07-12 Thread umargeek
Hello Jorn, I am unable to post the entire code due to some data sharing related issues. Use Case: I am performing aggregations after reading data from HDFS file every min would like to understand how to perform using vectorisation enabled and what are pre requisite to successfully to enable

Re: How to validate orc vectorization is working within spark application?

2018-06-19 Thread Jörn Franke
Full code? What is expected performance and actual ? What is the use case? > On 20. Jun 2018, at 05:33, umargeek wrote: > > Hi Folks, > > I would just require few pointers on the above query w.r.t vectorization > looking forward for support from the community. > > Thanks, > Umar > > > > --

Re: How to validate orc vectorization is working within spark application?

2018-06-19 Thread umargeek
Hi Folks, I would just require few pointers on the above query w.r.t vectorization looking forward for support from the community. Thanks, Umar -- Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/ - To

How to validate orc vectorization is working within spark application?

2018-05-22 Thread umargeek
Hi Folks, I have enabled below listed configurations within my spark streaming application but I did not gain performance benefit even after setting these parameters ,can you please help me is there a way to validate whether vectorization is working as expeced/enabled correctly ! Note: I am