Re: Usage of transform for code reuse between Streaming and Batch job affects the performance ?

2015-10-05 Thread Adrian Tanase
It shouldn't, as lots of the streaming operations delegate to transform under the hood. Easiest way to make sure is to look at the source code - with a decent IDE navigating around should be a breeze. As a matter of fact, for more advanced operations where you may want to control the

Usage of transform for code reuse between Streaming and Batch job affects the performance ?

2015-10-04 Thread swetha
Hi, I have the following code for code reuse between the batch and the streaming job * val groupedAndSortedSessions = sessions.transform(rdd=>JobCommon.getGroupedAndSortedSessions(rdd))* The same code without code reuse between the batch and the streaming has the following. * val