Problem in avg function Spark 1.6.3 using spark-shell

2017-06-25 Thread Eko Susilo
Hi, I have a data frame collection called “secondDf” when I tried to perform groupBy and then sum of each column it works perfectly. However when I tried to calculate average of that column it says the column name is not found. The details are as follow val total = secondDf.filter("ImageWidth

Re: Spark Streaming for time consuming job

2014-10-02 Thread Eko Susilo
Ph: +1 (760) 203 3257 http://www.sigmoidanalytics.com @mayur_rustagi https://twitter.com/mayur_rustagi On Tue, Sep 30, 2014 at 3:22 PM, Eko Susilo eko.harmawan.sus...@gmail.com wrote: Hi All, I have a problem that i would like to consult about spark streaming. I have a spark streaming

Spark Streaming for time consuming job

2014-09-30 Thread Eko Susilo
by the framework such that i don't have to run a separate thread for this? Thank you for your attention. -- Best Regards, Eko Susilo

Spark Streaming reset state

2014-08-29 Thread Eko Susilo
-- Best Regards, Eko Susilo

Re: Spark Streaming reset state

2014-08-29 Thread Eko Susilo
Spark Streaming holds on to the RDDs itself. Clarify what you mean by reset? codes provides a stream of RDDs that contain your computation over a window of time. New RDDs come with the computation over new data. On Fri, Aug 29, 2014 at 4:30 PM, Eko Susilo eko.harmawan.sus...@gmail.com wrote