Thank you very much for your reply , Daniel On Thu, Dec 8, 2016 at 7:07 PM, Daniel Haviv < daniel.ha...@veracity-group.com> wrote:
> There's no need to extend Spark's API, look at mapWithState for examples. > > On Thu, Dec 8, 2016 at 4:49 AM, Anty Rao <ant....@gmail.com> wrote: > >> >> >> On Wed, Dec 7, 2016 at 7:42 PM, Anty Rao <ant....@gmail.com> wrote: >> >>> Hi >>> I'm new to Spark. I'm doing some research to see if spark streaming can >>> solve my problem. I don't want to keep per-key state,b/c my data set is >>> very huge and keep a little longer time, it not viable to keep all per key >>> state in memory.Instead, i want to have a bloom filter based state. Does it >>> possible to achieve this in Spark streaming. >>> >>> Is it possible to achieve this by extending Spark API? >> >>> -- >>> Anty Rao >>> >> >> >> >> -- >> Anty Rao >> > > -- Anty Rao
