@Daniel Thanks for your reply. I will try it. On Wed, Dec 7, 2016 at 8:47 PM, Daniel Haviv < daniel.ha...@veracity-group.com> wrote:
> Hi Anty, > What you could do is keep in the state only the existence of a key and > when necessary pull it from a secondary state store like HDFS or HBASE. > > Daniel > > On Wed, Dec 7, 2016 at 1:42 PM, Anty Rao <ant....@gmail.com> wrote: > >> Hi >> I'm new to Spark. I'm doing some research to see if spark streaming can >> solve my problem. I don't want to keep per-key state,b/c my data set is >> very huge and keep a little longer time, it not viable to keep all per key >> state in memory.Instead, i want to have a bloom filter based state. Does it >> possible to achieve this in Spark streaming. >> >> -- >> Anty Rao >> > > -- Anty Rao
