Can you change the storage process so that each new set of data goes to a different table? Then you could sweep information to a aggregation table without any conflicts.
On Wed, Sep 4, 2013 at 1:21 PM, John Vines <[email protected]> wrote: > No, the iterator structures, which filters are built on, do not provide a > mechanism for writing to another table. While you could put a writer in > there, it is HIGHLY not recommended because it can potentially cause > deadlocking. You will need an external process to manage writing of old > records to a new table. > > > On Wed, Sep 4, 2013 at 1:11 PM, Slater, David M. > <[email protected]>wrote: > >> Hey,**** >> >> ** ** >> >> I was wondering if it was possible (and a good idea) to modify an >> AgeOffFilter so that when a record is aged off, it gets sent to a different >> table. **** >> >> ** ** >> >> Basically, I am storing netflow data and aggregating that information >> (through a Combiner) into a single entry. Updates to the aggregated record >> come regularly when the flow is active, but when the no updates have come >> in a certain amount of time (e.g. 30 seconds), I assume that the flow is >> done and want to migrate that to a different table (with further indexing). >> It is important that I migrate it from the current table so that the >> combiner doesn’t combine two different flows (separated sufficiently in >> time) with the same Key (ips, ports, and protocol). **** >> >> ** ** >> >> Is there a way to do this with AgeOffFilters? Would I need an external >> program running that would periodically scan the table with an ageofffilter >> and then re-ingest those netflows into a new table? **** >> >> ** ** >> >> Regards, >> David**** >> > >
