That's the reason for the warning! In my understanding it may happen.
Events deleted by RocksDB TTL may be added back if restored from the
changelog.
Boris.
On Wed, Feb 3, 2016 at 5:46 PM, David Garcia
wrote:
> Boris, thank you for the clarification. But just to make sure I
> understand, is it
Hi David,
My understanding is that Samza changelog could still be logging those
entries deleted by RocksDB TTL. You could refer to SAMZA-677 for more
information.
Thanks,
-Tao
On Wed, Feb 3, 2016 at 5:46 PM, David Garcia
wrote:
> Boris, thank you for the clarification. But just to make sure I
Boris, thank you for the clarification. But just to make sure I
understand, is it correct to say that entries deleted by the TTL-policy in
rocksDB will NOT be logged in the change-log? My job processes a lot of
data and saves a large portion of it to RocksDB (for reference later…but
subject to a
As Jacob mentioned there is not direct relationship between the rocksdb tts
(internal to rocksdb) and changelog (done by Samza).
The problem may arise if the store is restored from the changelog, since
the log will have the expired entries, and they will be entered with the
NEW date (and as Yi ment
Hi, David,
The "compaction" referred to together w/ TTL is referring to RocksDb's
compaction, not the Kafka-based changelog topic. Currently, TTL is not
applied to Kafka-based changelog topic. SAMZA-677 is opened for this.
-Yi
On Thu, Jan 28, 2016 at 11:36 AM, David Garcia wrote:
> Ok, that ma
Ok, that makes sense. I had assumed that the changelog was supported
because the docs mention that TTL is enforced upon ³compaction² (I had
assumed compaction of the DB changelog). Which topic does the TTL policy
listen for the compaction of (since compaction policies of topics can
differ)?
-Dav
Here's my understanding. The others can correct me if I'm mistaken.
Samza provides the changelog functionality by intercepting RocksDB "put"
and "delete" operations. However, TTL is managed by RocksDB internally and
there aren't any hooks exposed in the RocksDB JNI. So there are 2 problems
that ar
So, I saw this very scary message:
ERROR - e.kv.RocksDbKeyValueStore$ - sessionJoinStore is a TTL based store,
changelog is not supported for TTL based stores, use at your own discretion
A few of questions:
1.) Does this mean that this store is NOT backed by the changelog?
2.) Provided tha
