Re: Kafka connect JDBC DB SCNs

2016-02-24 Thread Chris Riccomini
Given how messed up MySQL is, in particular, it seems to me like adding a config that can do a loobback (either ID or timestamp-based) would we useful, and pretty straight forward to add. This will at least help in preventing data loss, at the expense of duplicate rows. If people *really* care abou

Re: Kafka connect JDBC DB SCNs

2016-02-24 Thread Chris Riccomini
Hey all, Some more details on my question. My concern about using even IDs on immutable tables as a way to replicate data seems to be confirmed, at least for InnoDB if not configured properly. This page: http://dev.mysql.com/doc/refman/5.7/en/innodb-auto-increment-handling.html Describes how aut

Kafka connect JDBC DB SCNs

2016-02-23 Thread Chris Riccomini
Hey all, I was reviewing the Kafka connect JDBC driver, and I had a question. Is it possible to use the JDBC driver with a look-back configured? The reason that I ask is that there are some known issues with using a modified timestamp: Slide 14 here explains one with Oracle: https://qconsf.com/s