JDBC plugin supports MySQL streaming mode out of the box: https://github.com/jprante/elasticsearch-river-jdbc/issues/520#issuecomment-89789655
If it does not work, I'm available for help to find alternatives. I would love to know more about the observed DB timeout. The MySQL timeouts can be changed in configuration, but I am unsure if streaming mode can result in timeouts - that would be a defect on MySQL JDBC driver level. Jörg On Sun, Apr 5, 2015 at 12:18 AM, Jonathan Zhang <topcod...@gmail.com> wrote: > does ES jdbc river/feeder have a mechanism to feed data from database into > ES via jdbc incrementally in a stepwise function? > I have billions of records to be imported into ES from mysql and a select > * from my table would take a long time and be interrupted due to db > timeout. I need to do select * from my_table where my_id>start_id and > my_id<end_id and iterate through somehow. feeder/river doesn't seem to > provides such mechanism, or am I missing something? > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to elasticsearch+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/f8640525-fc46-4dcd-8fbe-a4ce0cd50f78%40googlegroups.com > <https://groups.google.com/d/msgid/elasticsearch/f8640525-fc46-4dcd-8fbe-a4ce0cd50f78%40googlegroups.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEzaS3y97PBSr%3DHRnsvYvPSNwU_%2Bv6sb%2BVXX%2BimQxf7Hg%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
