For multi tenant, the river concept is awkward. River is a singleton and is bound to single user execution, and you are right, creating river instances per DB and per index does not scale.
There are several options: - write a more sophisticated plugin which acts as a service and not as a singleton. The ES service component, which would maintain state in the cluster state, could accept job requests where each job request is equivalent to a JDBC pull. The job requests are delegated to a node which is not very busy with jobs (load balancing). The code of the JDBC river can be reused for that. - write a separate middleware for your tenants where they can have separate access to the DB and prepare ES JSON bulk files from (maybe be by REST API calls similar in style to ES). This would be a domain specific solution but offers most flexibility to the tenants, they are free to decide how and when to create and index the data from DB. Jörg On Tue, Aug 26, 2014 at 11:21 AM, Nitin Maheshwari <ask4ni...@gmail.com> wrote: > Hi Jörg, > > I am working on a multi tenant application where each tenant has its own > database. I am planning to use ES for indexing the data, and JDBC river for > doing periodic bulk indexing. I do not want to create one river per DB per > object type. This will lead to too many rivers. > > I wanted to modify the JDBC river so that I can give parent DB location, > where all tenant db connection information is available. And then inside > the river, modify it such that a feader thread is created for each river. > > Do you see any issue with this or do you have any other recommendation? > > Thanks, > Nitin > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to elasticsearch+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/771fc3a8-2203-4db8-a07b-067430e7a473%40googlegroups.com > <https://groups.google.com/d/msgid/elasticsearch/771fc3a8-2203-4db8-a07b-067430e7a473%40googlegroups.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEZumpR1oazTuVn6Ofad71jgEMkSBOKARizK9-gOFpVsA%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
