Hi Karl, I was thinking about may be we can add shards from the job UI. On second thought it’s out of our scope. User should do it himself/herself.
I thought that good seeding model increasing the MCF performance. GridFS connector works with MODEL_ALL. Assume that the user also stores added and changed documents’ metadata(or key) in a mongodb collection. If the user wants to select other seeding model, may be we can get from the user a query which returns the added and changed documents then the user can use MODEL_ADD_CHANGE. What do you think? Muhammed On 19 Jun 2014, at 00:40, Karl Wright <[email protected]> wrote: > Hi Muhammed, > > Can you go into more depth about these: > >>>>>>> > 1) Sharding support > 2) Selectable seeding model. > <<<<<< > > Thanks, > Karl > > > > On Wed, Jun 18, 2014 at 5:38 PM, Karl Wright <[email protected]> wrote: > >> bq. What is "non-SQL data store" ? You mean to remove MFC's dependency to >> PostgreSQL, MySQL, Derby etc? >> >> See CONNECTORS-286. >> >> bq. What do you think about this? Can MCF be dih replacement? How is our >> DB crawler compared to DIH? >> >> In theory it could. I'd hesitate before claiming feature-to-feature >> compatibility though, and I'm not sure whether Solr people would officially >> recommend MCF in any case, especially since they have wanted to solve >> document security in their own way (but have never gotten around to it in >> the 3+ years this first came to my attention). >> >> Karl >> >> >> >> On Wed, Jun 18, 2014 at 5:26 PM, Ahmet Arslan <[email protected]> >> wrote: >> >>> Hi, >>> >>> What is "non-SQL data store" ? You mean to remove MFC's dependency to >>> PostgreSQL, MySQL, Derby etc? >>> >>> >>> By the way solr guys are looking for a Data Import Handler (DIH) >>> replacement. >>> >>> See for the thread : http://search-lucene.com/m/WwzTb2z1w7F >>> >>> DIH is mostly used to sync RDBMS to Solr. >>> >>> What do you think about this? Can MCF be dih replacement? >>> >>> How is our DB crawler compared to DIH? >>> >>> Ahmet >>> >>> >>> On Wednesday, June 18, 2014 11:33 PM, Muhammed Olgun <[email protected]> >>> wrote: >>> Hi all, >>> >>> I think that a non-SQL solution would be great. I have also two new ideas >>> for GridFS connector, >>> >>> 1) Sharding support >>> 2) Selectable seeding model. >>> >>> Muhammed >>> >>> >>> >>> >>> >>> On 18 Jun 2014, at 23:22, Karl Wright <[email protected]> wrote: >>> >>>> Hi Piergiorgio, >>>> >>>> Just to clarify -- I don't have a workable plan yet for a non-SQL data >>>> store, so maybe that waits until 3.0. >>>> >>>> Karl >>>> >>>> >>>> >>>> On Wed, Jun 18, 2014 at 3:13 PM, Piergiorgio Lucidi < >>> [email protected]> >>>> wrote: >>>> >>>>> +1 from me for breaking backwords compatibility and focusing on non-SQL >>>>> data store. >>>>> >>>>> Piergiorgio >>>>> >>>>> >>>>> 2014-06-18 18:19 GMT+02:00 Karl Wright <[email protected]>: >>>>> >>>>>> Hi all, >>>>>> >>>>>> By now it is becoming clear that ManifoldCF has accumulated a lot of >>>>>> backwards-compatibility dead weight we have to carry around from >>> release >>>>> to >>>>>> release. However, ManifoldCF 2.0 will present an opportunity to break >>>>>> backwards compatibility with the 1.x releases. Originally, I was >>>>> thinking >>>>>> that MCF 2.0 would be the proper release vehicle for an >>> implementation on >>>>>> top of a non-SQL data store, but now I am looking at this instead as a >>>>>> great way to clean out deprecated tabs, methods, and even whole >>>>> connectors >>>>>> from the codebase. >>>>>> >>>>>> I'd like to consider making the MCF 2.0 release be the next one after >>>>> 1.7. >>>>>> Since 1.7 is scheduled for end of August, 2.0 would come out some >>> months >>>>>> after that. Please comment on whether you agree with this basic >>> plan, or >>>>>> you have other priorities we should know about. ;-) >>>>>> >>>>>> FWIW, if this *is* a good idea to you, please also list one or two >>> main >>>>>> areas we should work on for 2.0 that involve breaking backwards >>>>>> compatibility. >>>>>> >>>>>> Thanks, >>>>>> Karl >>>>>> >>>>>> -- >>>>>> Piergiorgio Lucidi >>>>>> Open Source ECM Specialist >>>>>> http://www.open4dev.com >>>>>> >>>>> >>> >> >>
