Hi Muhammed, I think it is important that each connector describe its seeding model in the most precise way possible. If you thing GridFS could specify a tighter seeding model, then by all means create a ticket and fix it. But I don't think it's necessary to wait until MCF 2.0 to do it, since the documents returned for MODEL_ALL are a superset of those returned for MODEL_ADD_CHANGE.
Thanks, Karl On Thu, Jun 19, 2014 at 3:10 AM, Muhammed Olgun <[email protected]> wrote: > Hi Karl, > > I was thinking about may be we can add shards from the job UI. On second > thought it’s out of our scope. User should do it himself/herself. > > I thought that good seeding model increasing the MCF performance. GridFS > connector works with MODEL_ALL. Assume that the user also stores added and > changed documents’ metadata(or key) in a mongodb collection. If the user > wants to select other seeding model, may be we can get from the user a > query which returns the added and changed documents then the user can use > MODEL_ADD_CHANGE. > > What do you think? > > Muhammed > > On 19 Jun 2014, at 00:40, Karl Wright <[email protected]> wrote: > > > Hi Muhammed, > > > > Can you go into more depth about these: > > > >>>>>>> > > 1) Sharding support > > 2) Selectable seeding model. > > <<<<<< > > > > Thanks, > > Karl > > > > > > > > On Wed, Jun 18, 2014 at 5:38 PM, Karl Wright <[email protected]> wrote: > > > >> bq. What is "non-SQL data store" ? You mean to remove MFC's dependency > to > >> PostgreSQL, MySQL, Derby etc? > >> > >> See CONNECTORS-286. > >> > >> bq. What do you think about this? Can MCF be dih replacement? How is > our > >> DB crawler compared to DIH? > >> > >> In theory it could. I'd hesitate before claiming feature-to-feature > >> compatibility though, and I'm not sure whether Solr people would > officially > >> recommend MCF in any case, especially since they have wanted to solve > >> document security in their own way (but have never gotten around to it > in > >> the 3+ years this first came to my attention). > >> > >> Karl > >> > >> > >> > >> On Wed, Jun 18, 2014 at 5:26 PM, Ahmet Arslan <[email protected] > > > >> wrote: > >> > >>> Hi, > >>> > >>> What is "non-SQL data store" ? You mean to remove MFC's dependency to > >>> PostgreSQL, MySQL, Derby etc? > >>> > >>> > >>> By the way solr guys are looking for a Data Import Handler (DIH) > >>> replacement. > >>> > >>> See for the thread : http://search-lucene.com/m/WwzTb2z1w7F > >>> > >>> DIH is mostly used to sync RDBMS to Solr. > >>> > >>> What do you think about this? Can MCF be dih replacement? > >>> > >>> How is our DB crawler compared to DIH? > >>> > >>> Ahmet > >>> > >>> > >>> On Wednesday, June 18, 2014 11:33 PM, Muhammed Olgun < > [email protected]> > >>> wrote: > >>> Hi all, > >>> > >>> I think that a non-SQL solution would be great. I have also two new > ideas > >>> for GridFS connector, > >>> > >>> 1) Sharding support > >>> 2) Selectable seeding model. > >>> > >>> Muhammed > >>> > >>> > >>> > >>> > >>> > >>> On 18 Jun 2014, at 23:22, Karl Wright <[email protected]> wrote: > >>> > >>>> Hi Piergiorgio, > >>>> > >>>> Just to clarify -- I don't have a workable plan yet for a non-SQL data > >>>> store, so maybe that waits until 3.0. > >>>> > >>>> Karl > >>>> > >>>> > >>>> > >>>> On Wed, Jun 18, 2014 at 3:13 PM, Piergiorgio Lucidi < > >>> [email protected]> > >>>> wrote: > >>>> > >>>>> +1 from me for breaking backwords compatibility and focusing on > non-SQL > >>>>> data store. > >>>>> > >>>>> Piergiorgio > >>>>> > >>>>> > >>>>> 2014-06-18 18:19 GMT+02:00 Karl Wright <[email protected]>: > >>>>> > >>>>>> Hi all, > >>>>>> > >>>>>> By now it is becoming clear that ManifoldCF has accumulated a lot of > >>>>>> backwards-compatibility dead weight we have to carry around from > >>> release > >>>>> to > >>>>>> release. However, ManifoldCF 2.0 will present an opportunity to > break > >>>>>> backwards compatibility with the 1.x releases. Originally, I was > >>>>> thinking > >>>>>> that MCF 2.0 would be the proper release vehicle for an > >>> implementation on > >>>>>> top of a non-SQL data store, but now I am looking at this instead > as a > >>>>>> great way to clean out deprecated tabs, methods, and even whole > >>>>> connectors > >>>>>> from the codebase. > >>>>>> > >>>>>> I'd like to consider making the MCF 2.0 release be the next one > after > >>>>> 1.7. > >>>>>> Since 1.7 is scheduled for end of August, 2.0 would come out some > >>> months > >>>>>> after that. Please comment on whether you agree with this basic > >>> plan, or > >>>>>> you have other priorities we should know about. ;-) > >>>>>> > >>>>>> FWIW, if this *is* a good idea to you, please also list one or two > >>> main > >>>>>> areas we should work on for 2.0 that involve breaking backwards > >>>>>> compatibility. > >>>>>> > >>>>>> Thanks, > >>>>>> Karl > >>>>>> > >>>>>> -- > >>>>>> Piergiorgio Lucidi > >>>>>> Open Source ECM Specialist > >>>>>> http://www.open4dev.com > >>>>>> > >>>>> > >>> > >> > >> > >
