File system based indexers bring new challenges in the container world. It also poses challenges in HA (multinode) and multi regional deployments. Therefore if we're selecting a solr indexing approach, the only practical solution is to go with a solr server based architecture.
Even with a solr server assisted architecture, we still have complexities when dealing with multi regional deployments. Also since API artifacts reside in the database, if we're loading search results from an index, we have to sync the permissions of the artifacts in the index too. Plus this makes the deployment complex. Given the complexities that have to be addressed in an indexing solution, I'd prefer to opt with the DB based search at least to start off with. Since APIs and their permissions are both maintained in the DB, it would be much simple if the search also works on that. Unlike in C4 this database will only have data of 1 tenant. Hence we're not expecting the API count to be in the thousands. Therefore I think searching through the database should be feasible. Thanks, NuwanD. On Sat, 14 Jan 2017 at 8:27 am, Chandana Napagoda <chand...@wso2.com> wrote: > Hi Rajith, > > I believe indexing based approach is more suitable for any search solution > because its design is to support fast search results. If you use database > search, you have to use multiple DB indexes to improve search performance, > and that will reduce the performance of insert operations. > > I think, REG_LOG kind of history table is not necessary for APIM product, > since it is totally aware of the APIM artifacts(APIs, Docs, forum posts) > and you can directly connect to the DB[1] from the text search engine and > index the resources. As per the Solr documentation, it is capable of > importing only the new additions(delta) for indexing. > > [1]. > https://cwiki.apache.org/confluence/display/solr/Uploading+Structured+Data+Store+Data+with+the+Data+Import+Handler > > Regards, > Chandana > > > On Fri, Jan 13, 2017 at 11:44 AM, Rajith Roshan <raji...@wso2.com> wrote: > > Hi all, > > We are currently evaluating how to perform full text search at the > database level for C5 based API Manager. We will be evaluating this for > different types of databases to find their implementation complexities and > limitations. > Other option available for us to use indexing based approach (use Solr) > > *Database full text search * > *Pros* > > - Less complications when using container based approach > - Clustering will require only database syncing. > - No need to maintain and ship external search engine. > > *Cons* > > - Implementation may vary significantly based on the database type > - There can be limitation in full text search for particular database > types (For ex: mysql full text support only prefix search) > - Queries will differ based on database type > - Document search will not be available, because they are stored as > blobs > > > *Indexing based approach * > *Pros* > > - Document search > - Search will be efficient (No need to access database) > > *Cons* > > - Since indexing data is written to file system , when going for > container based approach we would require mechanisms to file system > mounting > - Syncing indexers in a cluster would require something similar to > existing C4 based registry architecture (use of REG_LOG table) > - Maintaining (for ex: Version updates) and shipping external search > engine. > > Your valuable input regrading this is highly appreciated. > > Thanks! > Rajith > > -- > Rajith Roshan > Software Engineer, WSO2 Inc. > > > Mobile: +94-72-642-8350 <%2B94-71-554-8430> > > > > > > > > -- > *Chandana Napagoda* > Associate Technical Lead > WSO2 Inc. - http://wso2.org > > *Email : chand...@wso2.com <chand...@wso2.com>**Mobile : +94718169299* > > *Blog : http://cnapagoda.blogspot.com <http://cnapagoda.blogspot.com> > | http://chandana.napagoda.com <http://chandana.napagoda.com>* > > *Linkedin : http://www.linkedin.com/in/chandananapagoda > <http://www.linkedin.com/in/chandananapagoda>* > > > >
_______________________________________________ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
