Hi, The clients are different services in a larger micro-service landscape. Some of them will store small amounts of data (less than 1GB, maybe even less than 100MB), others large amounts. The services with small amounts of data make me worry about efficient usage of memory and in-memory-stands. If they share a database, the shared database could have larger in-memory stands (in contrast to many small in-memory stands of the individual databases). I assume that larger in-memory stands perform much better in peak moments?! Additionally, it is easier to tune the configuration of one database vs. to tune the configuration of many databases.
On the other hand, we want to have an easy backup & restore process. Do you have any suggestions or experience on how this could be done in a shared database on a directory level? The backup could be done with the MLCP (export, point-in-time). The restore with MLCP would be a step-process: remove all content from the directory, then import the backup. This is not as straight-forward as the builtin backup features. Security, SLAs and data sharing are relevant topics which I feel comfortable with. Maybe we'll go with a mix of shared and individual databases, even though this means a more complex architecture. Thanks, Andreas 2017-11-23 21:18 GMT+01:00 David Gorbet <[email protected]>: > If these are completely separate use cases please consider completely > separate clusters. You can use virtualization to make the hardware work out. > > On Nov 23, 2017, at 12:04 PM, Geert Josten <[email protected]> > wrote: > > Hi Andreas, > > I think each forest has its own in-memory stand, so if each client has a > reasonable amount of data, you’ll need several forests per client anyhow. > One or multiple databases wouldn’t matter much in that case. I wouldn’t > worry too much about in-memory stands though. Memory is much faster than > disk, so worth using. And you’ll want spare resources anyhow to handle peak > moments, so not fully utilizing resources all the time isn’t bad > necessarily. An average use of 30% of cpu and mem is pretty typical i’d say. > > I would suggest looking at it more from a business or functional > perspective. For instance: > > - Do you need to guarantee clients won’t be able to see each others > data? That would be a strong argument to want to keep things separate > without doubt. > - Could different clients have different SLA terms? Another vote for > keeping things separate. > - What if one clients wants to step out, and you need to purge its > data? Dead simple with separate databases > - Is there any change one of the clients would like to run it on-site, > rather than hosted? > - Or for the opposite: would there be any need to mix datasets from > different clients? Any kind of sharing for instance, even if just of > statistics, or some anonymous cross-validation? > > And you can probably think of many more yourself. > > Cheers, > Geert > > From: <[email protected]> on behalf of Andreas > Hubmer <[email protected]> > Reply-To: MarkLogic Developer Discussion <[email protected]> > Date: Thursday, November 23, 2017 at 4:53 PM > To: MarkLogic Developer Discussion <[email protected]> > Subject: [MarkLogic Dev General] Multi-Database Architecture > > Hi, > > I am planning the architecture of an application with dozens of individual > clients. I think of using either one database for all data or a separate > database per client. > > The main pros and cons for me are efficient memory usage and the > possibility of individual backup&restore. I tend to prefer the first and > accept more complicated restore scenarios. > > These are my considerations. > > one-db: > * each client would use a different base directory (security: > uri-privileges) > * 1 in-memory-stand -> more efficient memory usage. Do you agree that this > is relevant? > ** *individual backup & restore of data of one client => complicated > (MLCP?) > > many-dbs (one db per client): > * many in-memory-stands -> less efficient memory usage / more smaller > stands / more merging. Do you agree? > * builtin backup & restore of data of one client is possible > * very flexible configuration (individual indexes, ...) > * deployment more complex > > For configuration we will use Roxy. > > Thanks, > Andreas > > -- > Andreas Hubmer > Senior IT Consultant > > EBCONT enterprise technologies GmbH > Millennium Tower > Handelskai 94-96 > A-1200 Vienna > > Mobile: +43 664 60651861 <+43%20664%2060651861> > Fax: +43 2772 512 69-9 > Email: [email protected] > Web: http://www.ebcont.com > > OUR TEAM IS YOUR SUCCESS > > UID-Nr. ATU68135644 > HG St.Pölten - FN 399978 d > > VERTRAULICHKEITSHINWEIS/HAFTUNGSAUSSCHLUSS: > Der Inhalt dieser E-Mail und alle beigefügten Anhänge sind vertraulich zu > behandeln, sind vor Veröffentlichung rechtlich geschützt und sind > ausschließlich für den bezeichneten Adressaten bestimmt. Wenn Sie nicht der > vorgesehene Empfänger sind, informieren Sie den Absender bitte umgehend und > vernichten Sie diese E-Mail samt allen beigefügten Anhängen. Der Inhalt > dieser Email darf nicht an/oder von dritten weitergeleitet, veröffentlicht, > verwendet, kopiert oder auf andere Medien gespeichert werden. Wir > übernehmen keine Haftung für eventuelle Schäden, die durch diese E-Mail > oder deren Anhänge entstehen könnten. > > CONFIDENTIALITY/DISCLAIMER: > This email and any files transmitted with it are confidential, are legally > protected before publication and are intended solely for the use of the > individual or entity to whom they are addressed. If you have received this > email in error, please notify the sender immediately and destroy this > e-mail together with all attachments. The content of this e-mail may not be > be disseminated, published, copied or stored on third parties. We assume no > liability for any damage that may result from this e-mail or its annexes. > > _______________________________________________ > General mailing list > [email protected] > Manage your subscription at: > http://developer.marklogic.com/mailman/listinfo/general > > > _______________________________________________ > General mailing list > [email protected] > Manage your subscription at: > http://developer.marklogic.com/mailman/listinfo/general > > -- Andreas Hubmer Senior IT Consultant EBCONT enterprise technologies GmbH Millennium Tower Handelskai 94-96 A-1200 Vienna Mobile: +43 664 60651861 Fax: +43 2772 512 69-9 Email: [email protected] Web: http://www.ebcont.com OUR TEAM IS YOUR SUCCESS UID-Nr. ATU68135644 HG St.Pölten - FN 399978 d VERTRAULICHKEITSHINWEIS/HAFTUNGSAUSSCHLUSS: Der Inhalt dieser E-Mail und alle beigefügten Anhänge sind vertraulich zu behandeln, sind vor Veröffentlichung rechtlich geschützt und sind ausschließlich für den bezeichneten Adressaten bestimmt. Wenn Sie nicht der vorgesehene Empfänger sind, informieren Sie den Absender bitte umgehend und vernichten Sie diese E-Mail samt allen beigefügten Anhängen. Der Inhalt dieser Email darf nicht an/oder von dritten weitergeleitet, veröffentlicht, verwendet, kopiert oder auf andere Medien gespeichert werden. Wir übernehmen keine Haftung für eventuelle Schäden, die durch diese E-Mail oder deren Anhänge entstehen könnten. CONFIDENTIALITY/DISCLAIMER: This email and any files transmitted with it are confidential, are legally protected before publication and are intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error, please notify the sender immediately and destroy this e-mail together with all attachments. The content of this e-mail may not be be disseminated, published, copied or stored on third parties. We assume no liability for any damage that may result from this e-mail or its annexes.
_______________________________________________ General mailing list [email protected] Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
