Actually, thanks for asking.
We are currently setting up a repo which, in the end, will hold ~ 10 M
objs (10 TB of disk space). As for now, we have only ~ 3.5 M objs,
ingestion is finally running and we started reasoning on HA and DR.
The infrastructure we envisaged is a Fedora leader/follower on
high-perf disks for HA and a separate ("lukewarm") DR repo on low-perf
disks, since the time to get ~ 10 TB of data from standard tape
backups is out of the project specs.
So, my first approach was rsync-ing (with some malicious tricks to
avoid parsing ~ 10Mobjs and/or rsyncing eventually corrupted DS)
nightly $FEDORA_HOME/data/(objectStore|datastreamStore) from leader to
DR, and then perform a rebuild on DR. But since fedora-rebuild
actually does a "DELETE FROM " + origTableName" (which makes totally
sense when doing a real rebuild) but in our case this means rebuilding
nightly and from scratch a huge Sql DB /ResourceIndex, which is not
efficient.
So, we are currently pallning to go for rsync (includingresourceIndex)
+ DB export on leader/DB import on DR, to get the DR up-to-date.
Does this make sense ?
What's your opinion about the whole architecture ?
Ideas/comments/suggestions/criticisms welcomed...
On Tue, Jun 19, 2012 at 10:34 PM, [email protected] <[email protected]> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> It's not quite clear to me why you would want to do this... rebuilding the
> database is meant to ensure that your database is completely coherent with
> your data (object XML and datastreams). Can you tell us more about your use
> case?
>
> - ---
> A. Soroka
> Software & Systems Engineering :: Online Library Environment
> the University of Virginia Library
>
> On Jun 19, 2012, at 5:17 AM, Matteo Boschini wrote:
>
>> Hi all,
>>
>> I browsed the docs and some source code. I was wondering if there's a
>> way I can avoid deleteing (blanking) existing tables when running
>> fedora-rebuild SQL.
>> Is it an option of fedora-rebuild ? (doesn't look like), a line in
>> fedora.fcfg or do I need to make my own copy of
>> org.fcrepo.server.utilities.rebuild.SQLRebuilder and get rid of
>> blankExistingTables ?
>>
>> thanks already
>>
>> Matteo Boschini
>>
>> ------------------------------------------------------------------------------
>> Live Security Virtual Conference
>> Exclusive live event will cover all the ways today's security and
>> threat landscape has changed and how IT managers can respond. Discussions
>> will include endpoint security, mobile security and the latest in malware
>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>> _______________________________________________
>> Fedora-commons-users mailing list
>> [email protected]
>> https://lists.sourceforge.net/lists/listinfo/fedora-commons-users
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
> Comment: GPGTools - http://gpgtools.org
>
> iQEcBAEBAgAGBQJP4OJiAAoJEATpPYSyaoIkP1QH/2QNQzQiPJ/qqOM/3dpEy40B
> rh16L6h9hr6eS8HJYtYNwLDbq3CxaoMP2UYIbagSNPXV0ZUcvUk/7unwmvo/Zl8a
> Q4ukaCLAPG+FiRGgGU0wAwcyK0GJ9jCsrz00hLmJsqUsh7WtavfCSD4pLiajz1+D
> HxxD4ZecRdMKU3uJJEOv6V0yrkFnu0mH7z8RmZLrLsvwjoxugfUFbLEMm3E2p0FY
> uHCIwvcbdtq6Lq6+2dQWsmm90WJ/PMKPyFIt9D7sisqVa3mIvF4FW/RrQX+KVQKy
> s9t6m5ICeUSjyU5v+0kv+y8wnBQY92Q5hbPhwpcWAUG9STaxUozH2bMQehHtQs4=
> =YU5m
> -----END PGP SIGNATURE-----
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Fedora-commons-users mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/fedora-commons-users
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Fedora-commons-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/fedora-commons-users
