Hello,
We can't keep buffer in memory because it is too expensive. For example, if
we have a 15 minutes interval and 30 Mb/s, than we have to use about 3 gb of
memory. Sounds ugly. Instead of that we can store somewhere a list of backed
up files, so we can recover a job, that has been hanged up. Does it sound
reasonable, or I'm missing something?
--
Ilya M. Slepnev
On 12/10/07, Kern Sibbald <[EMAIL PROTECTED]> wrote:
>
> Hello,
>
> On Monday 10 December 2007 14:44, Ilya M. Slepnev wrote:
> > Hello,
> >
> > I have a problem with backuping some large amounts of data (about
> several
> > terabytes).
> > It lasts approximately 3-4 days, and sometimes network is falling down
> for
> > 15-20 minutes. I'd like to see if some rollback points can be used? Any
> > ideas?
> >
>
> Well, my first comment would be to rethink how you are doing backups so
> that
> no individual job runs more than about 12 hours (well maybe 24 at a
> maximum),
> because longer jobs are virtually guaranteed to fail at some point causing
> a
> lot of wasted time and data storage ... You can do this by splitting
> your
> job into a number of smaller ones, that could even run simultaneously. It
>
> will be more complicated for restores, but will probably save you a lot of
> headaches.
>
> Now to respond to your question: Bacula was designed with the assumption
> that
> one will always have a good TCP/IP connection and that TCP/IP already
> guarantees valid end to end data. As a consequence, a major redesign of
> Bacula would be needed to implement rollbacks points.
>
> However, an alternative (that is not on my list of things to do) would be
> for
> someone to implement new code in the low level Bacula "bsock"
> communications
> routines that would detect a hang up, reconnect, and resynchronize.
>
> Detecting a hang up and reconnecting is relatively straight forward code.
> However resynchronizing is a bit more complicated. Currently for
> efficiency
> reasons, Bacula simply blasts the data from the FD to the SD. The FD does
> continuous writes into the socket and the SD does continuous reads on the
> other end. Doing a read on the socket in the FD after every write is
> *very*
> expensive and slows Bacula down by about 10 times, so we need something
> more
> sophisticated. To resynchronize, there would have to be some sort of
> handshake and counter, and would probably require buffering a number of
> packets (probably 10-100) then sending back an ACK when they arrive. That
> way, if a disconnect occurred, the FD/SD could reestablish a connection
> and
> resend any lost buffers, then continue. This would be "effectively"
> equivalent to your rollback point ...
>
> I'll be happy to give advice and review any proposed schemes, but at the
> moment, there are way too many more important projects on my personal
> list.
> If anyone wants to program something, be sure to read the Developer's
> Guide
> (on the web site) and fill out and send in a fiduciary license agreement
> (see
> www.bacula.org -> FSFE License).
>
> Best regards,
>
> Kern
>
-------------------------------------------------------------------------
SF.Net email is sponsored by:
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
Bacula-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/bacula-devel
