Re: [CentOS] Backup solution to backup /var/spool/imap above 150GB data

[Rajagopal Swaminathan]

Greetings,

On Fri, Feb 26, 2010 at 5:50 AM, Benjamin Smith wrote:

> Years ago, I set up a backup tool that wrapped rsync. It has faithfully and
> reliably backed up a dozen hosts and too many TB of data to mention,
> offsite,
> automatically saving as many backup points as disk space allows.
> You're certainly welcome to try it!
>
> http://www.effortlessis.com/thisisnotbackupbuddy/
>
> It works on an "ascending powers" basis, EG:
>
> 1 day ago,
> 2 days ago,
> 4 days ago,
> 8 days ago,
> 16 days ago...
> 
> until out of disk space.
>
> =)
>

 it is a criminal offence (in free software world) to
hide this gem from the world for all this long. ;)

Regards,

Rajagopal
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] Backup solution to backup /var/spool/imap above 150GB data

[Benjamin Smith]

Years ago, I set up a backup tool that wrapped rsync. It has faithfully and 
reliably backed up a dozen hosts and too many TB of data to mention, offsite, 
automatically saving as many backup points as disk space allows. 
You're certainly welcome to try it! 

http://www.effortlessis.com/thisisnotbackupbuddy/

It works on an "ascending powers" basis, EG: 

1 day ago, 
2 days ago, 
4 days ago, 
8 days ago, 
16 days ago... 
 
until out of disk space. 

=) 

-Ben 

On Thursday 25 February 2010 10:22:13 am Mike McCarty wrote:
> Agnello George wrote:
> > The requirement fro backup  is not  primarily  for HDD failure , but
> > human error failure . In case one of our user ( eg: the COO with huge 
> > mailbox size has delete all his certain very important mails, and he
> > want to recover them , the contacts us as we are supposed to maintain
> > his mail backup for a week, and we should restore his backup immediately
> >  )  this the main requirement  for the backup  and that too on the same
> > server different partition .
> 
> Have you considered using a snapshot approach? By that, I mean one
> which uses hard links to create the "backup", and as files get added/
> modified, the data are copied, and links are created. Usually, one
> has a snapshot directory with something like a daily snapshot, and
> 24 hourly ones, something like that.
> 
> Mike

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] Backup solution to backup /var/spool/imap above 150GB data

[Mike McCarty]

Karanbir Singh wrote:
> On 02/24/2010 07:44 PM, Les Mikesell wrote:
>>> Err.. raid is NOT backup solution.
>> Neither is a snapshot in another location on the same machine.
> 
> Thats not true, raid is an online setup - different location could be 
> point in time, and on blockdev;s that dont share user access load. Which 

I think that, without causing any more dispute, I can point out
that "backup" covers a wide range of solutions to a less broad
but still not uniquely one set of needs. No one of the means to
backup is a full solution to all the needs which "backup" satisfies.

Even when one is using the term "backup" narrowly in the sense
of "protection from disaster", there are still different kinds
of backup. For example, there is the "full disaster recovery"
or "bare metal" backup, which is intended to work with another
piece of identical hardware, starting with blank fixed storage,
and ending up with a working system which looks identical to the
original at the epoch at which the backup was made. This is
significantly different from one intended merely to restore the
user altered or created data on a machine which has been newly
installed with a compatible version of the OS, for example.

That's why one needs to know the intended use of the backup set
before making any recommendations on procedure and content of the
backup set.

Mike
-- 
p="p=%c%s%c;main(){printf(p,34,p,34);}";main(){printf(p,34,p,34);}
Oppose globalization and One World Governments like the UN.
This message made from 100% recycled bits.
You have found the bank of Larn.
I speak only for myself, and I am unanimous in that!
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] Backup solution to backup /var/spool/imap above 150GB data

[Mike McCarty]

Agnello George wrote:
> The requirement fro backup  is not  primarily  for HDD failure , but human
> error failure . In case one of our user ( eg: the COO with huge  mailbox
> size has delete all his certain very important mails, and he want to recover
> them , the contacts us as we are supposed to maintain his mail backup for a
> week, and we should restore his backup immediately  )  this the main
> requirement  for the backup  and that too on the same server different
> partition .

Have you considered using a snapshot approach? By that, I mean one
which uses hard links to create the "backup", and as files get added/
modified, the data are copied, and links are created. Usually, one
has a snapshot directory with something like a daily snapshot, and
24 hourly ones, something like that.

Mike
-- 
p="p=%c%s%c;main(){printf(p,34,p,34);}";main(){printf(p,34,p,34);}
Oppose globalization and One World Governments like the UN.
This message made from 100% recycled bits.
You have found the bank of Larn.
I speak only for myself, and I am unanimous in that!
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] Backup solution to backup /var/spool/imap above 150GB data

[John Doe]

From: Agnello George 
> is it possible with " brackup "  to back it up to a different server on the 
> same lan   instead of /backup  . Is there any documentation on the same .

It apparently support:
  Brackup::Target::Amazon  backup to Amazon's S3 service
  Brackup::Target::CloudFiles  backup to Rackspace's CloudFiles Service
  Brackup::Target::Filebased   
  Brackup::Target::Filesystem  backup to a locally mounted filesystem
  Brackup::Target::Ftp backup to an FTP server
  Brackup::Target::Sftpbackup to an SSH/SFTP server
So, you could use ftp or sftp...

JD


  
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] Backup solution to backup /var/spool/imap above 150GB data

[JohnS]

On Thu, 2010-02-25 at 12:33 +, Karanbir Singh wrote:

> 
> [1] Changeset and data/system model over time relation mapping for an 
> adaptive system sizing feedback loop! ( how'se that for buzzword 
> injection! )
---
Well if you run vacum on a Postgres DB then all that goes to the
crapper...  So we resort to real time  backups or replication.  Of which
with replication on a postgres db with the 2GB Blobs being inserted
there is going to be a problem... even if you have enough shared memory
configured.  The thing is regardless you are never in RT Replication.

John

___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] Backup solution to backup /var/spool/imap above 150GB data

[Les Mikesell]

Karanbir Singh wrote:
> On 02/24/2010 07:44 PM, Les Mikesell wrote:
>>> Err.. raid is NOT backup solution.
>> Neither is a snapshot in another location on the same machine.
> 
> Thats not true, raid is an online setup - different location could be 
> point in time, and on blockdev;s that dont share user access load. Which 
> in turn makes it easier to do intensive complete backups to offsite 
> without impacting user level of service the machine can deliver, amongst 
> other things[1].
> 
> Dont compare apples to banana's and call them oranges.

Yes and no... There's an overlapping set of possibilities that they do and 
don't 
back up.  They both cover single disk failures.  They don't cover big operator 
errors (rm -rf /), building/site disasters, some types of controller/electrical 
issues, etc.  The snapshots give you a short history that can help with small 
user/operator errors at the expense of being out of date when the live disk 
fails. So have several types of fruit to stay healthy.

-- 
   Les Mikesell
lesmikes...@gmail.com
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] Backup solution to backup /var/spool/imap above 150GB data

[Karanbir Singh]

On 02/24/2010 07:44 PM, Les Mikesell wrote:
>> Err.. raid is NOT backup solution.
>
> Neither is a snapshot in another location on the same machine.

Thats not true, raid is an online setup - different location could be 
point in time, and on blockdev;s that dont share user access load. Which 
in turn makes it easier to do intensive complete backups to offsite 
without impacting user level of service the machine can deliver, amongst 
other things[1].

Dont compare apples to banana's and call them oranges.

- KB

[1] Changeset and data/system model over time relation mapping for an 
adaptive system sizing feedback loop! ( how'se that for buzzword 
injection! )

___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] Backup solution to backup /var/spool/imap above 150GB data

[Gavin Carr]

Have you read Brackup::Manual::Overview? Your questions are all answered in 
the man pages there or linked from there.

On Thu, Feb 25, 2010 at 05:21:47PM +0530, Agnello George wrote:
> On Wed, Feb 24, 2010 at 7:05 PM, Gavin Carr  wrote:
> > Brackup will backup to local disk, or remotely to ftp, sftp, Amazon S3, or
> > Rackspace CloudFiles targets/servers. So yes, on a lan you can backup over
> > ftp or sftp just fine.
> >
> > Re docs, install brackup, 'man Brackup::Manual::Overview'. I've also
> > written
> > a few blog posts on it: http://www.openfusion.net/tags/brackup.
> >
> I am trying to install the  brackup app on my system, the documentations
> seems very helpful ( http://www.openfusion.net/net/fun_with_brackup)
> But i have a  few  queries  with the config file :
> [TARGET:backups]
> type = Filesystem
> path = /backup
> 
> [SOURCE:imapsource]
> path = /var/spool/imap
> chunk_size = 5m   # what does this mean 
> gpg_recipient = 5E1B3EC5 # what does this mean 

man Brackup::Manual::Overview; man Brackup::Root

> [SOURCE:bradhome]
> chunk_size = 64MB
> path = /raid/bradfitz/
> ignore = ^\.thumbnails/
> ignore = ^\.kde/share/thumbnails/
> ignore = ^\.ee/minis/
> ignore = ^build/
> ignore = ^(gqview|nautilus)/thumbnails/
> 
> 
> and suppose i want to backup it up to another server with scp / ssh how
> is this attatined .

man Brackup::Target::Sftp

> secondly in whant format is the backup maintained .

Backups are trees of file chunks, and a metadata file to put the chunks back 
together as files. So you get de-duplication for free between files and
across backups.

Cheers,
Gavin

___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] Backup solution to backup /var/spool/imap above 150GB data

[Agnello George]

On Wed, Feb 24, 2010 at 7:05 PM, Gavin Carr  wrote:

> On Wed, Feb 24, 2010 at 05:06:08PM +0530, Agnello George wrote:
> > On Wed, Feb 24, 2010 at 4:57 PM, Eero Volotinen  >wrote:
> > > 2010/2/24 Agnello George :
> > > > On Wed, Feb 24, 2010 at 4:13 PM, Gavin Carr  >
> > > wrote:
> > > >> You might want to try brackup (http://code.google.com/p/brackup/).
> For
> > > >> very
> > > >> large trees of relatively small files it seems to significantly
> > > >> out-perform
> > > >> rsync-based backups. I've got brackup packages in my repository (see
> > > >> http://www.openfusion.net/linux/openfusion_rpm_repository).
> > > >>
> > > >> Cheers,
> > > >> Gavin
> > > >>
> > > > is it possible with " brackup "  to back it up to a different server
> on
> > > > the same lan   instead of /backup  . Is there any documentation on
> the
> > > > same .
> > >
> > > rsync or rdiff-backup works on local disk or remote disk.(and other
> > > backup methods too!)
> >
> > Does http://code.google.com/p/brackup/  also work in on remote machines
> .
>
> Brackup will backup to local disk, or remotely to ftp, sftp, Amazon S3, or
> Rackspace CloudFiles targets/servers. So yes, on a lan you can backup over
> ftp or sftp just fine.
>
> Re docs, install brackup, 'man Brackup::Manual::Overview'. I've also
> written
> a few blog posts on it: http://www.openfusion.net/tags/brackup.
>
> Cheers,
> Gavin
>
> I am trying to install the  brackup app on my system, the documentations
seems very helpful ( http://www.openfusion.net/net/fun_with_brackup)
But i have a  few  queries  with the config file :
[TARGET:backups]
type = Filesystem
path = /backup

[SOURCE:imapsource]
path = /var/spool/imap
chunk_size = 5m   # what does this mean 
gpg_recipient = 5E1B3EC5 # what does this mean 

[SOURCE:bradhome]
chunk_size = 64MB
path = /raid/bradfitz/
ignore = ^\.thumbnails/
ignore = ^\.kde/share/thumbnails/
ignore = ^\.ee/minis/
ignore = ^build/
ignore = ^(gqview|nautilus)/thumbnails/


and suppose i want to backup it up to another server with scp / ssh how
is this attatined .
secondly in whant format is the backup maintained .

-- 
Regards
Agnello D'souza
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] Backup solution to backup /var/spool/imap above 150GB data

[Agnello George]

On Thu, Feb 25, 2010 at 1:19 AM, Mike McCarty wrote:

> Eero Volotinen wrote:
> > 2010/2/24 Mike McCarty :
> >> Agnello George wrote:
> >>> Hi
> >>>
> >>> We have  an issue with one of our clients , they have a mail server
> with
> >>> the  /var/spool/imap partition as 150 GB . They need to take
> differential
> >>> backup on to /backup partition ( a different HDD  of total 250 GB space
> )  .
> >> You've stated things in terms of solutions. You may possibly get better
> >> answers if you state your goal. There is some capability you are
> >> trying to achieve. Tell us what that is, and you may make more progress.
> >>
> >> IOW, what is the purpose of the backup? As one mentioned, RAID may
> >> handle your needs.
> >
> > Err.. raid is NOT backup solution.
>
> Of course not. RAID is a means to achieve availability,
> which may be his goal. Karanbir already stated a means to do
> what he seemed to want, but it seemed not to satisfy his needs.
>
> Unless the query is placed in terms of requirements and goals,
> instead of solutions, it'll be difficult to achieve satisfactory
> results.
>
> The purpose of backup is some degree of disaster recovery, and
> perhaps also migration. If that's truly his goal, then ISTM
> that Karanbir suggested a viable solution to achieving avialability
> while also performing backup, by doing on-the-fly duplication
> of the data onto another file system which can then be backed up
> at liesure.
>
> Doing so in a manner which ensures a true snapshot may be more
> difficult to achieve, while still ensuring availability. I normally
> do my backups in single user mode with all file systems mounted read
> only, except the one to receive the backup. That of course precludes
> availability during the backup procedure.
>
> That's why I would like to see what he actually wants to achieve,
> instead of how he has chosen to go about it.
>
> Mike
> --
>

The requirement fro backup  is not  primarily  for HDD failure , but human
error failure . In case one of our user ( eg: the COO with huge  mailbox
size has delete all his certain very important mails, and he want to recover
them , the contacts us as we are supposed to maintain his mail backup for a
week, and we should restore his backup immediately  )  this the main
requirement  for the backup  and that too on the same server different
partition .

-- 
Regards
Agnello D'souza
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] Backup solution to backup /var/spool/imap above 150GB data

[Mike McCarty]

Eero Volotinen wrote:
> 2010/2/24 Mike McCarty :
>> Agnello George wrote:
>>> Hi
>>>
>>> We have  an issue with one of our clients , they have a mail server with
>>> the  /var/spool/imap partition as 150 GB . They need to take differential
>>> backup on to /backup partition ( a different HDD  of total 250 GB space )  .
>> You've stated things in terms of solutions. You may possibly get better
>> answers if you state your goal. There is some capability you are
>> trying to achieve. Tell us what that is, and you may make more progress.
>>
>> IOW, what is the purpose of the backup? As one mentioned, RAID may
>> handle your needs.
> 
> Err.. raid is NOT backup solution.

Of course not. RAID is a means to achieve availability,
which may be his goal. Karanbir already stated a means to do
what he seemed to want, but it seemed not to satisfy his needs.

Unless the query is placed in terms of requirements and goals,
instead of solutions, it'll be difficult to achieve satisfactory
results.

The purpose of backup is some degree of disaster recovery, and
perhaps also migration. If that's truly his goal, then ISTM
that Karanbir suggested a viable solution to achieving avialability
while also performing backup, by doing on-the-fly duplication
of the data onto another file system which can then be backed up
at liesure.

Doing so in a manner which ensures a true snapshot may be more
difficult to achieve, while still ensuring availability. I normally
do my backups in single user mode with all file systems mounted read
only, except the one to receive the backup. That of course precludes
availability during the backup procedure.

That's why I would like to see what he actually wants to achieve,
instead of how he has chosen to go about it.

Mike
-- 
p="p=%c%s%c;main(){printf(p,34,p,34);}";main(){printf(p,34,p,34);}
Oppose globalization and One World Governments like the UN.
This message made from 100% recycled bits.
You have found the bank of Larn.
I speak only for myself, and I am unanimous in that!
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] Backup solution to backup /var/spool/imap above 150GB data

[Les Mikesell]

On 2/24/2010 1:31 PM, Eero Volotinen wrote:

>>>
>>> We have  an issue with one of our clients , they have a mail server with
>>> the  /var/spool/imap partition as 150 GB . They need to take differential
>>> backup on to /backup partition ( a different HDD  of total 250 GB space )  .
>>
>> You've stated things in terms of solutions. You may possibly get better
>> answers if you state your goal. There is some capability you are
>> trying to achieve. Tell us what that is, and you may make more progress.
>>
>> IOW, what is the purpose of the backup? As one mentioned, RAID may
>> handle your needs.
>
> Err.. raid is NOT backup solution.

Neither is a snapshot in another location on the same machine.  But both 
will cover the most likely thing to fail, with raid doing it 
transparently, the snapshot losing data from the time the last snapshot 
copy happened.  Usually what you want is raid _and_ a history of backups 
kept elsewhere.

-- 
   Les Mikesell
lesmikes...@gmail.com
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] Backup solution to backup /var/spool/imap above 150GB data

[Eero Volotinen]

2010/2/24 Mike McCarty :
> Agnello George wrote:
>> Hi
>>
>> We have  an issue with one of our clients , they have a mail server with
>> the  /var/spool/imap partition as 150 GB . They need to take differential
>> backup on to /backup partition ( a different HDD  of total 250 GB space )  .
>
> You've stated things in terms of solutions. You may possibly get better
> answers if you state your goal. There is some capability you are
> trying to achieve. Tell us what that is, and you may make more progress.
>
> IOW, what is the purpose of the backup? As one mentioned, RAID may
> handle your needs.

Err.. raid is NOT backup solution.

--
Eero
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] Backup solution to backup /var/spool/imap above 150GB data

[Mike McCarty]

Agnello George wrote:
> Hi
> 
> We have  an issue with one of our clients , they have a mail server with
> the  /var/spool/imap partition as 150 GB . They need to take differential
> backup on to /backup partition ( a different HDD  of total 250 GB space )  .

You've stated things in terms of solutions. You may possibly get better
answers if you state your goal. There is some capability you are
trying to achieve. Tell us what that is, and you may make more progress.

IOW, what is the purpose of the backup? As one mentioned, RAID may
handle your needs.

Mike
-- 
p="p=%c%s%c;main(){printf(p,34,p,34);}";main(){printf(p,34,p,34);}
Oppose globalization and One World Governments like the UN.
This message made from 100% recycled bits.
You have found the bank of Larn.
I speak only for myself, and I am unanimous in that!
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] Backup solution to backup /var/spool/imap above 150GB data

[Karanbir Singh]

On 02/24/2010 01:07 PM, Agnello George wrote:
> yes just spoke to my senior and confrimed that this was alreday tried
> out a delayed replication  is possible .
> but the current suitation is we need to  take backup  on  the same
> server on a different partition /backup   :(

you can replicate to a local mail store as well. just make sure you put 
it on a block device that is suiteable and fits in with the rest of your 
backup strategy. And if you put it in an isolated enough place on the 
block dev, it wont contest with the users access.

- KB
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] Backup solution to backup /var/spool/imap above 150GB data

[Les Mikesell]

Agnello George wrote:
> Hi
> 
> We have  an issue with one of our clients , they have a mail server with 
> the  /var/spool/imap partition as 150 GB . They need to take 
> differential  backup on to /backup partition ( a different HDD  of total 
> 250 GB space )  . We have tried dar , rsync, rdiff and impasync . But 
> its is not sufficing the need as to take a lot of time and consumes a 
> lot of I/O .
> 
> Is there any back up solution that you can think of , that can work in 
> this situation  - open source or proprietary 

If you are just concerned about a single disk failure you could set up RAID1 on 
the disks (with some downtime to rebuild...) to keep the copy in realtime with 
little loss of speed.

Rsync should work as well as anything for snapshots but you might need to 
update 
to a 3.x version to speed up handling large numbers of files.  The 2.x version 
included in Centos will read the entire directory tree into memory before 
starting the comparisons and copies.  The rpmforge repo has a packaged 3.0.7 
version but I haven't tried it.

-- 
   Les Mikesell
lesmikes...@gmail.com
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] Backup solution to backup /var/spool/imap above 150GB data

[Agnello George]

On Wed, Feb 24, 2010 at 7:05 PM, Gavin Carr  wrote:

> On Wed, Feb 24, 2010 at 05:06:08PM +0530, Agnello George wrote:
> > On Wed, Feb 24, 2010 at 4:57 PM, Eero Volotinen  >wrote:
> > > 2010/2/24 Agnello George :
> > > > On Wed, Feb 24, 2010 at 4:13 PM, Gavin Carr  >
> > > wrote:
> > > >> You might want to try brackup (http://code.google.com/p/brackup/).
> For
> > > >> very
> > > >> large trees of relatively small files it seems to significantly
> > > >> out-perform
> > > >> rsync-based backups. I've got brackup packages in my repository (see
> > > >> http://www.openfusion.net/linux/openfusion_rpm_repository).
> > > >>
> > > >> Cheers,
> > > >> Gavin
> > > >>
> > > > is it possible with " brackup "  to back it up to a different server
> on
> > > > the same lan   instead of /backup  . Is there any documentation on
> the
> > > > same .
> > >
> > > rsync or rdiff-backup works on local disk or remote disk.(and other
> > > backup methods too!)
> >
> > Does http://code.google.com/p/brackup/  also work in on remote machines
> .
>
> Brackup will backup to local disk, or remotely to ftp, sftp, Amazon S3, or
> Rackspace CloudFiles targets/servers. So yes, on a lan you can backup over
> ftp or sftp just fine.
>
> Re docs, install brackup, 'man Brackup::Manual::Overview'. I've also
> written
> a few blog posts on it: http://www.openfusion.net/tags/brackup.
>
> Cheers,
> Gavin
>
> ___
> CentOS mailing list
> CentOS@centos.org
> http://lists.centos.org/mailman/listinfo/centos
>

it will take me some time to try this .. will get back on its output !! ..
thanks

-- 
Regards
Agnello D'souza
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] Backup solution to backup /var/spool/imap above 150GB data

[Gavin Carr]

On Wed, Feb 24, 2010 at 05:06:08PM +0530, Agnello George wrote:
> On Wed, Feb 24, 2010 at 4:57 PM, Eero Volotinen wrote:
> > 2010/2/24 Agnello George :
> > > On Wed, Feb 24, 2010 at 4:13 PM, Gavin Carr 
> > wrote:
> > >> You might want to try brackup (http://code.google.com/p/brackup/). For
> > >> very
> > >> large trees of relatively small files it seems to significantly
> > >> out-perform
> > >> rsync-based backups. I've got brackup packages in my repository (see
> > >> http://www.openfusion.net/linux/openfusion_rpm_repository).
> > >>
> > >> Cheers,
> > >> Gavin
> > >>
> > > is it possible with " brackup "  to back it up to a different server on
> > > the same lan   instead of /backup  . Is there any documentation on the 
> > > same .
> >
> > rsync or rdiff-backup works on local disk or remote disk.(and other
> > backup methods too!)
> 
> Does http://code.google.com/p/brackup/  also work in on remote machines .

Brackup will backup to local disk, or remotely to ftp, sftp, Amazon S3, or
Rackspace CloudFiles targets/servers. So yes, on a lan you can backup over 
ftp or sftp just fine.

Re docs, install brackup, 'man Brackup::Manual::Overview'. I've also written
a few blog posts on it: http://www.openfusion.net/tags/brackup.

Cheers,
Gavin

___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] Backup solution to backup /var/spool/imap above 150GB data

[Agnello George]

On Wed, Feb 24, 2010 at 6:19 PM, Karanbir Singh wrote:

> On 02/24/2010 11:21 AM, Agnello George wrote:
> > -rw--- 1 cyrus mail 4805500 Feb 24 16:23 4799.
> > -rw--- 1 cyrus mail 22920 Feb 24 16:23 cyrus.cache
> > -rw--- 1 cyrus mail 204 Dec 10 16:27 cyrus.header
> > -rw--- 1 cyrus mail 896 Feb 24 16:23 cyrus.index
> > -rw--- 1 cyrus mail 8669 Feb 24 11:28 cyrus.squat
> >
> > this is Just a very small user and a example
>
> About 90% of your problem is already solved here, you are using cyrus
> which has built in mail level replication. All you need to do is setup a
> lvm volume away from this main store and run your mail replica over to
> it. then just backup using whatever tools you want.
>
> Free win you get is online failover, backup in whatever manner you want!
>
yes just spoke to my senior and confrimed that this was alreday tried out a
delayed replication  is possible .
but the current suitation is we need to  take backup  on  the same server on
a different partition /backup   :(



-- 
Regards
Agnello D'souza
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] Backup solution to backup /var/spool/imap above 150GB data

[Karanbir Singh]

On 02/24/2010 11:21 AM, Agnello George wrote:
> -rw--- 1 cyrus mail 4805500 Feb 24 16:23 4799.
> -rw--- 1 cyrus mail 22920 Feb 24 16:23 cyrus.cache
> -rw--- 1 cyrus mail 204 Dec 10 16:27 cyrus.header
> -rw--- 1 cyrus mail 896 Feb 24 16:23 cyrus.index
> -rw--- 1 cyrus mail 8669 Feb 24 11:28 cyrus.squat
>
> this is Just a very small user and a example

About 90% of your problem is already solved here, you are using cyrus 
which has built in mail level replication. All you need to do is setup a 
lvm volume away from this main store and run your mail replica over to 
it. then just backup using whatever tools you want.

Free win you get is online failover, backup in whatever manner you want!

- KB
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] Backup solution to backup /var/spool/imap above 150GB data

[Eero Volotinen]

2010/2/24 Agnello George :
>
>
> On Wed, Feb 24, 2010 at 4:57 PM, Eero Volotinen 
> wrote:
>>
>> 2010/2/24 Agnello George :
>> >
>> >
>> > On Wed, Feb 24, 2010 at 4:13 PM, Gavin Carr 
>> > wrote:
>> >>
>> >> On Wed, Feb 24, 2010 at 10:38:32AM +0100, David Hrbáč wrote:
>> >> > Dne 24.2.2010 10:00, Agnello George napsal(a):
>> >> > > We have  an issue with one of our clients , they have a mail server
>> >> > > with
>> >> > > the  /var/spool/imap partition as 150 GB . They need to take
>> >> > > differential
>> >> > > backup on to /backup partition ( a different HDD  of total 250 GB
>> >> > > space )  .
>> >> > > We have tried dar , rsync, rdiff and impasync . But its is not
>> >> > > sufficing the
>> >> > > need as to take a lot of time and consumes a lot of I/O .
>> >> > >
>> >> > > Is there any back up solution that you can think of , that can work
>> >> > > in
>> >> > > this
>> >> > > situation  - open source or proprietary
>> >> >
>> >> > Is seems to me, that you are using mbox format. So, differential
>> >> > backup
>> >> > is hard to achieve. Migrate to maildir, every mail is a file, easy to
>> >> > backup differentially.
>> >>
>> >> rsync and rdiff should handle mbox format okay though. Though I agree
>> >> Maildir
>> >> is generally nicer for differential backups.
>> >>
>> >> Agnello, how long is "a lot of time"? A backup is always going to have
>> >> to
>> >> walk
>> >> the entire tree and checksum (or at least stat) every file, so there's
>> >> a
>> >> minimum
>> >> cost you're always going to have. How long does a 'find /var/spool/imap
>> >> -ls'
>> >> take, for instance?
>> >>
>> >> You might want to try brackup (http://code.google.com/p/brackup/). For
>> >> very
>> >> large trees of relatively small files it seems to significantly
>> >> out-perform
>> >> rsync-based backups. I've got brackup packages in my repository (see
>> >> http://www.openfusion.net/linux/openfusion_rpm_repository).
>> >>
>> >> Cheers,
>> >> Gavin
>> >>
>> > is it possible with " brackup "  to back it up to a different server on
>> > the
>> > same lan   instead of /backup  . Is there any documentation on the same
>> > .
>>
>> rsync or rdiff-backup works on local disk or remote disk.(and other
>> backup methods too!)


Summary from webpage: "Flexible backup tool. Slices, dices, encrypts,
and sprays across the net, notably to Amazon's S3. "


--
Eero
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] Backup solution to backup /var/spool/imap above 150GB data

[Agnello George]

On Wed, Feb 24, 2010 at 4:57 PM, Eero Volotinen wrote:

> 2010/2/24 Agnello George :
> >
> >
> > On Wed, Feb 24, 2010 at 4:13 PM, Gavin Carr 
> wrote:
> >>
> >> On Wed, Feb 24, 2010 at 10:38:32AM +0100, David Hrbáč wrote:
> >> > Dne 24.2.2010 10:00, Agnello George napsal(a):
> >> > > We have  an issue with one of our clients , they have a mail server
> >> > > with
> >> > > the  /var/spool/imap partition as 150 GB . They need to take
> >> > > differential
> >> > > backup on to /backup partition ( a different HDD  of total 250 GB
> >> > > space )  .
> >> > > We have tried dar , rsync, rdiff and impasync . But its is not
> >> > > sufficing the
> >> > > need as to take a lot of time and consumes a lot of I/O .
> >> > >
> >> > > Is there any back up solution that you can think of , that can work
> in
> >> > > this
> >> > > situation  - open source or proprietary
> >> >
> >> > Is seems to me, that you are using mbox format. So, differential
> backup
> >> > is hard to achieve. Migrate to maildir, every mail is a file, easy to
> >> > backup differentially.
> >>
> >> rsync and rdiff should handle mbox format okay though. Though I agree
> >> Maildir
> >> is generally nicer for differential backups.
> >>
> >> Agnello, how long is "a lot of time"? A backup is always going to have
> to
> >> walk
> >> the entire tree and checksum (or at least stat) every file, so there's a
> >> minimum
> >> cost you're always going to have. How long does a 'find /var/spool/imap
> >> -ls'
> >> take, for instance?
> >>
> >> You might want to try brackup (http://code.google.com/p/brackup/). For
> >> very
> >> large trees of relatively small files it seems to significantly
> >> out-perform
> >> rsync-based backups. I've got brackup packages in my repository (see
> >> http://www.openfusion.net/linux/openfusion_rpm_repository).
> >>
> >> Cheers,
> >> Gavin
> >>
> > is it possible with " brackup "  to back it up to a different server on
> the
> > same lan   instead of /backup  . Is there any documentation on the same .
>
> rsync or rdiff-backup works on local disk or remote disk.(and other
> backup methods too!)
>

Does http://code.google.com/p/brackup/  also work in on remote machines .
-- 
Regards
Agnello D'souza
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] Backup solution to backup /var/spool/imap above 150GB data

[Eero Volotinen]

2010/2/24 Agnello George :
>
>
> On Wed, Feb 24, 2010 at 4:13 PM, Gavin Carr  wrote:
>>
>> On Wed, Feb 24, 2010 at 10:38:32AM +0100, David Hrbáč wrote:
>> > Dne 24.2.2010 10:00, Agnello George napsal(a):
>> > > We have  an issue with one of our clients , they have a mail server
>> > > with
>> > > the  /var/spool/imap partition as 150 GB . They need to take
>> > > differential
>> > > backup on to /backup partition ( a different HDD  of total 250 GB
>> > > space )  .
>> > > We have tried dar , rsync, rdiff and impasync . But its is not
>> > > sufficing the
>> > > need as to take a lot of time and consumes a lot of I/O .
>> > >
>> > > Is there any back up solution that you can think of , that can work in
>> > > this
>> > > situation  - open source or proprietary
>> >
>> > Is seems to me, that you are using mbox format. So, differential backup
>> > is hard to achieve. Migrate to maildir, every mail is a file, easy to
>> > backup differentially.
>>
>> rsync and rdiff should handle mbox format okay though. Though I agree
>> Maildir
>> is generally nicer for differential backups.
>>
>> Agnello, how long is "a lot of time"? A backup is always going to have to
>> walk
>> the entire tree and checksum (or at least stat) every file, so there's a
>> minimum
>> cost you're always going to have. How long does a 'find /var/spool/imap
>> -ls'
>> take, for instance?
>>
>> You might want to try brackup (http://code.google.com/p/brackup/). For
>> very
>> large trees of relatively small files it seems to significantly
>> out-perform
>> rsync-based backups. I've got brackup packages in my repository (see
>> http://www.openfusion.net/linux/openfusion_rpm_repository).
>>
>> Cheers,
>> Gavin
>>
> is it possible with " brackup "  to back it up to a different server on the
> same lan   instead of /backup  . Is there any documentation on the same .

rsync or rdiff-backup works on local disk or remote disk.(and other
backup methods too!)

--
Eero
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] Backup solution to backup /var/spool/imap above 150GB data

[Agnello George]

On Wed, Feb 24, 2010 at 4:13 PM, Gavin Carr  wrote:

> On Wed, Feb 24, 2010 at 10:38:32AM +0100, David Hrbáč wrote:
> > Dne 24.2.2010 10:00, Agnello George napsal(a):
> > > We have  an issue with one of our clients , they have a mail server
> with
> > > the  /var/spool/imap partition as 150 GB . They need to take
> differential
> > > backup on to /backup partition ( a different HDD  of total 250 GB space
> )  .
> > > We have tried dar , rsync, rdiff and impasync . But its is not
> sufficing the
> > > need as to take a lot of time and consumes a lot of I/O .
> > >
> > > Is there any back up solution that you can think of , that can work in
> this
> > > situation  - open source or proprietary
> >
> > Is seems to me, that you are using mbox format. So, differential backup
> > is hard to achieve. Migrate to maildir, every mail is a file, easy to
> > backup differentially.
>
> rsync and rdiff should handle mbox format okay though. Though I agree
> Maildir
> is generally nicer for differential backups.
>
> Agnello, how long is "a lot of time"? A backup is always going to have to
> walk
> the entire tree and checksum (or at least stat) every file, so there's a
> minimum
> cost you're always going to have. How long does a 'find /var/spool/imap
> -ls'
> take, for instance?
>
> You might want to try brackup (http://code.google.com/p/brackup/). For
> very
> large trees of relatively small files it seems to significantly out-perform
> rsync-based backups. I've got brackup packages in my repository (see
> http://www.openfusion.net/linux/openfusion_rpm_repository).
>
> Cheers,
> Gavin
>
>  
>
is it possible with " brackup "   to back
it up to a different server on the same lan   instead of /backup  . Is there
any documentation on the same .


-- 
Regards
Agnello D'souza
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] Backup solution to backup /var/spool/imap above 150GB data

[Agnello George]

2010/2/24 Eero Volotinen 

> 2010/2/24 Agnello George :
> >
> >
> > On Wed, Feb 24, 2010 at 3:08 PM, David Hrbáč 
> wrote:
> >>
> >> Dne 24.2.2010 10:00, Agnello George napsal(a):
> >> > Hi
> >> >
> >> > We have  an issue with one of our clients , they have a mail server
> with
> >> > the  /var/spool/imap partition as 150 GB . They need to take
> >> > differential
> >> > backup on to /backup partition ( a different HDD  of total 250 GB
> space
> >> > )  .
> >> > We have tried dar , rsync, rdiff and impasync . But its is not
> sufficing
> >> > the
> >> > need as to take a lot of time and consumes a lot of I/O .
> >> >
> >> > Is there any back up solution that you can think of , that can work in
> >> > this
> >> > situation  - open source or proprietary
> >>
> >> Is seems to me, that you are using mbox format. So, differential backup
> >> is hard to achieve. Migrate to maildir, every mail is a file, easy to
> >> backup differentially.
> >> David
> >
> > backup directory structure is /var/spool/imap/a
> > /adomain.com/a/agnello^dsouza/
> >
>
> Well, does that directory contains one file or lot of files ?
>
> Usually maildir structure is like dir++/tmp/current/new directories
> and each message is in own file on mailbox all files are inside one
> file.
>
> its in a maildir format and the structure slightly   different from what i
mentioned earlier :
r...@server1 ~]# ls -la /var/spool/imap/a/user/ajay/
total 5180
-rw--- 1 cyrus mail 34616 Feb 24 16:02 4790.
-rw--- 1 cyrus mail 4490 Feb 24 16:03 4791.
-rw--- 1 cyrus mail 199253 Feb 24 16:07 4792.
-rw--- 1 cyrus mail 22930 Feb 24 16:09 4793.
-rw--- 1 cyrus mail 8485 Feb 24 16:11 4794.
-rw--- 1 cyrus mail 12664 Feb 24 16:13 4795.
-rw--- 1 cyrus mail 4296 Feb 24 16:13 4796.
-rw--- 1 cyrus mail 5337 Feb 24 16:15 4797.
-rw--- 1 cyrus mail 111030 Feb 24 16:21 4798.
-rw--- 1 cyrus mail 4805500 Feb 24 16:23 4799.
-rw--- 1 cyrus mail 22920 Feb 24 16:23 cyrus.cache
-rw--- 1 cyrus mail 204 Dec 10 16:27 cyrus.header
-rw--- 1 cyrus mail 896 Feb 24 16:23 cyrus.index
-rw--- 1 cyrus mail 8669 Feb 24 11:28 cyrus.squat

this is Just a very small user and a example



-- 
Regards
Agnello D'souza
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] Backup solution to backup /var/spool/imap above 150GB data

[Gavin Carr]

On Wed, Feb 24, 2010 at 10:38:32AM +0100, David Hrbáč wrote:
> Dne 24.2.2010 10:00, Agnello George napsal(a):
> > We have  an issue with one of our clients , they have a mail server with
> > the  /var/spool/imap partition as 150 GB . They need to take differential
> > backup on to /backup partition ( a different HDD  of total 250 GB space )  .
> > We have tried dar , rsync, rdiff and impasync . But its is not sufficing the
> > need as to take a lot of time and consumes a lot of I/O .
> > 
> > Is there any back up solution that you can think of , that can work in this
> > situation  - open source or proprietary
> 
> Is seems to me, that you are using mbox format. So, differential backup
> is hard to achieve. Migrate to maildir, every mail is a file, easy to
> backup differentially.

rsync and rdiff should handle mbox format okay though. Though I agree Maildir 
is generally nicer for differential backups.

Agnello, how long is "a lot of time"? A backup is always going to have to walk
the entire tree and checksum (or at least stat) every file, so there's a minimum
cost you're always going to have. How long does a 'find /var/spool/imap -ls' 
take, for instance?

You might want to try brackup (http://code.google.com/p/brackup/). For very 
large trees of relatively small files it seems to significantly out-perform 
rsync-based backups. I've got brackup packages in my repository (see 
http://www.openfusion.net/linux/openfusion_rpm_repository).

Cheers,
Gavin

___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] Backup solution to backup /var/spool/imap above 150GB data

[Eero Volotinen]

2010/2/24 Agnello George :
>
>
> On Wed, Feb 24, 2010 at 3:08 PM, David Hrbáč  wrote:
>>
>> Dne 24.2.2010 10:00, Agnello George napsal(a):
>> > Hi
>> >
>> > We have  an issue with one of our clients , they have a mail server with
>> > the  /var/spool/imap partition as 150 GB . They need to take
>> > differential
>> > backup on to /backup partition ( a different HDD  of total 250 GB space
>> > )  .
>> > We have tried dar , rsync, rdiff and impasync . But its is not sufficing
>> > the
>> > need as to take a lot of time and consumes a lot of I/O .
>> >
>> > Is there any back up solution that you can think of , that can work in
>> > this
>> > situation  - open source or proprietary
>>
>> Is seems to me, that you are using mbox format. So, differential backup
>> is hard to achieve. Migrate to maildir, every mail is a file, easy to
>> backup differentially.
>> David
>
> backup directory structure is /var/spool/imap/a
> /adomain.com/a/agnello^dsouza/
>

Well, does that directory contains one file or lot of files ?

Usually maildir structure is like dir++/tmp/current/new directories
and each message is in own file on mailbox all files are inside one
file.

--
Eero
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] Backup solution to backup /var/spool/imap above 150GB data

[Agnello George]

On Wed, Feb 24, 2010 at 3:08 PM, David Hrbáč  wrote:

> Dne 24.2.2010 10:00, Agnello George napsal(a):
> > Hi
> >
> > We have  an issue with one of our clients , they have a mail server with
> > the  /var/spool/imap partition as 150 GB . They need to take differential
> > backup on to /backup partition ( a different HDD  of total 250 GB space )
>  .
> > We have tried dar , rsync, rdiff and impasync . But its is not sufficing
> the
> > need as to take a lot of time and consumes a lot of I/O .
> >
> > Is there any back up solution that you can think of , that can work in
> this
> > situation  - open source or proprietary
>
> Is seems to me, that you are using mbox format. So, differential backup
> is hard to achieve. Migrate to maildir, every mail is a file, easy to
> backup differentially.
> David
>
backup directory structure is /var/spool/imap/a /adomain.com/a/agnello^dsouza/



-- 
Regards
Agnello D'souza
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] Backup solution to backup /var/spool/imap above 150GB data

[David Hrbáč]

Dne 24.2.2010 10:00, Agnello George napsal(a):
> Hi
> 
> We have  an issue with one of our clients , they have a mail server with
> the  /var/spool/imap partition as 150 GB . They need to take differential
> backup on to /backup partition ( a different HDD  of total 250 GB space )  .
> We have tried dar , rsync, rdiff and impasync . But its is not sufficing the
> need as to take a lot of time and consumes a lot of I/O .
> 
> Is there any back up solution that you can think of , that can work in this
> situation  - open source or proprietary

Is seems to me, that you are using mbox format. So, differential backup
is hard to achieve. Migrate to maildir, every mail is a file, easy to
backup differentially.
David
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] Backup solution to backup /var/spool/imap above 150GB data

[Eero Volotinen]

2010/2/24 Agnello George :
> Hi
>
> We have  an issue with one of our clients , they have a mail server with
> the  /var/spool/imap partition as 150 GB . They need to take differential
> backup on to /backup partition ( a different HDD  of total 250 GB space )  .
> We have tried dar , rsync, rdiff and impasync . But its is not sufficing the
> need as to take a lot of time and consumes a lot of I/O .

So, you need to add more disk i/o? (so, add some disk space with faster raid?)

Take a look at: http://rdiff-backup.nongnu.org/

--
Eero
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

[CentOS] Backup solution to backup /var/spool/imap above 150GB data

[Agnello George]

Hi

We have  an issue with one of our clients , they have a mail server with
the  /var/spool/imap partition as 150 GB . They need to take differential
backup on to /backup partition ( a different HDD  of total 250 GB space )  .
We have tried dar , rsync, rdiff and impasync . But its is not sufficing the
need as to take a lot of time and consumes a lot of I/O .

Is there any back up solution that you can think of , that can work in this
situation  - open source or proprietary

-- 
Regards
Agnello D'souza
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos