Holger Parplies wrote at about 04:08:01 +0200 on Saturday, September 12, 2009:
> * *Data within the pool* is *never* modified. Files are created, linked to,
> and deleted. That's it. [Wait, that's wrong. rsync checksums may be added
> later, but they're *appended*, aren't they? Only once any
Hi list,
I could probably quote roughly 1GB from this discussion, or top-post and
append the whole thread for those of you who want to read it again, but I
won't.
I just want to share some thoughts that seem to be missing from this
discussion so far - for whatever use anyone can make of them.
*
Timothy J Massey wrote:
>
>> I'm not convinced that any of that matters when the real issue is moving
>> a physical disk head around.
>
> If that's all you want out of life, then pick whatever. For most people,
> vMotion is a killer app. The only reason I would use a solution that
> *doesn'
Les Mikesell wrote on 09/11/2009 12:14:22 PM:
> Timothy J Massey wrote:
> >
> > So you're attempting to convert a physical BackupPC server into a
virtual
> > image? VMware has conversion tools that do this. I've only used the
> > Windows version of VMware Converter, but it has worked perfec
Timothy J Massey wrote:
>
> So you're attempting to convert a physical BackupPC server into a virtual
> image? VMware has conversion tools that do this. I've only used the
> Windows version of VMware Converter, but it has worked perfectly for
> converting a physical host into a virtual host.
On Fri, Sep 11, 2009 at 04:47:31PM +1000, Adam Goryachev wrote:
> Timothy J Massey wrote:
> > Of course, now we've come full circle: how do you copy a physical
> > block device in an rsync-like manner? :)
> Why not just use lvm to take a snapshot, use dd to take 2G chunks (or
> whatever size you
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Timothy J Massey wrote:
>
> But the ability to rsync collections of VMDK files to a remote host
> *is* appealing. Interesting...
>
> However, you could achieve the same thing in other ways without
> having to run BackupPC in a virtualized guest. You
Les Mikesell wrote on 09/10/2009 10:38:56 PM:
> Maybe I'm not making this clear. I want the physical backuppc host
> to be able
> to mirror its current 750 GB backuppc partition onto what appears at
> the time to
> be a virtual partition, but which results in the updating of the files
that
Les Mikesell wrote at about 13:41:58 -0500 on Thursday, September 10, 2009:
> Timothy J Massey wrote:
> >
> > VMware Sever 2.0 supports disks in single files, or split in
> > arbitrary-sized pieces (usually 2GB), both fully allocated or
> > grow-on-demand. ESX/ESXi only supports single-fil
Timothy J Massey wrote:
>
>> I know there are there in the form of a directory of files which will be
>
>> nice for the rsync step. What I want to know is if the physical host
>> can see the virtual disk/partition the same way a guest can. There is a
>> vmware-mount that I think lets you see t
Les Mikesell wrote on 09/10/2009 05:23:20 PM:
> Timothy J Massey wrote:
> >
> >> I thought there was a way to access the vmx image directly from
linux,
> >> but I don't know if it has to be mounted or if you can see a raw
> >> partition.
> >
> > Depends: are we talking VMware Server, or ESX
Christian Völker wrote on 09/10/2009 05:36:27 PM:
> Thanks for the great mail. It subsummed really nearly everything which
> is related to all this.
Thank you. I'm glad it helped.
> Just one point where you seem to be wrong:
> You mentioned "VMware tools" (don't get confused with the "official
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Hi Tim,
> Anyway, the idea of trying to automatically work with VMware images at the
> block level without VMware's tools is not that great.
True.
Thanks for the great mail. It subsummed really nearly everything which
is related to all this.
Just o
Timothy J Massey wrote:
>
>> I thought there was a way to access the vmx image directly from linux,
>> but I don't know if it has to be mounted or if you can see a raw
>> partition.
>
> Depends: are we talking VMware Server, or ESX/ESXi? And if we're talking
> ESXi, are we talking SAN, NAS o
Les Mikesell wrote on 09/10/2009 02:41:58 PM:
> Timothy J Massey wrote:
> >
> > VMware Sever 2.0 supports disks in single files, or split in
> > arbitrary-sized pieces (usually 2GB), both fully allocated or
> > grow-on-demand. ESX/ESXi only supports single-file images, and it is
> > designed
Timothy J Massey wrote:
>
> VMware Sever 2.0 supports disks in single files, or split in
> arbitrary-sized pieces (usually 2GB), both fully allocated or
> grow-on-demand. ESX/ESXi only supports single-file images, and it is
> designed to use VMFS.
I thought there was a way to access the vmx
Les Mikesell wrote on 09/10/2009 12:02:15 PM:
> > And I have another idea 'coz of rsync copying the whole .vdmk file
over-
> > this could be related to access time. Obviously the times changes as
the
> > file is under continously access from the ESX host. I'll see if I can
> > skip the time ide
Carl Wilhelm Soderstrom wrote on 09/10/2009
12:42:33 PM:
> On 09/10 11:02 , Les Mikesell wrote:
> > You'll almost certainly end up with an unusable copy if you let the VM
> > run while copying
>
>
> To test this, some years ago I actually tried archiving and restoring a
> vmware disk image w
On 09/10 11:02 , Les Mikesell wrote:
> You'll almost certainly end up with an unusable copy if you let the VM
> run while copying
To test this, some years ago I actually tried archiving and restoring a
vmware disk image while it was in use. Turns out it worked just fine.
That said, the vmware se
Christian Völker wrote:
>
>> Also you might be able to use vmware's snapshots to see how much really
>> is changing.
> I did this.
> I took a VMware snapshot yest evening. And it has been grown within
> 12hrs now aprox 5GB. I'll do some further monitoring, but it seem
> indeed, there's not so many
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Hi,
> Also you might be able to use vmware's snapshots to see how much really
> is changing.
I did this.
I took a VMware snapshot yest evening. And it has been grown within
12hrs now aprox 5GB. I'll do some further monitoring, but it seem
indeed, ther
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Hi,
>> Third try was to use "dump" for this hoping it would transfer only
>> changed blocks after the initial dump. No way. After one day it
>> transferred 300GB (!). I thought, dump might not bee a good solution...
> I suspect dump sees the link coun
Christian Völker wrote:
>
> First, my environment:
> 28 hosts to back up. Mostly idle machines with minor services (so no big
> databases and so on). Partially fileserver with only little daily
> changes. So I expected not too much daily changes on the pool.
> I want to copy the pool to a remote lo
On Wed, Sep 09, 2009 at 09:33:03AM +0200, Christian Völker wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> Hi,
>
> as I have the same issue with storing my BackupPC outside I tried
> another way the last days:
>
> First, my environment:
> 28 hosts to back up. Mostly idle machines wi
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Hi,
> I'd say: Replace that USB 2.0 disk by something else like something
> connected via Firewire or eSATA. USB 2.0 is very, very slow, especially
> for random access.
I know, but that's not the point here. Speed doesn't concern me- the
copy should l
Hi Christian,
On Wed, Sep 09, 2009 at 09:33:03AM +0200, Christian Völker wrote:
> First, my environment:
> 28 hosts to back up. Mostly idle machines with minor services (so no big
> databases and so on). Partially fileserver with only little daily
> changes. So I expected not too much daily chang
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Hi,
as I have the same issue with storing my BackupPC outside I tried
another way the last days:
First, my environment:
28 hosts to back up. Mostly idle machines with minor services (so no big
databases and so on). Partially fileserver with only litt
On Sat, Sep 05, 2009 at 06:35:16PM -0600, dan wrote:
[...]
> Thinking about the logistics in the method I have thought up a few hurdles.
> The source disks must remain unchanged during the entire sync.
> You would need to either have a spare disk in a raid1 mirror that you
> could remove from
On Sat, Sep 5, 2009 at 10:25 AM, Les Mikesell wrote:
> dan wrote:
> > because bittorrent stores the file list in a file and bittorrent clients
> > use an index for downloaded bits. rsync stores the filelist in ram.
>
> But how good is bittorrent at finding arbitrarily small differences between
>
dan wrote:
> because bittorrent stores the file list in a file and bittorrent clients
> use an index for downloaded bits. rsync stores the filelist in ram.
But how good is bittorrent at finding arbitrarily small differences between the
old/new copies and resynchronizing on the matches?
--
because bittorrent stores the file list in a file and bittorrent clients use
an index for downloaded bits. rsync stores the filelist in ram.
Also, there is a patch out there for bittorrent (very easy to apply) that
allows you to make a torrent of a block device. rsync wont do this.
One more benn
On Thu, Sep 03, 2009 at 11:35:50AM -0500, Les Mikesell wrote:
> Pieter Wuille wrote:
> >
> >>> You're very right, and i thought about it too. Instead of using a RAID1 on
> >>> the offsite backup, there are two separate backups on the offsite machine,
> >>> and synchronisation switches between them
Pieter Wuille wrote:
>
>>> You're very right, and i thought about it too. Instead of using a RAID1 on
>>> the offsite backup, there are two separate backups on the offsite machine,
>>> and synchronisation switches between them. This also enables the use of
>>> rsync's --inplace option.
>> That sho
On Wed, Sep 02, 2009 at 01:08:27PM -0500, Les Mikesell wrote:
> Pieter Wuille wrote:
> > You're very right, and i thought about it too. Instead of using a RAID1 on
> > the offsite backup, there are two separate backups on the offsite machine,
> > and synchronisation switches between them. This also
On Wed, Sep 02, 2009 at 02:18:41PM -0600, dan wrote:
> Can I offer an alternative solution? How about using bittorrent?
I don't see the benefits over using the patched rsync... What am I
missing? After all it's still read-all-blocks - compare checksums -
transfer changes, right?
Tino.
--
"Wha
Can I offer an alternative solution? How about using bittorrent?
if you patch the btmakemeta and download.py files as show here:
http://osdir.com/ml/network.bit-torrent.general/2003-12/msg00356.html
(stop backuppc, unmount filesystem)
you can create a torrent of your block device
btmakemeta /dev
Les Mikesell wrote:
> a VMware .vmx image file using the options to pre-allocate the space and
> segment into chunks as an intermediate that would be directly usable by
> a vmware guest.
There is a solution for VMware vSphere (ESX/VC 4.0) which would be
perfect. VMware Data Recovery claims to b
Pieter Wuille wrote:
> >> The one thing that would bother me about this approach is that you would
>> have a fairly long window of time while the remote filesystem chunks are
>> being updated. While rsync normally creates a copy of an individual
>> file and does not delete the original until th
On Wed, Sep 02, 2009 at 10:14:05AM -0500, Les Mikesell wrote:
> Pieter Wuille wrote:
> > In our case, the BackupPC pool is stored on an XFS filesystem on an LVM
> > volume, allowing a xfsfreeze/sync/snapshot/xfsunfreeze, and using
> > devfiles.pl on the snapshot. Instead of xfsfreeze+unfreeze, a ba
Les Mikesell wrote at about 10:14:05 -0500 on Wednesday, September 2, 2009:
> Pieter Wuille wrote:
> >
> > To overcome this issue, i wrote a perl/fuse filesystem that allows you to
> > "mount" a block device (or real file) as a directory containing files
> > part0001.img, part0002.img, ... eac
Pieter Wuille wrote:
>
> To overcome this issue, i wrote a perl/fuse filesystem that allows you to
> "mount" a block device (or real file) as a directory containing files
> part0001.img, part0002.img, ... each representing 1 GiB of data of the
> original device:
>
> https://svn.ulyssis.org/repos
On Wed, Sep 2, 2009 at 7:56 AM, Daniel
Berteaud wrote:
> Le mercredi 02 septembre 2009 à 12:10 +0200, Pieter Wuille a écrit :
>> Hello everyone,
>>
>> trying to come up with a way for efficiently synchronising a BackupPC archive
>> on one server with a remote and encrypted offsite backup, the follo
Le mercredi 02 septembre 2009 à 12:10 +0200, Pieter Wuille a écrit :
> Hello everyone,
>
> trying to come up with a way for efficiently synchronising a BackupPC archive
> on one server with a remote and encrypted offsite backup, the following
> problems
> arise:
> * As often pointed out on this l
Hello everyone,
trying to come up with a way for efficiently synchronising a BackupPC archive
on one server with a remote and encrypted offsite backup, the following problems
arise:
* As often pointed out on this list, filesystem-level synchronisation is
extremely cpu and memory-intensive. Not a
44 matches
Mail list logo
