Re: reliable, dd over simple ip network
You know ssh will compress what goes through its tunnel to begin with, right? So, you can eliminate at least one command there.. On 10/17/08, Girish Venkatachalam [EMAIL PROTECTED] wrote: On 17:29:56 Oct 17, Mike wrote: will work out much faster and better than plain old dd(1). On the other side you have to run # input | restore xf - -Girish whats the input going to be? Sorry I was wrong. It was meant to be done in one step from the dump side. This works for me. # dump af - /dev/rwd0d | gzip -c - | ssh hostname gzip -d -| restore rf - Hope it works out for you. Thanks. -Girish
Re: reliable, dd over simple ip network
On Fri, Oct 17, 2008 at 10:52 PM, Johan Beisser [EMAIL PROTECTED] wrote: You know ssh will compress what goes through its tunnel to begin with, right? ssh_config(5) says Compression defaults to no.
Re: reliable, dd over simple ip network
On Sat, Oct 18, 2008 at 6:57 AM, Matthew Dempsky [EMAIL PROTECTED] wrote: On Fri, Oct 17, 2008 at 10:52 PM, Johan Beisser [EMAIL PROTECTED] wrote: You know ssh will compress what goes through its tunnel to begin with, right? ssh_config(5) says Compression defaults to no. If you use ssh -C it'll compress
Re: reliable, dd over simple ip network
On 23:57:17 Oct 17, Matthew Dempsky wrote: On Fri, Oct 17, 2008 at 10:52 PM, Johan Beisser [EMAIL PROTECTED] wrote: You know ssh will compress what goes through its tunnel to begin with, right? ssh_config(5) says Compression defaults to no. That is quite correct. And I left out the cd /destir for the restore command that happens at the other side. Moreover with gzip you can select a compression level between 0 and 9 that suits your network and processing speeds best. And you could loop this command line for all the partitions in a simple shell script after you setup ssh-agent(1). -Girish
Re: reliable, dd over simple ip network
On Sat, Oct 18, 2008 at 12:12 AM, Mike [EMAIL PROTECTED] wrote: If you use ssh -C it'll compress I know, but I understood ssh will compress what goes through its tunnel to begin with to imply this is the default behavior. Maybe Johan meant can instead of will.
Re: reliable, dd over simple ip network
On Oct 18, 2008, at 2:23 AM, Matthew Dempsky wrote: I know, but I understood ssh will compress what goes through its tunnel to begin with to imply this is the default behavior. Maybe Johan meant can instead of will. You're right, I did. Sorry for the confusion, I was typing on the blackberry vs on something useful, like my laptop.
Re: reliable, dd over simple ip network
will work out much faster and better than plain old dd(1). On the other side you have to run # input | restore xf - -Girish whats the input going to be?
Re: reliable, dd over simple ip network
complete binary data of wd0c (more than 6 partitions) on one fly. neko --- On Fri, 10/17/08, Mike [EMAIL PROTECTED] wrote: From: Mike [EMAIL PROTECTED] Subject: Re: reliable, dd over simple ip network To: [EMAIL PROTECTED], misc@openbsd.org Date: Friday, October 17, 2008, 1:29 PM will work out much faster and better than plain old dd(1). On the other side you have to run # input | restore xf - -Girish whats the input going to be? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
Re: reliable, dd over simple ip network
On 17:29:56 Oct 17, Mike wrote: will work out much faster and better than plain old dd(1). On the other side you have to run # input | restore xf - -Girish whats the input going to be? Sorry I was wrong. It was meant to be done in one step from the dump side. This works for me. # dump af - /dev/rwd0d | gzip -c - | ssh hostname gzip -d -| restore rf - Hope it works out for you. Thanks. -Girish
Re: reliable, dd over simple ip network
On 21:28:56 Oct 15, Neko wrote: Good day to all of you, i have found a really dirty way of going around this, so im fishing for advices on finding a reliable way to dd over simple ip network with the generic bsd. could this be done in a straight pipe ? i have an ftp on the generic bsd, containing data, this bsd system is on a multiple os drive. i have no choice to dd, since multiple partition got updated out of hand, no way to single track specific updated folders. *well actually yes, its the dirty way stipulated above* since my partitions have 16% free on all systems, i cant tarball the drive sent it to target machine and uncompress, anyays, if you have suggestion on opensource pkgs, services i could open, or any bright idea i would like to hear them, dd(1) is not a good idea. If you want to back up across the n/w, then dump(8) with ssh(8) may be interesting. # dump af - | ssh ... will work out much faster and better than plain old dd(1). On the other side you have to run # input | restore xf - -Girish
Re: reliable, dd over simple ip network
On Wed, Oct 15, 2008 at 09:28:56PM -0700, Neko wrote: since my partitions have 16% free on all systems, i cant tarball the drive sent it to target machine and uncompress, Tarball it up, pipe the output somewhere, eg via ssh (disclaimer: untested; concept only) [tar commands, to stdout] | ssh [EMAIL PROTECTED] cat - [tar commands to untar the ball] or tarball.tgz Or use rsync? Doug.
Re: reliable, dd over simple ip network
Maybe the simplest usage: tar cfz - /somedir | ssh somehost dd of=/somefile.tgz John On Thu, Oct 16, 2008 at 10:42:17AM -0400, Douglas A. Tutty wrote: On Wed, Oct 15, 2008 at 09:28:56PM -0700, Neko wrote: since my partitions have 16% free on all systems, i cant tarball the drive sent it to target machine and uncompress, Tarball it up, pipe the output somewhere, eg via ssh (disclaimer: untested; concept only) [tar commands, to stdout] | ssh [EMAIL PROTECTED] cat - [tar commands to untar the ball] or tarball.tgz Or use rsync? Doug.
Re: reliable, dd over simple ip network
On Wed, Oct 15, 2008 at 10:28 PM, Neko [EMAIL PROTECTED] wrote: i have found a really dirty way of going around this, so im fishing for advices on finding a reliable way to dd over simple ip network with the generic bsd. could this be done in a straight pipe ? i have an ftp on the generic bsd, containing data, this bsd system is on a multiple os drive. i have no choice to dd, since multiple partition got updated out of hand, no way to single track specific updated folders. *well actually yes, its the dirty way stipulated above* since my partitions have 16% free on all systems, i cant tarball the drive sent it to target machine and uncompress, anyays, if you have suggestion on opensource pkgs, services i could open, or any bright idea i would like to hear them You could easily use a few pipes, dd and the built-in netcatand add some compression too if you wanted.
Re: reliable, dd over simple ip network
Daniel Melameth escribis: On Wed, Oct 15, 2008 at 10:28 PM, Neko [EMAIL PROTECTED] wrote: i have found a really dirty way of going around this, so im fishing for advices on finding a reliable way to dd over simple ip network with the generic bsd. could this be done in a straight pipe ? i have an ftp on the generic bsd, containing data, this bsd system is on a multiple os drive. i have no choice to dd, since multiple partition got updated out of hand, no way to single track specific updated folders. *well actually yes, its the dirty way stipulated above* since my partitions have 16% free on all systems, i cant tarball the drive sent it to target machine and uncompress, anyays, if you have suggestion on opensource pkgs, services i could open, or any bright idea i would like to hear them You could easily use a few pipes, dd and the built-in netcatand add some compression too if you wanted. nc -l , tar and gzip also looks great. -Jesus
Re: reliable, dd over simple ip network
wow thanks for your time, yes i already master dd, and i have to use it since im cloning two disk that are identical both disk with more that 5 partition / 6 os. i have no choice I HAVE TO binary copy the disk, and their is a catch since i want to update all my partitions in a fly, since many of them had updates, i normally got track of all of the changes and ftp the small tarballs to the appropriate system, and what i meant by catch is disks are identical i have 16% free on both disk, i cant afford *in the design not monetarly* to dump a tarball that would weight more that a hundred time what i have left for ressources. i am using ip4/ftp/ssh/sftp as of openservices, my question is i need to create a device that could stream the binary flow straight to my disk tru the ip4/sftp/ssh net. more suggestion ? thanks neko --- On Thu, 10/16/08, Mr D R Hughes [EMAIL PROTECTED] wrote: From: Mr D R Hughes [EMAIL PROTECTED] Subject: Re: reliable, dd over simple ip network To: [EMAIL PROTECTED] Date: Thursday, October 16, 2008, 5:43 AM Neko wrote: Good day to all of you, i have found a really dirty way of going around this, so im fishing for advices on finding a reliable way to dd over simple ip network with the generic bsd. could this be done in a straight pipe ? i have an ftp on the generic bsd, containing data, this bsd system is on a multiple os drive. i have no choice to dd, since multiple partition got updated out of hand, no way to single track specific updated folders. *well actually yes, its the dirty way stipulated above* since my partitions have 16% free on all systems, i cant tarball the drive sent it to target machine and uncompress, If you can mount the destination (eg; via NFS or Samba) then you can still use tar (it should also be possible to pipe the tarred stdin through scp to an sshd enabled destination if you can't mount it):- cd TargetDir tar cvpf - . | ( cd DestinationDir tar xvpf - . ) ; sync ; sync Other choices would include dump (re; $ man dump) and rsync (re; OpenBSD packages), but if for some reason you really must use dd (eg; to clone a disk/partition), I've not tried it but dd should work using the following or similar command over a network mounted filesystem after booting to single user with network support mode:- dd if=/dev/TargetDisk | ( cd /DestinationDir dd of=BackupFile.image ) ; sync ; sync Note that this process is likely to take a very long time unless you give dd appropriate ibs and obs or bs value/s to speed it up (see $ man dd). The destination backup file will also be a raw data (ie; image) file which you'll have to mount as a vnode pseudo-device (see $ man vnconfig) if you only want to restore a few files and not the whole disk or partition at a later date. Generally dd isn't a good choice for backing up data unless you want to keep clones of hard disks or partitions for replication. Also when cloning disks or partitions it is usually more convenient to remove the source disk/s and fit it and the destination disk/s to a spare machine for cloning. Rhys anyays, if you have suggestion on opensource pkgs, services i could open, or any bright idea i would like to hear them, since my solution for now is screwdrivers :C thanks neko
Re: reliable, dd over simple ip network
since tar can be a device, and ssh open a port can i use straight device to device using both engines ? --- On Thu, 10/16/08, John Jackson [EMAIL PROTECTED] wrote: From: John Jackson [EMAIL PROTECTED] Subject: Re: reliable, dd over simple ip network To: misc@openbsd.org Date: Thursday, October 16, 2008, 12:26 PM Maybe the simplest usage: tar cfz - /somedir | ssh somehost dd of=/somefile.tgz John On Thu, Oct 16, 2008 at 10:42:17AM -0400, Douglas A. Tutty wrote: On Wed, Oct 15, 2008 at 09:28:56PM -0700, Neko wrote: since my partitions have 16% free on all systems, i cant tarball the drive sent it to target machine and uncompress, Tarball it up, pipe the output somewhere, eg via ssh (disclaimer: untested; concept only) [tar commands, to stdout] | ssh [EMAIL PROTECTED] cat - [tar commands to untar the ball] or tarball.tgz Or use rsync? Doug.
Re: reliable, dd over simple ip network
On 2008-10-16, Neko [EMAIL PROTECTED] wrote: yes i already master dd, and i have to use it since im cloning two disk that are identical both disk with more that 5 partition / 6 os. If you've mastered it, you'll know it can output or input data over a pipe to/from another program. Like ssh.
