Re: reliable, dd over simple ip network
On Oct 18, 2008, at 2:23 AM, Matthew Dempsky wrote: I know, but I understood "ssh will compress what goes through its tunnel to begin with" to imply this is the default behavior. Maybe Johan meant "can" instead of "will." You're right, I did. Sorry for the confusion, I was typing on the blackberry vs on something useful, like my laptop.
Re: reliable, dd over simple ip network
On Sat, Oct 18, 2008 at 12:12 AM, Mike <[EMAIL PROTECTED]> wrote: > If you use ssh -C it'll compress I know, but I understood "ssh will compress what goes through its tunnel to begin with" to imply this is the default behavior. Maybe Johan meant "can" instead of "will."
Re: reliable, dd over simple ip network
On 23:57:17 Oct 17, Matthew Dempsky wrote: > On Fri, Oct 17, 2008 at 10:52 PM, Johan Beisser <[EMAIL PROTECTED]> wrote: > > You know ssh will compress what goes through its tunnel to begin with, > > right? > > ssh_config(5) says Compression defaults to "no". That is quite correct. And I left out the "cd /destir" for the restore command that happens at the other side. Moreover with gzip you can select a compression level between 0 and 9 that suits your network and processing speeds best. And you could loop this command line for all the partitions in a simple shell script after you setup ssh-agent(1). -Girish
Re: reliable, dd over simple ip network
On Sat, Oct 18, 2008 at 6:57 AM, Matthew Dempsky <[EMAIL PROTECTED]> wrote: > On Fri, Oct 17, 2008 at 10:52 PM, Johan Beisser <[EMAIL PROTECTED]> wrote: >> You know ssh will compress what goes through its tunnel to begin with, right? > > ssh_config(5) says Compression defaults to "no". > > If you use ssh -C it'll compress
Re: reliable, dd over simple ip network
On Fri, Oct 17, 2008 at 10:52 PM, Johan Beisser <[EMAIL PROTECTED]> wrote: > You know ssh will compress what goes through its tunnel to begin with, right? ssh_config(5) says Compression defaults to "no".
Re: reliable, dd over simple ip network
You know ssh will compress what goes through its tunnel to begin with, right? So, you can eliminate at least one command there.. On 10/17/08, Girish Venkatachalam <[EMAIL PROTECTED]> wrote: > On 17:29:56 Oct 17, Mike wrote: >> > >> > will work out much faster and better than plain old dd(1). >> > >> > On the other side you have to run >> > >> > # | restore xf - >> > >> > -Girish >> > >> >> whats the going to be? > > Sorry I was wrong. It was meant to be done in one step from the "dump > side". > > This works for me. > > # dump af - /dev/rwd0d | gzip -c - | ssh "gzip -d -| > restore rf -" > > Hope it works out for you. > > Thanks. > > -Girish
Re: reliable, dd over simple ip network
On 17:29:56 Oct 17, Mike wrote: > > > > will work out much faster and better than plain old dd(1). > > > > On the other side you have to run > > > > # | restore xf - > > > > -Girish > > > > whats the going to be? Sorry I was wrong. It was meant to be done in one step from the "dump side". This works for me. # dump af - /dev/rwd0d | gzip -c - | ssh "gzip -d -| restore rf -" Hope it works out for you. Thanks. -Girish
Re: reliable, dd over simple ip network
complete binary data of wd0c (more than 6 partitions) on one fly. neko --- On Fri, 10/17/08, Mike <[EMAIL PROTECTED]> wrote: > From: Mike <[EMAIL PROTECTED]> > Subject: Re: reliable, dd over simple ip network > To: [EMAIL PROTECTED], misc@openbsd.org > Date: Friday, October 17, 2008, 1:29 PM > > will work out much faster and better than plain old > dd(1). > > > > On the other side you have to run > > > > # | restore xf - > > > > -Girish > > > > whats the going to be? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
Re: reliable, dd over simple ip network
> will work out much faster and better than plain old dd(1). > > On the other side you have to run > > # | restore xf - > > -Girish > whats the going to be?
Re: reliable, dd over simple ip network
On 2008-10-16, Neko <[EMAIL PROTECTED]> wrote: > yes i already master dd, and i have to use it since im cloning two > disk that are identical both disk with more that 5 partition / 6 os. If you've mastered it, you'll know it can output or input data over a pipe to/from another program. Like ssh.
Re: reliable, dd over simple ip network
since tar can be a device, and ssh open a port can i use straight device to device using both engines ? --- On Thu, 10/16/08, John Jackson <[EMAIL PROTECTED]> wrote: > From: John Jackson <[EMAIL PROTECTED]> > Subject: Re: reliable, dd over simple ip network > To: misc@openbsd.org > Date: Thursday, October 16, 2008, 12:26 PM > Maybe the simplest usage: > > tar cfz - /somedir | ssh somehost "dd > of=/somefile.tgz" > > John > > On Thu, Oct 16, 2008 at 10:42:17AM -0400, Douglas A. Tutty > wrote: > > On Wed, Oct 15, 2008 at 09:28:56PM -0700, Neko wrote: > > > > > since my partitions have 16% free on all systems, > i cant tarball the > > > drive sent it to target machine and uncompress, > > > > Tarball it up, pipe the output somewhere, eg via ssh > > > > (disclaimer: untested; concept only) > > > > [tar commands, to stdout] | ssh [EMAIL PROTECTED] "cat > - [tar commands to > > untar the ball] or > tarball.tgz > > > > Or use rsync? > > > > Doug.
Re: reliable, dd over simple ip network
wow thanks for your time, yes i already master dd, and i have to use it since im cloning two disk that are identical both disk with more that 5 partition / 6 os. i have no choice I HAVE TO binary copy the disk, and their is a catch since i want to update all my partitions in a fly, since many of them had updates, i normally got track of all of the changes and ftp the small tarballs to the appropriate system, and what i meant by catch is disks are identical i have 16% free on both disk, i cant afford *in the design not monetarly* to dump a tarball that would weight more that a hundred time what i have left for ressources. i am using ip4/ftp/ssh/sftp as of openservices, my question is i need to create a device that could stream the binary flow straight to my disk tru the ip4/sftp/ssh net. more suggestion ? thanks neko --- On Thu, 10/16/08, Mr D R Hughes <[EMAIL PROTECTED]> wrote: > From: Mr D R Hughes <[EMAIL PROTECTED]> > Subject: Re: reliable, dd over simple ip network > To: [EMAIL PROTECTED] > Date: Thursday, October 16, 2008, 5:43 AM > Neko wrote: > > Good day to all of you, > > > > i have found a really dirty way of going around this, > > so im fishing for advices on finding a reliable way > > to dd over simple ip network with the generic bsd. > > > > could this be done in a straight pipe ? > > > > i have an ftp on the generic bsd, containing data, > this > > bsd system is on a multiple os drive. i have no choice > to > > dd, since multiple partition got updated out of hand, > no way > > to single track specific updated folders. *well > actually yes, its > > the dirty way stipulated above* > > > > since my partitions have 16% free on all systems, i > cant tarball the > > drive sent it to target machine and uncompress, > > If you can mount the destination (eg; via NFS or Samba) > then you can > still use tar (it should also be possible to pipe the > tarred stdin > through scp to an sshd enabled destination if you can't > mount it):- > > cd TargetDir && tar cvpf - . | ( cd DestinationDir > && tar xvpf - . ) ; > sync ; sync > > Other choices would include dump (re; $ man dump) and rsync > (re; OpenBSD > packages), but if for some reason you really must use dd > (eg; to clone > a disk/partition), I've not tried it but dd should work > using the > following or similar command over a network mounted > filesystem after > booting to single user with network support mode:- > > dd if=/dev/TargetDisk | ( cd /DestinationDir && dd > of=BackupFile.image ) > ; sync ; sync > > Note that this process is likely to take a very long time > unless you > give dd appropriate ibs and obs or bs value/s to speed it > up (see $ man > dd). > > The destination backup file will also be a raw data (ie; > image) file > which you'll have to mount as a vnode pseudo-device > (see $ man vnconfig) > if you only want to restore a few files and not the whole > disk or > partition at a later date. > > Generally dd isn't a good choice for backing up data > unless you want to > keep clones of hard disks or partitions for replication. > Also when > cloning disks or partitions it is usually more convenient > to remove the > source disk/s and fit it and the destination disk/s to a > spare machine > for cloning. > > Rhys > > > > > anyays, if you have suggestion on opensource pkgs, > services i could open, > > or any bright idea i would like to hear them, > > > > since my solution for now is screwdrivers :C > > > > thanks > > > > neko
Re: reliable, dd over simple ip network
Daniel Melameth escribis: On Wed, Oct 15, 2008 at 10:28 PM, Neko <[EMAIL PROTECTED]> wrote: i have found a really dirty way of going around this, so im fishing for advices on finding a reliable way to dd over simple ip network with the generic bsd. could this be done in a straight pipe ? i have an ftp on the generic bsd, containing data, this bsd system is on a multiple os drive. i have no choice to dd, since multiple partition got updated out of hand, no way to single track specific updated folders. *well actually yes, its the dirty way stipulated above* since my partitions have 16% free on all systems, i cant tarball the drive sent it to target machine and uncompress, anyays, if you have suggestion on opensource pkgs, services i could open, or any bright idea i would like to hear them You could easily use a few pipes, dd and the built-in netcatand add some compression too if you wanted. nc -l , tar and gzip also looks great. -Jesus
Re: reliable, dd over simple ip network
On Wed, Oct 15, 2008 at 10:28 PM, Neko <[EMAIL PROTECTED]> wrote: > i have found a really dirty way of going around this, > so im fishing for advices on finding a reliable way > to dd over simple ip network with the generic bsd. > > could this be done in a straight pipe ? > > i have an ftp on the generic bsd, containing data, this > bsd system is on a multiple os drive. i have no choice to > dd, since multiple partition got updated out of hand, no way > to single track specific updated folders. *well actually yes, its > the dirty way stipulated above* > > since my partitions have 16% free on all systems, i cant tarball the > drive sent it to target machine and uncompress, > > anyays, if you have suggestion on opensource pkgs, services i could open, > or any bright idea i would like to hear them You could easily use a few pipes, dd and the built-in netcatand add some compression too if you wanted.
Re: reliable, dd over simple ip network
Maybe the simplest usage: tar cfz - /somedir | ssh somehost "dd of=/somefile.tgz" John On Thu, Oct 16, 2008 at 10:42:17AM -0400, Douglas A. Tutty wrote: > On Wed, Oct 15, 2008 at 09:28:56PM -0700, Neko wrote: > > > since my partitions have 16% free on all systems, i cant tarball the > > drive sent it to target machine and uncompress, > > Tarball it up, pipe the output somewhere, eg via ssh > > (disclaimer: untested; concept only) > > [tar commands, to stdout] | ssh [EMAIL PROTECTED] "cat - [tar commands to > untar the ball] or > tarball.tgz > > Or use rsync? > > Doug.
Re: reliable, dd over simple ip network
On Wed, Oct 15, 2008 at 09:28:56PM -0700, Neko wrote: > since my partitions have 16% free on all systems, i cant tarball the > drive sent it to target machine and uncompress, Tarball it up, pipe the output somewhere, eg via ssh (disclaimer: untested; concept only) [tar commands, to stdout] | ssh [EMAIL PROTECTED] "cat - [tar commands to untar the ball] or > tarball.tgz Or use rsync? Doug.
Re: reliable, dd over simple ip network
On 21:28:56 Oct 15, Neko wrote: > Good day to all of you, > > i have found a really dirty way of going around this, > so im fishing for advices on finding a reliable way > to dd over simple ip network with the generic bsd. > > could this be done in a straight pipe ? > > i have an ftp on the generic bsd, containing data, this > bsd system is on a multiple os drive. i have no choice to > dd, since multiple partition got updated out of hand, no way > to single track specific updated folders. *well actually yes, its > the dirty way stipulated above* > > since my partitions have 16% free on all systems, i cant tarball the > drive sent it to target machine and uncompress, > > anyays, if you have suggestion on opensource pkgs, services i could open, > or any bright idea i would like to hear them, > dd(1) is not a good idea. If you want to back up across the n/w, then dump(8) with ssh(8) may be interesting. # dump af - | ssh ... will work out much faster and better than plain old dd(1). On the other side you have to run # | restore xf - -Girish
