Re: [OmniOS-discuss] announcement znapzend
Hi Hafiz sure ... from the manpage :) --mbuffer=/usr/bin/mbuffer:31337 Specifiy the path to your copy of the mbuffer utility and the port used on the destination. Caution: znapzend will send the data directly from source mbuffer to destination mbuffer, thus data stream is not encrypted. note this has only been added recently cheers tobi Yesterday Hafiz Rafibeyli wrote: Thanks for quick answer Tobi, could you please share info about znapsend via mbuffer only(without ssh) syntax? is this something like this? znapzendzetup create --recursive --mbuffer=/opt/omni/bin/mbuffer \ --mbuffersize=1G --tsformat='%Y-%m-%d-%H%M%S' \ SRC '7d=1h,30d=4h,90d=1d' tank/home \ DST:a '7d=1h,30d=4h,90d=1d,1y=1w,10y=1month' -O 10.0.0.1:9090 - Original Message - From: Tobias Oetiker t...@oetiker.ch To: Hafiz Rafibeyli rafibe...@gmail.com Cc: omnios-discuss@lists.omniti.com Sent: Monday, 11 August, 2014 11:44:21 AM Subject: Re: [OmniOS-discuss] announcement znapzend Hi Hafiz, Today Hafiz Rafibeyli wrote: Tobias thank you for great job,it was missing backup part for zfs on omnios, I think ssh will slow for bigger datasets,as you mention znapzend 0.11 supporting use of mbuffer. I could not find mbuffer package for omnios,could you explain how to setup/use mbuffer on omnios please? mbuffer is in the omniti repo # pkg set-publisher -g http://pkg.omniti.com/omniti-ms/ ms.omniti.com # pkg install mbuffer cheers tobi regards - Original Message - From: omnios-discuss-requ...@lists.omniti.com To: omnios-discuss@lists.omniti.com Sent: Tuesday, 29 July, 2014 10:29:42 PM Subject: OmniOS-discuss Digest, Vol 28, Issue 8 Send OmniOS-discuss mailing list submissions to omnios-discuss@lists.omniti.com To subscribe or unsubscribe via the World Wide Web, visit http://lists.omniti.com/mailman/listinfo/omnios-discuss or, via email, send a message with subject or body 'help' to omnios-discuss-requ...@lists.omniti.com You can reach the person managing the list at omnios-discuss-ow...@lists.omniti.com When replying, please edit your Subject line so it is more specific than Re: Contents of OmniOS-discuss digest... Today's Topics: 1. announcement znapzend a new zfs backup tool (Tobias Oetiker) 2. Re: announcement znapzend a new zfs backup tool (Theo Schlossnagle) 3. Re: announcement znapzend a new zfs backup tool (Saso Kiselkov) 4. Re: Slow scrub performance (wuffers) -- Message: 1 Date: Tue, 29 Jul 2014 17:50:02 +0200 (CEST) From: Tobias Oetiker t...@oetiker.ch To: omnios-discuss@lists.omniti.com Subject: [OmniOS-discuss] announcement znapzend a new zfs backup tool Message-ID: alpine.deb.2.02.1407291748500.6...@froburg.oetiker.ch Content-Type: TEXT/PLAIN; charset=US-ASCII Just out: ZnapZend a Multilevel Backuptool for ZFS It is on Github. Check out http://www.znapzend.org cheers tobi -- Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland www.oetiker.ch t...@oetiker.ch +41 62 775 9902 ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] announcement znapzend
Hi Tobi, but where to specify destination itself?is this syntax correct? znapzendzetup create --recursive --mbuffer=/opt/omni/bin/mbuffer:9090 \ --mbuffersize=1G --tsformat='%Y-%m-%d-%H%M%S' \ SRC '7d=1h,30d=4h,90d=1d' tank/home \ DST:a '7d=1h,30d=4h,90d=1d,1y=1w,10y=1month' -O 10.0.0.1:9090 - Original Message - From: Tobias Oetiker t...@oetiker.ch To: Hafiz Rafibeyli rafibe...@gmail.com Cc: omnios-discuss@lists.omniti.com Sent: Tuesday, 12 August, 2014 9:12:01 AM Subject: Re: [OmniOS-discuss] announcement znapzend Hi Hafiz sure ... from the manpage :) --mbuffer=/usr/bin/mbuffer:31337 Specifiy the path to your copy of the mbuffer utility and the port used on the destination. Caution: znapzend will send the data directly from source mbuffer to destination mbuffer, thus data stream is not encrypted. note this has only been added recently cheers tobi Yesterday Hafiz Rafibeyli wrote: Thanks for quick answer Tobi, could you please share info about znapsend via mbuffer only(without ssh) syntax? is this something like this? znapzendzetup create --recursive --mbuffer=/opt/omni/bin/mbuffer \ --mbuffersize=1G --tsformat='%Y-%m-%d-%H%M%S' \ SRC '7d=1h,30d=4h,90d=1d' tank/home \ DST:a '7d=1h,30d=4h,90d=1d,1y=1w,10y=1month' -O 10.0.0.1:9090 - Original Message - From: Tobias Oetiker t...@oetiker.ch To: Hafiz Rafibeyli rafibe...@gmail.com Cc: omnios-discuss@lists.omniti.com Sent: Monday, 11 August, 2014 11:44:21 AM Subject: Re: [OmniOS-discuss] announcement znapzend Hi Hafiz, Today Hafiz Rafibeyli wrote: Tobias thank you for great job,it was missing backup part for zfs on omnios, I think ssh will slow for bigger datasets,as you mention znapzend 0.11 supporting use of mbuffer. I could not find mbuffer package for omnios,could you explain how to setup/use mbuffer on omnios please? mbuffer is in the omniti repo # pkg set-publisher -g http://pkg.omniti.com/omniti-ms/ ms.omniti.com # pkg install mbuffer cheers tobi regards - Original Message - From: omnios-discuss-requ...@lists.omniti.com To: omnios-discuss@lists.omniti.com Sent: Tuesday, 29 July, 2014 10:29:42 PM Subject: OmniOS-discuss Digest, Vol 28, Issue 8 Send OmniOS-discuss mailing list submissions to omnios-discuss@lists.omniti.com To subscribe or unsubscribe via the World Wide Web, visit http://lists.omniti.com/mailman/listinfo/omnios-discuss or, via email, send a message with subject or body 'help' to omnios-discuss-requ...@lists.omniti.com You can reach the person managing the list at omnios-discuss-ow...@lists.omniti.com When replying, please edit your Subject line so it is more specific than Re: Contents of OmniOS-discuss digest... Today's Topics: 1. announcement znapzend a new zfs backup tool (Tobias Oetiker) 2. Re: announcement znapzend a new zfs backup tool (Theo Schlossnagle) 3. Re: announcement znapzend a new zfs backup tool (Saso Kiselkov) 4. Re: Slow scrub performance (wuffers) -- Message: 1 Date: Tue, 29 Jul 2014 17:50:02 +0200 (CEST) From: Tobias Oetiker t...@oetiker.ch To: omnios-discuss@lists.omniti.com Subject: [OmniOS-discuss] announcement znapzend a new zfs backup tool Message-ID: alpine.deb.2.02.1407291748500.6...@froburg.oetiker.ch Content-Type: TEXT/PLAIN; charset=US-ASCII Just out: ZnapZend a Multilevel Backuptool for ZFS It is on Github. Check out http://www.znapzend.org cheers tobi -- Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland www.oetiker.ch t...@oetiker.ch +41 62 775 9902 -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] announcement znapzend
Hi Hafiz, Today Hafiz Rafibeyli wrote: Hi Tobi, but where to specify destination itself?is this syntax correct? znapzendzetup create --recursive --mbuffer=/opt/omni/bin/mbuffer:9090 \ --mbuffersize=1G --tsformat='%Y-%m-%d-%H%M%S' \ SRC '7d=1h,30d=4h,90d=1d' tank/home \ DST:a '7d=1h,30d=4h,90d=1d,1y=1w,10y=1month' -O 10.0.0.1:9090 the destination is the same as if you were using ssh only ... znapzend still uses ssh to discover the snapshots present at the remote end, and to launch the mbuffer command at the receiving end znapzendzetup create --recursive --mbuffer=/opt/omni/bin/mbuffer:13872 \ --mbuffersize=1G --tsformat='%Y-%m-%d-%H%M%S' \ SRC '7d=1h,30d=4h,90d=1d' tank/home \ DST:a '7d=1h,30d=4h,90d=1d,1y=1w,10y=1month' root@bserv:backup/home cheers tobi - Original Message - From: Tobias Oetiker t...@oetiker.ch To: Hafiz Rafibeyli rafibe...@gmail.com Cc: omnios-discuss@lists.omniti.com Sent: Tuesday, 12 August, 2014 9:12:01 AM Subject: Re: [OmniOS-discuss] announcement znapzend Hi Hafiz sure ... from the manpage :) --mbuffer=/usr/bin/mbuffer:31337 Specifiy the path to your copy of the mbuffer utility and the port used on the destination. Caution: znapzend will send the data directly from source mbuffer to destination mbuffer, thus data stream is not encrypted. note this has only been added recently cheers tobi Yesterday Hafiz Rafibeyli wrote: Thanks for quick answer Tobi, could you please share info about znapsend via mbuffer only(without ssh) syntax? is this something like this? znapzendzetup create --recursive --mbuffer=/opt/omni/bin/mbuffer \ --mbuffersize=1G --tsformat='%Y-%m-%d-%H%M%S' \ SRC '7d=1h,30d=4h,90d=1d' tank/home \ DST:a '7d=1h,30d=4h,90d=1d,1y=1w,10y=1month' -O 10.0.0.1:9090 - Original Message - From: Tobias Oetiker t...@oetiker.ch To: Hafiz Rafibeyli rafibe...@gmail.com Cc: omnios-discuss@lists.omniti.com Sent: Monday, 11 August, 2014 11:44:21 AM Subject: Re: [OmniOS-discuss] announcement znapzend Hi Hafiz, Today Hafiz Rafibeyli wrote: Tobias thank you for great job,it was missing backup part for zfs on omnios, I think ssh will slow for bigger datasets,as you mention znapzend 0.11 supporting use of mbuffer. I could not find mbuffer package for omnios,could you explain how to setup/use mbuffer on omnios please? mbuffer is in the omniti repo # pkg set-publisher -g http://pkg.omniti.com/omniti-ms/ ms.omniti.com # pkg install mbuffer cheers tobi regards - Original Message - From: omnios-discuss-requ...@lists.omniti.com To: omnios-discuss@lists.omniti.com Sent: Tuesday, 29 July, 2014 10:29:42 PM Subject: OmniOS-discuss Digest, Vol 28, Issue 8 Send OmniOS-discuss mailing list submissions to omnios-discuss@lists.omniti.com To subscribe or unsubscribe via the World Wide Web, visit http://lists.omniti.com/mailman/listinfo/omnios-discuss or, via email, send a message with subject or body 'help' to omnios-discuss-requ...@lists.omniti.com You can reach the person managing the list at omnios-discuss-ow...@lists.omniti.com When replying, please edit your Subject line so it is more specific than Re: Contents of OmniOS-discuss digest... Today's Topics: 1. announcement znapzend a new zfs backup tool (Tobias Oetiker) 2. Re: announcement znapzend a new zfs backup tool (Theo Schlossnagle) 3. Re: announcement znapzend a new zfs backup tool (Saso Kiselkov) 4. Re: Slow scrub performance (wuffers) -- Message: 1 Date: Tue, 29 Jul 2014 17:50:02 +0200 (CEST) From: Tobias Oetiker t...@oetiker.ch To: omnios-discuss@lists.omniti.com Subject: [OmniOS-discuss] announcement znapzend a new zfs backup tool Message-ID: alpine.deb.2.02.1407291748500.6...@froburg.oetiker.ch Content-Type: TEXT/PLAIN; charset=US-ASCII Just out: ZnapZend a Multilevel Backuptool for ZFS It is on Github. Check out http://www.znapzend.org cheers tobi -- Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland www.oetiker.ch t...@oetiker.ch +41 62 775 9902 ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] announcement znapzend
On Mon, 11 Aug 2014, Theo Schlossnagle wrote: OmniOS ships with pipeviewer (pv), if you use pv -s several megs, it would have close to the same effect as using mbuffer. According to the 'pv' manual page (read on a Ubuntu installation), I assume that you mean -B (or --buffer-size) rather than -s? Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] Getting detailed NFS lock information on an OmniOS NFS server
This may help (or may not). I presume this will work on OmniOS as it works on other Illumos variants. You can enable logging on an nfs share by adding 'log' to the sharenfs property of a zfs file system. For example, on one of my systems I have $ zfs get sharenfs pool0/media NAME PROPERTY VALUE SOURCE pool0/example sharenfs sec=sys,log,rw=@10.1.1.0/24:... local After this change, svc:/network/nfs/log:default is enabled automatically and a log file 'nfslog' (and other related files) can be found int /var/nfs. I don't know if this records the locks but it will at least show who/what is accessing files under an nfs share. BTW, the nfs/log service collects information for a bit before it drops it into the nfslog file. -- Scott LeFevre 317-696-1010 On Mon, 2014-08-11 at 23:49 -0400, Chris Siebenmann wrote: Here's a question: does anyone know of a good way to get detailed NFS lock information on an OmniOS NFS server, especially information like which client has a lock on a particular file? 'mdb -k' can be used to extract basic lock information, like the full paths of all files that have active locks against them[*], but my old Solaris mdb commands for getting more detailed NFS information don't work in the OmniOS mdb any more. Thanks in advance. (I'll happily take pointers to where to look in the kernel implementation to follow struct pointers by hand with mdb's ::print and so on. From what I've found so far, I'm looking at usr/src/uts/common/klm to start with; is this right?) - cks [*: via mdb's ::lminfo and '::walk lock_graph', see eg http://utcc.utoronto.ca/~cks/space/blog/solaris/ListingFileLocks Solaris 10 also had ::nlm_lockson, which is still present in mdb but doesn't seem to work any more, presumably because the actual NFS locking implementation changed. ] ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] Getting detailed NFS lock information on an OmniOS NFS server
On Aug 12, 2014, at 4:38 PM, Scott LeFevre slefe...@indy.rr.com wrote: This may help (or may not). I presume this will work on OmniOS as it works on other Illumos variants. NFS logging is a perfect way to destroy performance. It is very serial and will not scale well. One important Sun customer had the misfortune of wiring it into their app and when they had their big event and the systems were in production, performance was 15% of what they expected. The only solution was to disable logging, but they required it for their app. Needless to say, the app was rewritten, but they had lots of pain during the big event -- richard You can enable logging on an nfs share by adding 'log' to the sharenfs property of a zfs file system. For example, on one of my systems I have $ zfs get sharenfs pool0/media NAME PROPERTY VALUE SOURCE pool0/example sharenfs sec=sys,log,rw=@10.1.1.0/24:... local After this change, svc:/network/nfs/log:default is enabled automatically and a log file 'nfslog' (and other related files) can be found int /var/nfs. I don't know if this records the locks but it will at least show who/what is accessing files under an nfs share. BTW, the nfs/log service collects information for a bit before it drops it into the nfslog file. -- Scott LeFevre 317-696-1010 On Mon, 2014-08-11 at 23:49 -0400, Chris Siebenmann wrote: Here's a question: does anyone know of a good way to get detailed NFS lock information on an OmniOS NFS server, especially information like which client has a lock on a particular file? 'mdb -k' can be used to extract basic lock information, like the full paths of all files that have active locks against them[*], but my old Solaris mdb commands for getting more detailed NFS information don't work in the OmniOS mdb any more. Thanks in advance. (I'll happily take pointers to where to look in the kernel implementation to follow struct pointers by hand with mdb's ::print and so on. From what I've found so far, I'm looking at usr/src/uts/common/klm to start with; is this right?) - cks [*: via mdb's ::lminfo and '::walk lock_graph', see eg http://utcc.utoronto.ca/~cks/space/blog/solaris/ListingFileLocks Solaris 10 also had ::nlm_lockson, which is still present in mdb but doesn't seem to work any more, presumably because the actual NFS locking implementation changed. ] ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
