date:20170824

[OmniOS-discuss] zfs recv causes system to crash / hang

2017-08-24 Thread Oliver Weinmann

Hi all,

every time trying to do zfs send | recv between OmniOS 151022.x and Nexenta 
4.0.5x the Nexenta node crashes. This is very critical to us as we either want 
to use the OmniOS system for DR or migrate some files between the two. I raised 
a ticket with Nexenta and they pointed me in the right direction. It seems that 
this problem had already been reported on Illumos mailinglist. So far no real 
fix has been provided.

https://www.mail-archive.com/discuss@lists.illumos.org/msg02699.html

The only way to fix this is either patch the receiving side (Nexenta). They 
have not done this for 4.0.5.x yet. But I was told that it will be done in 5.1 
and it is planned to upstream the fix to Illumos.

6393 zfs receive a full send as a clone

or the sending side.

6536 zfs send: want a way to disable setting of DRR_FLAG_FREERECORDS

But the patch for the sending side (OmniOS) doesn't work. The last thing on the 
post is an advice to implement this fix on the sending side:

https://gist.github.com/pcd1193182/fcb9f8d43dcbcf32ba736ea7ef600658

It seems that the problem not only affects zfs send | recv between Illumos 
based an Nexenta systems. Nexenta told us that they have a fix for NS 5.1 but 
currently upgrading to 5.x is not an option for us as this version has some 
limitations and it currently doesn't have this very important fix implemented:

https://www.illumos.org/issues/8543

Is there anyone else effected by this bug?

Best Regards,
Oliver



[cid:Logo_Telespazio_180_px_signature_eng_b58fa623-e26d-4116-9230-766adacfe55e1.png]

Oliver Weinmann
Senior Unix VMWare, Storage Engineer

Telespazio VEGA Deutschland GmbH
Europaplatz 5 - 64293 Darmstadt - Germany
Ph: + 49 (0)6151 8257 744 | Fax: +49 (0)6151 8257 799
oliver.weinm...@telespazio-vega.de
http://www.telespazio-vega.de

Registered office/Sitz: Darmstadt, Register court/Registergericht: Darmstadt, 
HRB 89231; Managing Director/Gesch?ftsf?hrer: Sigmar Keller
___
OmniOS-discuss mailing list
OmniOS-discuss@lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss

Re: [OmniOS-discuss] Constantly losing nfs shares smb shares?

2017-08-24 Thread Oliver Weinmann

Hi,

I have done some more investigation and I found the cause for this problem. It 
always happens when running zfs send from a Nexenta system.




[cid:Logo_Telespazio_180_px_signature_eng_b58fa623-e26d-4116-9230-766adacfe55e1.png]

Oliver Weinmann
Senior Unix VMWare, Storage Engineer

Telespazio VEGA Deutschland GmbH
Europaplatz 5 - 64293 Darmstadt - Germany
Ph: + 49 (0)6151 8257 744 | Fax: +49 (0)6151 8257 799
oliver.weinm...@telespazio-vega.de
http://www.telespazio-vega.de

Registered office/Sitz: Darmstadt, Register court/Registergericht: Darmstadt, 
HRB 89231; Managing Director/Geschäftsführer: Sigmar Keller
From: OmniOS-discuss [mailto:omnios-discuss-boun...@lists.omniti.com] On Behalf 
Of Oliver Weinmann
Sent: Montag, 21. August 2017 10:00
To: omnios-discuss 
Subject: [OmniOS-discuss] Constantly losing nfs shares smb shares?

Hi,

I have no clue why but on our omnios box (151022k) we are constantly losing all 
our nfs and smb  shares. To fix it I have two shell scripts that just reset the 
sharenfs and sharesmb options. But this is not really a good fix as it happens 
at random times. I don't know where to start investigating. I have nothing 
suspicious in /var/adm/messages.

Best Regards,
Oliver



[cid:image001.png@01D31CBE.95E4A530]

Oliver Weinmann
Senior Unix VMWare, Storage Engineer

Telespazio VEGA Deutschland GmbH
Europaplatz 5 - 64293 Darmstadt - Germany
Ph: + 49 (0)6151 8257 744 | Fax: +49 (0)6151 8257 799
oliver.weinm...@telespazio-vega.de
http://www.telespazio-vega.de

Registered office/Sitz: Darmstadt, Register court/Registergericht: Darmstadt, 
HRB 89231; Managing Director/Geschäftsführer: Sigmar Keller
___
OmniOS-discuss mailing list
OmniOS-discuss@lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss

[OmniOS-discuss] Upgrade to 151022m from 014 - horrible NFS performance

2017-08-24 Thread Schweiss, Chip

I just move one of my production systems to OmniOS CE 151022m from 151014
and my NFS performance has tanked.

Here's a snapshot of nfssvrtop:

2017 Aug 24 07:34:39, load: 1.54, read: 5427 KB, swrite: 104  KB,
awrite: 9634 KB
Ver Client   NFSOPS   Reads SWrites AWrites Commits   Rd_bw
 SWr_bw  AWr_bwRd_t   SWr_t   AWr_t   Com_t  Align%
3   10.28.17.10   0   0   0   0   0
  0   0   0   0   0   0   0
3   all   0   0   0   0   0   0
  0   0   0   0   0   0   0
4   10.28.17.19   0   0   0   0   0
  0   0   0   0   0   0   0
4   10.28.16.160 17   0   0   0   0   0
  0   0   0   0   0   0   0
4   10.28.16.127 20   0   0   0   0   0
  0   0   0   0   0   0   0
4   10.28.16.113 74   6   6   0   0  48
 56   01366   20824   0   0 100
4   10.28.16.64 338  16   0  36   3 476
  01065 120   0 130  117390 100
4   10.28.16.54 696  68   0  91   52173
  02916  52   0  93  142083 100
4   all1185  90   6 127   82697
 563996 151   20824 104  133979 100

The pool is not doing anything but serving NFS.   Before the upgrade, the
pool would sustain 20k NFS ops.

Is there some significant change in NFS that I need to adjust its tuning?

-Chip
___
OmniOS-discuss mailing list
OmniOS-discuss@lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss

Re: [OmniOS-discuss] Upgrade to 151022m from 014 - horrible NFS performance

2017-08-24 Thread Dan McDonald


> On Aug 24, 2017, at 8:41 AM, Schweiss, Chip  wrote:
> 
> I just move one of my production systems to OmniOS CE 151022m from 151014 and 
> my NFS performance has tanked.  
> 
> Here's a snapshot of nfssvrtop:
> 
> 2017 Aug 24 07:34:39, load: 1.54, read: 5427 KB, swrite: 104  KB, 
> awrite: 9634 KB
> Ver Client   NFSOPS   Reads SWrites AWrites Commits   Rd_bw  
> SWr_bw  AWr_bwRd_t   SWr_t   AWr_t   Com_t  Align%
> 3   10.28.17.10   0   0   0   0   0   
> 0   0   0   0   0   0   0
> 3   all   0   0   0   0   0   0   
> 0   0   0   0   0   0   0
> 4   10.28.17.19   0   0   0   0   0   
> 0   0   0   0   0   0   0
> 4   10.28.16.160 17   0   0   0   0   0   
> 0   0   0   0   0   0   0
> 4   10.28.16.127 20   0   0   0   0   0   
> 0   0   0   0   0   0   0
> 4   10.28.16.113 74   6   6   0   0  48  
> 56   01366   20824   0   0 100
> 4   10.28.16.64 338  16   0  36   3 476   
> 01065 120   0 130  117390 100
> 4   10.28.16.54 696  68   0  91   52173   
> 02916  52   0  93  142083 100
> 4   all1185  90   6 127   82697  
> 563996 151   20824 104  133979 100
> 
> The pool is not doing anything but serving NFS.   Before the upgrade, the 
> pool would sustain 20k NFS ops.   
> 
> Is there some significant change in NFS that I need to adjust its tuning?

Oh my.

I'd start pinging the illumos list on this.  Also, are there any special tweaks 
you made in the 014 configuration?  IF you did, I'd start back removing them 
and seeing what a default system does, just in case.

I know Delphix and Nexenta still care about NFS quite a bit, so I can't believe 
something would be that bad.

Maintainers:  Check for NFS changes RIGHT AFTER 022 closed for blanket upstream 
pull-ins.  Maybe it closed during a poor-performance window?

Dan

___
OmniOS-discuss mailing list
OmniOS-discuss@lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss

Re: [OmniOS-discuss] Upgrade to 151022m from 014 - horrible NFS performance

2017-08-24 Thread Schweiss, Chip

I switched back to 014 for now, it was too bad to inflict on my users.

I have some new systems coming in soon that I'll test on r151022 before
making them live.   I will start with the NFS defaults.

-Chip

On Thu, Aug 24, 2017 at 8:35 AM, Dan McDonald  wrote:

>
> > On Aug 24, 2017, at 8:41 AM, Schweiss, Chip  wrote:
> >
> > I just move one of my production systems to OmniOS CE 151022m from
> 151014 and my NFS performance has tanked.
> >
> > Here's a snapshot of nfssvrtop:
> >
> > 2017 Aug 24 07:34:39, load: 1.54, read: 5427 KB, swrite: 104
> KB, awrite: 9634 KB
> > Ver Client   NFSOPS   Reads SWrites AWrites Commits   Rd_bw
> SWr_bw  AWr_bwRd_t   SWr_t   AWr_t   Com_t  Align%
> > 3   10.28.17.10   0   0   0   0   0
>  0   0   0   0   0   0   0
> > 3   all   0   0   0   0   0   0
>  0   0   0   0   0   0   0
> > 4   10.28.17.19   0   0   0   0   0
>  0   0   0   0   0   0   0
> > 4   10.28.16.160 17   0   0   0   0   0
>  0   0   0   0   0   0   0
> > 4   10.28.16.127 20   0   0   0   0   0
>  0   0   0   0   0   0   0
> > 4   10.28.16.113 74   6   6   0   0  48
> 56   01366   20824   0   0 100
> > 4   10.28.16.64 338  16   0  36   3 476
>  01065 120   0 130  117390 100
> > 4   10.28.16.54 696  68   0  91   52173
>  02916  52   0  93  142083 100
> > 4   all1185  90   6 127   82697
> 563996 151   20824 104  133979 100
> >
> > The pool is not doing anything but serving NFS.   Before the upgrade,
> the pool would sustain 20k NFS ops.
> >
> > Is there some significant change in NFS that I need to adjust its tuning?
>
> Oh my.
>
> I'd start pinging the illumos list on this.  Also, are there any special
> tweaks you made in the 014 configuration?  IF you did, I'd start back
> removing them and seeing what a default system does, just in case.
>
> I know Delphix and Nexenta still care about NFS quite a bit, so I can't
> believe something would be that bad.
>
> Maintainers:  Check for NFS changes RIGHT AFTER 022 closed for blanket
> upstream pull-ins.  Maybe it closed during a poor-performance window?
>
> Dan
>
>
___
OmniOS-discuss mailing list
OmniOS-discuss@lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss

Re: [OmniOS-discuss] Upgrade to 151022m from 014 - horrible NFS performance

2017-08-24 Thread Bob Friesenhahn


On Thu, 24 Aug 2017, Schweiss, Chip wrote:


I switched back to 014 for now, it was too bad to inflict on my users.

I have some new systems coming in soon that I'll test on r151022 before
making them live.   I will start with the NFS defaults.


No evidence has been presented that there is a NFS problem.  It could 
be an underlying zfs pool issue.


Bob
--
Bob Friesenhahn
bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,http://www.GraphicsMagick.org/
___
OmniOS-discuss mailing list
OmniOS-discuss@lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss

Re: [OmniOS-discuss] Upgrade to 151022m from 014 - horrible NFS performance

2017-08-24 Thread Richard Elling


> On Aug 24, 2017, at 5:41 AM, Schweiss, Chip  wrote:
> 
> I just move one of my production systems to OmniOS CE 151022m from 151014 and 
> my NFS performance has tanked.  
> 
> Here's a snapshot of nfssvrtop:
> 
> 2017 Aug 24 07:34:39, load: 1.54, read: 5427 KB, swrite: 104  KB, 
> awrite: 9634 KB
> Ver Client   NFSOPS   Reads SWrites AWrites Commits   Rd_bw  
> SWr_bw  AWr_bwRd_t   SWr_t   AWr_t   Com_t  Align%
> 3   10.28.17.10   0   0   0   0   0   
> 0   0   0   0   0   0   0
> 3   all   0   0   0   0   0   0   
> 0   0   0   0   0   0   0
> 4   10.28.17.19   0   0   0   0   0   
> 0   0   0   0   0   0   0
> 4   10.28.16.160 17   0   0   0   0   0   
> 0   0   0   0   0   0   0
> 4   10.28.16.127 20   0   0   0   0   0   
> 0   0   0   0   0   0   0
> 4   10.28.16.113 74   6   6   0   0  48  
> 56   01366   20824   0   0 100
> 4   10.28.16.64 338  16   0  36   3 476   
> 01065 120   0 130  117390 100
> 4   10.28.16.54 696  68   0  91   52173   
> 02916  52   0  93  142083 100
> 4   all1185  90   6 127   82697  
> 563996 151   20824 104  133979 100
> 
> The pool is not doing anything but serving NFS.   Before the upgrade, the 
> pool would sustain 20k NFS ops.   

The commit time is in microseconds, and it does look high. Is there a slog?
 — richard

> 
> Is there some significant change in NFS that I need to adjust its tuning?
> 
> -Chip
> 
> 
> 
> ___
> OmniOS-discuss mailing list
> OmniOS-discuss@lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss

___
OmniOS-discuss mailing list
OmniOS-discuss@lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss

Re: [OmniOS-discuss] Upgrade to 151022m from 014 - horrible NFS performance

2017-08-24 Thread Artem Penner

Hi, can you post output of
https://github.com/d-helios/dtrace/blob/master/zfs/zfs_rwlatency.d
And if you have separate log device output of
https://github.com/d-helios/dtrace/blob/master/zfs/zil_latency.d
---
What is your nfs-server settings? (sharectl get nfs)


чт, 24 авг. 2017 г. в 19:35, Richard Elling <
richard.ell...@richardelling.com>:

> On Aug 24, 2017, at 5:41 AM, Schweiss, Chip  wrote:
>
> I just move one of my production systems to OmniOS CE 151022m from 151014
> and my NFS performance has tanked.
>
> Here's a snapshot of nfssvrtop:
>
> 2017 Aug 24 07:34:39, load: 1.54, read: 5427 KB, swrite: 104  KB,
> awrite: 9634 KB
> Ver Client   NFSOPS   Reads SWrites AWrites Commits   Rd_bw
>  SWr_bw  AWr_bwRd_t   SWr_t   AWr_t   Com_t  Align%
> 3   10.28.17.10   0   0   0   0   0
> 0   0   0   0   0   0   0
> 3   all   0   0   0   0   0   0
> 0   0   0   0   0   0   0
> 4   10.28.17.19   0   0   0   0   0
> 0   0   0   0   0   0   0
> 4   10.28.16.160 17   0   0   0   0   0
> 0   0   0   0   0   0   0
> 4   10.28.16.127 20   0   0   0   0   0
> 0   0   0   0   0   0   0
> 4   10.28.16.113 74   6   6   0   0  48
>56   01366   20824   0   0 100
> 4   10.28.16.64 338  16   0  36   3 476
> 01065 120   0 130  117390 100
> 4   10.28.16.54 696  68   0  91   52173
> 02916  52   0  93  142083 100
> 4   all1185  90   6 127   82697
>563996 151   20824 104  133979 100
>
> The pool is not doing anything but serving NFS.   Before the upgrade, the
> pool would sustain 20k NFS ops.
>
>
> The commit time is in microseconds, and it does look high. Is there a slog?
>  — richard
>
>
> Is there some significant change in NFS that I need to adjust its tuning?
>
> -Chip
>
>
>
> ___
> OmniOS-discuss mailing list
> OmniOS-discuss@lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
> ___
> OmniOS-discuss mailing list
> OmniOS-discuss@lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
___
OmniOS-discuss mailing list
OmniOS-discuss@lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss

[OmniOS-discuss] failure in set-publisher for a zone to the new omniosce repository

2017-08-24 Thread Paul Jochum


Hi All:

I am working on updating a number of omnios machines from r151022 (under 
OmniOS TI) to R151022 (under OmniOS CE).  I am having problems updating 
one of my zones.  I am at the point now where I have unset all of the 
publishers in that zone, but I can not add the omniosce publisher.


# pkg publisher
PUBLISHER   TYPE STATUS P LOCATION
#
# /usr/bin/pkg set-publisher -P -g 
https://pkg.omniosce.org/r151022/core/ omnios
pkg set-publisher: The origin URIs for 'omnios' do not appear to point 
to a valid pkg repository.
Please verify the repository's location and the client's network 
configuration.

Additional details:

Unable to contact valid package repository
Encountered the following error(s):
Unable to contact any configured publishers.
This is likely a network configuration problem.
Unable to locate a CA directory: /etc/openssl/certs
Secure connection is not available.

This worked fine on the host of that zone, and on other zones (located 
on other hosts, but all at the same level of software). Any suggestions 
on how to fix this?  (And I checked, there is no /etc/openssl directory 
on this or any of my other omnios machines, but there is the 
/etc/ssl/certs directory and it looks very similar to other 
/etc/ssl/certs on machine which did not have a problem updating the 
publisher)


thanks

Paul
paul.joc...@nokia.com



___
OmniOS-discuss mailing list
OmniOS-discuss@lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss

Re: [OmniOS-discuss] failure in set-publisher for a zone to the new omniosce repository

2017-08-24 Thread John D Groenveld

In message , Paul Jochum writes
:
>configuration.
>Additional details:
>
>Unable to contact valid package repository
>Encountered the following error(s):
>Unable to contact any configured publishers.
>This is likely a network configuration problem.
>Unable to locate a CA directory: /etc/openssl/certs
>Secure connection is not available.
>
>This worked fine on the host of that zone, and on other zones (located 
>on other hosts, but all at the same level of software). Any suggestions 
>on how to fix this?  (And I checked, there is no /etc/openssl directory 
>on this or any of my other omnios machines, but there is the 
>/etc/ssl/certs directory and it looks very similar to other 
>/etc/ssl/certs on machine which did not have a problem updating the 
>publisher)

Shot in the dark assuming lipkg brand zone:
# zoneadm -z $zone halt
# zoneadm -z $zone detach
# /usr/bin/wget -P $zonepath/root/etc/ssl/pkg \
https://downloads.omniosce.org/ssl/omniosce-ca.cert.pem
# pkg -R $zonepath/root set-publisher -P \
-g https://pkg.omniosce.org/r151022/core/ omnios 
# zoneadm -z $zone attach -U

John
groenv...@acm.org
___
OmniOS-discuss mailing list
OmniOS-discuss@lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss

[OmniOS-discuss] zfs recv causes system to crash / hang

Re: [OmniOS-discuss] Constantly losing nfs shares smb shares?

[OmniOS-discuss] Upgrade to 151022m from 014 - horrible NFS performance

Re: [OmniOS-discuss] Upgrade to 151022m from 014 - horrible NFS performance

Re: [OmniOS-discuss] Upgrade to 151022m from 014 - horrible NFS performance

Re: [OmniOS-discuss] Upgrade to 151022m from 014 - horrible NFS performance

Re: [OmniOS-discuss] Upgrade to 151022m from 014 - horrible NFS performance

Re: [OmniOS-discuss] Upgrade to 151022m from 014 - horrible NFS performance

[OmniOS-discuss] failure in set-publisher for a zone to the new omniosce repository

Re: [OmniOS-discuss] failure in set-publisher for a zone to the new omniosce repository

10 matches

Site Navigation

Mail list logo

Footer information