Re: [OmniOS-discuss] Big Data
You might want to read this thread (especially the comments). http://lists.ceph.com/pipermail/ceph-users-ceph.com/2017-January/015599.html On Sat, Jun 16, 2018 at 10:46 AM Michael Talbott wrote: > We've been using OmniOS happily for years now for our storage server > needs. But we're rapidly increasing our data footprint and growing so much > (multiple PBs per year) that ideally I'd like to move to a cluster based > object store based system ontop of OmniOS. I successfully use BeeGFS inside > lxzones in OmniOS which seems to work nicely for our HPC scratch volume, > but, it doesn't sound like it scales to hundreds of millions of files very > well. > > I am hoping that someone has some ideas for me. Ideally I'd like something > that's cluster capable and has erasure coding like Ceph and have cluster > aware snapshots (not local zfs snaps) and an s3 compatibility/access layer. > > Any thoughts on the topic are greatly appreciated. > > Thanks, > > Michael > Sent from my iPhone > ___ > OmniOS-discuss mailing list > OmniOS-discuss@lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > -- Ian Kaufman Research Systems Administrator UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] questions
What about the attached vnics? Can you do: dladm show-linkprop vnic# for the vnics connected to the etherstub? There may be a maxbw setting ... On Thu, Sep 14, 2017 at 9:41 AM, Dirk Willems wrote: > just execute => dladm create-etherstub Backend_Switch0 > > > and having => Backend_Switch0 etherstub 9000 up > > On 14-09-17 18:26, Ian Kaufman wrote: > > Networking has always used *bps - that's been the standard for many years. > Megabits, Gigabits ... > > Disk tools have always measured in bytes since that is how the capacity is > defined. > > How did you create your etherstub? I know you can set a maxbw (maximum > bandiwdth), but I don't know what the default behavior is. > > Ian > > > > On Thu, Sep 14, 2017 at 9:13 AM, Dirk Willems > wrote: > >> Thank you all, the water is already clearing up :) >> >> >> So infiniband is 40 Gbps an not 40GB/s, very confusing GB/s Gbps why they >> not take a standaard and set everything in GB/s or MB/s ? >> >> A lot of people make a lot of mistakes between them, me too ... >> >> If it is 40 Gbps a factor of 8 then we theoretical have max 5 GB/s >> throughput. >> >> Little difference 40 or 5 :) >> >> So Ian you have the full blow with 36Gbps very cool looks more like it :) >> >> Did I play with the frame size, not really sure what you mean by that >> sorry but I think its default on 9000 >> >> Backend_Switch0 etherstub 9000 up >> >> >> Do understand that if we use UDP streams from process to process it will >> be much quicker over the etherstub gonna need more test to do. >> >> We used for a customer Mbuffer with zfs send over Lan that is also very >> quick sometimes I also use it at my home very good prog. >> >> But still do not understand how it is that I copy from 1 NGZ with >> 100MB/s, I receive on the other NGZ 250MB/s very strange ? >> >> >> the command dlstat difference between OmniOSce and Solaris ? >> >> RBYTES => receiving >> >> OBYTES => sending >> >> root@test2:~# dlstat -i 2 >> > >> > LINKIPKTS RBYTESOPKTS OBYTES >> >net1 25.76K 185.14M 10.08K2.62M >> >net1 27.04K 187.16M 11.23K3.22M >> >> >> >> BYTES => receiving and sending ? >> >> But then still if the copy is not running I have 0 so doesn't explain why >> I see 216 MB where come the rest of the 116 MB from is it compression ? >> >> root@NGINX:/root# dlstat show-link NGINX1 -i 2 >> > >> > LINK TYPE ID INDEX PKTSBYTES >> > NGINX1rx bcast --00 >> > NGINX1rx sw --00 >> > NGINX1tx bcast --00 >> > NGINX1tx sw --9.26K 692.00K >> > NGINX1rx local -- 26.00K 216.32M >> >> >> Thank you all for your feedback much appreciations ! >> >> >> Kind Regards, >> >> >> Dirk >> >> >> >> On 14-09-17 17:07, Ian Kaufman wrote: >> >> Some other things you need to take into account: >> >> QDR Infiniband is 40Gbps, not 40GB/s. That is a factor of 8 difference. >> That is also a theoretical maximum throughput, there is some overhead. In >> reality, you will never see 40Gbps. >> >> My system tested out at 6Gbps - 8Gbps using NFS over IPoIB, with DDR >> (20Gbps) nodes and a QDR (40Gbps) storage server. IPoIB drops the >> theoretical max rates to 18Gbps and 36Gbps respectively. >> >> If you are getting 185MB/s, you are seeing 1.48Gbps. >> >> Keep your B's and b's straight. Did you play with your frame size at all? >> >> Ian >> >> On Thu, Sep 14, 2017 at 7:10 AM, Jim Klimov wrote: >> >>> On September 14, 2017 2:26:13 PM GMT+02:00, Dirk Willems < >>> dirk.will...@exitas.be> wrote: >>> >Hello, >>> > >>> > >>> >I'm trying to understand something let me explain. >>> > >>> > >>> >Oracle always told to me that if you create a etherstub switch it has >>> >infiniband speed 40GB/s. >>> > >>> >But I have a customer running on Solaris (Yeah I know but let me >>> >explain) who is copy from 1 NGZ to another NGZ on the same GZ over Lan >>> >(I know told him to to use etherstub). >>> > >>> >The copy witch is performed for a Oracle database with sql command,
Re: [OmniOS-discuss] questions
Networking has always used *bps - that's been the standard for many years. Megabits, Gigabits ... Disk tools have always measured in bytes since that is how the capacity is defined. How did you create your etherstub? I know you can set a maxbw (maximum bandiwdth), but I don't know what the default behavior is. Ian On Thu, Sep 14, 2017 at 9:13 AM, Dirk Willems wrote: > Thank you all, the water is already clearing up :) > > > So infiniband is 40 Gbps an not 40GB/s, very confusing GB/s Gbps why they > not take a standaard and set everything in GB/s or MB/s ? > > A lot of people make a lot of mistakes between them, me too ... > > If it is 40 Gbps a factor of 8 then we theoretical have max 5 GB/s > throughput. > > Little difference 40 or 5 :) > > So Ian you have the full blow with 36Gbps very cool looks more like it :) > > Did I play with the frame size, not really sure what you mean by that > sorry but I think its default on 9000 > > Backend_Switch0 etherstub 9000 up > > > Do understand that if we use UDP streams from process to process it will > be much quicker over the etherstub gonna need more test to do. > > We used for a customer Mbuffer with zfs send over Lan that is also very > quick sometimes I also use it at my home very good prog. > > But still do not understand how it is that I copy from 1 NGZ with 100MB/s, > I receive on the other NGZ 250MB/s very strange ? > > > the command dlstat difference between OmniOSce and Solaris ? > > RBYTES => receiving > > OBYTES => sending > > root@test2:~# dlstat -i 2 > > > > LINKIPKTS RBYTESOPKTS OBYTES > >net1 25.76K 185.14M 10.08K2.62M > >net1 27.04K 187.16M 11.23K3.22M > > > > BYTES => receiving and sending ? > > But then still if the copy is not running I have 0 so doesn't explain why > I see 216 MB where come the rest of the 116 MB from is it compression ? > > root@NGINX:/root# dlstat show-link NGINX1 -i 2 > > > > LINK TYPE ID INDEX PKTSBYTES > > NGINX1rx bcast --00 > > NGINX1rx sw --00 > > NGINX1tx bcast --00 > > NGINX1 tx sw --9.26K 692.00K > > NGINX1rx local -- 26.00K 216.32M > > > Thank you all for your feedback much appreciations ! > > > Kind Regards, > > > Dirk > > > > On 14-09-17 17:07, Ian Kaufman wrote: > > Some other things you need to take into account: > > QDR Infiniband is 40Gbps, not 40GB/s. That is a factor of 8 difference. > That is also a theoretical maximum throughput, there is some overhead. In > reality, you will never see 40Gbps. > > My system tested out at 6Gbps - 8Gbps using NFS over IPoIB, with DDR > (20Gbps) nodes and a QDR (40Gbps) storage server. IPoIB drops the > theoretical max rates to 18Gbps and 36Gbps respectively. > > If you are getting 185MB/s, you are seeing 1.48Gbps. > > Keep your B's and b's straight. Did you play with your frame size at all? > > Ian > > On Thu, Sep 14, 2017 at 7:10 AM, Jim Klimov wrote: > >> On September 14, 2017 2:26:13 PM GMT+02:00, Dirk Willems < >> dirk.will...@exitas.be> wrote: >> >Hello, >> > >> > >> >I'm trying to understand something let me explain. >> > >> > >> >Oracle always told to me that if you create a etherstub switch it has >> >infiniband speed 40GB/s. >> > >> >But I have a customer running on Solaris (Yeah I know but let me >> >explain) who is copy from 1 NGZ to another NGZ on the same GZ over Lan >> >(I know told him to to use etherstub). >> > >> >The copy witch is performed for a Oracle database with sql command, the >> > >> >DBA witch have 5 streams say it's waiting on the disk, the disk are 50 >> >- >> >60 % busy the speed is 30 mb/s. >> > >> > >> >So I did some test just to see and understand if it's the database or >> >the system, but with doing my tests I get very confused ??? >> > >> > >> >On another Solaris at my work copy over etherstub switch => copy speed >> >is 185MB/s expected much more of infiniband speed ??? >> > >> > >> >root@test1:/export/home/Admin# scp test10G >> >Admin@192.168.1.2:/export/home/Admin/ >> >Password: >> >test10G 100% >> >|| >> >10240 >> >MB00:59 >> >
Re: [OmniOS-discuss] questions
t; NGINX1rx local -- 30.65K 253.73M > > NGINX1rx bcast --00 > > NGINX1rx sw --00 > > NGINX1tx bcast --00 > > NGINX1tx sw --8.95K 669.32K > > NGINX1rx local -- 29.10K 241.15M > > > > > >On the other NGZ I receive 250MB/s > > > > > >- So my question is how comes that the speed is equal to Lan 100MB/s on > > > >OmniOSce but i receive 250MB/s ? > > > >- Why is etherstub so slow if infiniband speed is 40GB/s ??? > > > > > >I'm very confused right now ... > > > > > >And want to know for sure how to understand and see this in the right > >way, because this customer will be the first customer from my who gonna > > > >switch complety over to OmniOSce on production and because this > >customer > >is one or the biggest company's in Belgium I really don't want to mess > >up !!! > > > > > >So any help and clarification will be highly appreciate !!! > > > > > >Thank you very much. > > > > > >Kind Regards, > > > > > >Dirk > > I am not sure where the infiniband claim comes from, but copying data disk > to disk, you involve the slow layers like disk, skewed by faster layers > like cache of already-read data and delayed writes :) > > If you have a wide pipe that you may fill, it doesn't mean you do have the > means to fill it with a few disks. > > To estimate the speeds, try pure UDP streams from process to process (no > disk), large-packet floodping, etc. > > I believe etherstub is not constrained artificially, and defaults to jumbo > frames. Going to LAN and back can in fact use external hardware (IIRC there > may be a system option to disable that, not sure) and so is constrained by > that. > > Jim > -- > Typos courtesy of K-9 Mail on my Android > ___ > OmniOS-discuss mailing list > OmniOS-discuss@lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > -- Ian Kaufman Research Systems Administrator UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] NFS mounts seems as nobody on Centos6
Are you using NFSv4? Are all machines using the same idmap domain? Ian On Thu, Jun 15, 2017 at 11:18 AM, Andries Annema wrote: > Hi Özkan, > > The first thing that comes to mind is "no_root_squash" and, based on your > example setting with options like "anon=root", I assume this option can be > set as well on OmniOS with the "sharenfs" setting. > > Maybe these will help: > https://serverfault.com/questions/364613/centos-6- > ldap-nfs-file-ownership-is-stuck-on-nobody > http://www.linuxtopia.org/online_books/rhel6/rhel_6_ > storage_admin/rhel_6_storage_s2-nfs-security-files.html > > > Disclaimer: I am not an expert on NFS! Far from it. The above suggestion > is based on some personal experience where I needed that "no_root_squash" > option (although that was with some Linux distro's), and some Google-fu. > With that said, I am not sure if it adds a security threat or something. > > Anyway, my two cents. > > Regards, > Andries > > > On 2017-06-14 17:44, Özkan Göksu wrote: > > Hello. > > I have a nfs share and zfs settings like= "sharenfs anon=root,rw=* " > When i mount it from Centos, owner changes as nobody. (yes im root on > Centos) > But in omnios or ubuntu i see as root when i mount it. > > What is the cause of this problem? > > > > > ___ > OmniOS-discuss mailing > listOmniOS-discuss@lists.omniti.comhttp://lists.omniti.com/mailman/listinfo/omnios-discuss > > > > ___ > OmniOS-discuss mailing list > OmniOS-discuss@lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > > -- Ian Kaufman Research Systems Administrator UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] OmniOS r151022 is now out!
Hmmm, having trouble going from 14 to 22. I cannot migrate to OpenSSH -
root@hiperq-data:/root# /usr/bin/pkg install --no-backup-be --reject
pkg:/network/ssh --reject pkg:/network/ssh/ssh-key --reject
pkg:/service/network/ssh --reject pkg:/service/network/ssh-common
pkg:/network/openssh pkg:/network/openssh-server
Creating Plan (Solver setup): \
pkg install: No matching version of network/openssh can be installed:
Reject: pkg://omnios/network/openssh@7.4.1,5.11-0.151022:20170511T001844Z
Reason: All versions matching 'require' dependency pkg:/library/security/
openssl@1.0.2.11,5.11-0.151022 are rejected
Reject: pkg://omnios/library/security/openssl@1.0.2.11
,5.11-0.151022:20170510T232316Z
Reason: This version is excluded by installed incorporation
pkg://omnios/incorporation/jeos/omnios-userland@11
,5.11-0.151014:20161221T194806Z
No matching version of network/openssh-server can be installed:
Reject: pkg://omnios/network/openssh-server@7.4.1
,5.11-0.151022:20170511T001853Z
Reason: All versions matching 'require' dependency
pkg:/SUNWcs@0.5.11,5.11-0.151022
are rejected
Reject: pkg://omnios/SUNWcs@0.5.11,5.11-0.151022:20170510T212740Z
Reason: This version is excluded by installed incorporation
pkg://omnios/consolidation/osnet/osnet-incorporation@0.5.11
,5.11-0.151014:20170301T162824Z
This version is excluded by installed incorporation
pkg://omnios/incorporation/jeos/illumos-gate@11
,5.11-0.151014:20160804T060038Z
On Fri, May 12, 2017 at 7:50 AM, Dan McDonald wrote:
> As always, please start with the release notes:
>
> https://omnios.omniti.com/wiki.php/ReleaseNotes/r151022
>
> And for this release, there are upgrade notes:
>
> https://omnios.omniti.com/wiki.php/Upgrade_to_r151022
>
> This is an LTS and Stable update. Highlights include:
>
> - BSD Loader is now available. This has its own wiki page, including how
> to stick with GRUB if you think you need to.
>
> - Kayak Interactive Installer --> No more Caiman, no more F-keys & curses,
> and no-more hacking timezone names. Also, one less repo to maintain for
> OmniOS. Kayak does it all now.
>
> - Python is now 2.7. This means an IPS/pkg(5) update too. There's a
> subtle, but noteworthy, update to linked-image semantics ("pkg update -r")
> AND upgrading non-linked-image "ipkg" zones is a bit more difficult due to
> the transition (chicken & egg problem with ipkg brand "attach").
>
> - Perl is now 5.24.1.
>
> - USB 3.0 support.
>
> With OmniTI withdrawing funding for OmniOS-as-product, this will be the
> last OmniTI-pushed "release" of OmniOS. It contains all of the most recent
> (as of May) illumos-gate changes, AND illumos-joyent LX brand changes.
>
> This release broke the 6-month cadence, due to Loader, Python2.7, and the
> need for Kayak to take over ALL installation duties. I'm sorry for it
> being 1-2 months late, but I wanted to make sure these changes were
> comfortably in place during the 021 bloody.
>
> Because of the sheer volume of change in this release, I had to update
> many OmniOS wiki pages. Here's a list of all of those I changed
> nontrivially due to r151022. Please check them out to make sure I'm not
> missing anything...
>
> https://omnios.omniti.com/wiki.php/ReleaseNotes/r151022
>
> https://omnios.omniti.com/wiki.php/Installation
>
> https://omnios.omniti.com/wiki.php/LXZones
>
> https://omnios.omniti.com/wiki.php/Upgrade_to_r151022
>
> https://omnios.omniti.com/wiki.php/NewLinkedImages
>
> https://omnios.omniti.com/wiki.php/linked_images
>
> https://omnios.omniti.com/wiki.php/KayakInteractive
>
> https://omnios.omniti.com/wiki.php/BSDLoader
>
> https://omnios.omniti.com/wiki.php/ReleaseCycle
>
> https://omnios.omniti.com/wiki.php/Packaging
>
> https://omnios.omniti.com/wiki.php/ReleaseNotes/
>
> https://omnios.omniti.com/wiki.php/WikiStart
>
> https://omnios.omniti.com/wiki.php/BuildInstructions
>
> https://omnios.omniti.com/wiki.php/Maintainers
>
> https://omnios.omniti.com/wiki.php/illumos-tools
>
> https://omnios.omniti.com/wiki.php/buildctl
>
> https://omnios.omniti.com/wiki.php/ReleaseMedia
>
> I'll have more to say about my departure from OmniTI in a distinct e-mail.
>
> Thanks, and happy upgrading!
> Dan
>
> ___
> OmniOS-discuss mailing list
> OmniOS-discuss@lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
--
Ian Kaufman
Research Systems Administrator
UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu
___
OmniOS-discuss mailing list
OmniOS-discuss@lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] Network >10Gb/s
Hmm, as far as I know, Omni-Path is not an ethernet replacement, but an IB replacement. While Intel is investing heavily in Omni-Path, it should not impact their ethernet offerings as they really are two different markets. Ian On Tue, Jan 10, 2017 at 8:10 AM, Schweiss, Chip wrote: > > On Tue, Jan 10, 2017 at 9:58 AM, Dan McDonald wrote: > >> >> > On Jan 10, 2017, at 8:41 AM, Schweiss, Chip wrote: >> > >> > It appears that my options for 40Gb/s Ethernet are Intel, Chelsio and >> SolarFlare. >> > >> > Can anyone comment on which of these is the most stable solution when >> running under OmniOS? What's the fastest NFS throughput you've been able >> to achieve? >> >> The Intel i40e driver is nascent, but it will receive more attention as >> time passes. Doug's point about SolarFlare is a good one. >> >> > I'm a bit concerned on the Intel because of posts like this: > https://news.ycombinator.com/item?id=11373848 and the fact that they > seem to have shifted their focus to Omni-Path which from my understanding > is incompatible with the existing 40G gear. > > SolarFlare seems promising, but I'd like to know of at least on success > story. > > -Chip > > > >> You may wish to ping the larger illumos community about this as well. >> > > >> Dan >> >> > > ___ > OmniOS-discuss mailing list > OmniOS-discuss@lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > > -- Ian Kaufman Research Systems Administrator UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] Increase default maximum NFS server threads?
I agree, and, it really has been a long time coming. Those defaults were set ages ago, systems and networking have vastly improved since then. Might as well set defaults accordingly. Ian On Tue, Dec 6, 2016 at 7:51 AM, Richard Elling < richard.ell...@richardelling.com> wrote: > > > On Dec 6, 2016, at 7:30 AM, Dan McDonald wrote: > > > > I got a link to this commit from the Delphix illumos repo a while back: > > > > https://github.com/openzfs/openzfs/pull/186/ > > > > I was curious if NFS-using people in the audience here would like to see > this one Just Land (TM) in illumos-omnios or not? > > just land it, no real downside > — richard > > > ___ > OmniOS-discuss mailing list > OmniOS-discuss@lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > -- Ian Kaufman Research Systems Administrator UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] LDAP external auth for CIFS service
In all honesty, the native Solaris LDAP client sucks. I would investigate installing an OpenLDAP client, or make the system an OpenLDAP slave to your 389 DS, and have the local client talk to the local OpenLDAP slave via loopback. That's how we were able to successfully set things up on our Thors so that they could talk to our OpenLDAP servers. Ian On Thu, Aug 18, 2016 at 11:28 AM, Andries Annema wrote: > *raises hand* > Here's another one interested in this matter. > > Researched the possibilities about two years ago myself, but eventually > gave > up; it didn't seem to be possible. > Would be awesome if it would be one day, though. > > Regards, > Andries > > > -Original Message- > From: OmniOS-discuss [mailto:omnios-discuss-boun...@lists.omniti.com] On > Behalf Of Tobias Oetiker > Sent: donderdag 18 augustus 2016 17:33 > To: Dan McDonald > Cc: omnios-discuss > Subject: Re: [OmniOS-discuss] LDAP external auth for CIFS service > > - On 18 Aug, 2016, at 17:15, Dan McDonald dan...@omniti.com wrote: > > >> On Aug 18, 2016, at 11:04 AM, Mick Burns wrote: > >> > >> *bump* > >> anyone ? > > > > I'm going to forward your note to someone I know who works on CIFS. He's > not on > > this list. > > looking forward to the answer ... :) I have always used an AD for this but > openldap would be so much cooler. > > cheers > tobi > ___ > OmniOS-discuss mailing list > OmniOS-discuss@lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > > ___ > OmniOS-discuss mailing list > OmniOS-discuss@lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > -- Ian Kaufman Research Systems Administrator UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] LDAP and Active Directory via rfc2307
Can you pull an complete user object via LDAP query? There might be secondary attributes that include a POSIX compliant short name. Ian On Fri, Apr 22, 2016 at 2:37 PM, Michael Talbott wrote: > It does have the unix extensions on it which is how I was able to get this > far (set uids/gids/etc in AD). But I don't have the old windows NIS service > running though, so I don't use the SFU30 or whatever attributes since I > believe those are all obsoleted and will soon likely disappear. > > > Michael Talbott > Systems Administrator > La Jolla Institute > > On Apr 22, 2016, at 1:18 PM, Ian Kaufman wrote: > > Does your AD have SFU (or whatever it is called these days) set up? > > Ian > > On Fri, Apr 22, 2016 at 12:58 PM, Michael Talbott > wrote: > >> You're exactly right. The DN in ad is the full name and if I create a >> user where the DN and shortname match, then everything works great. >> Unfortunately, I'm not sure if updating all the DNs to match the short name >> will break other dependancies of it deployed in existing software >> elsewhere. One day when I'm feeling brave and have a little downtime >> scheduled, I'll batch update all the entries and see if anything breaks. >> But, I suppose I'm stuck with winbind for the time being. But thank you for >> all the help. >> >> >> >> > On Apr 22, 2016, at 11:27 AM, Paul B. Henson wrote: >> > >> > On Thu, Apr 21, 2016 at 11:35:56PM -0700, Michael Talbott wrote: >> > >> >> all the group members are listed as "John Doe" rather than jdoe which >> >> means that when jdoe logs in, he can't access his groups due to the >> >> naming disconnect. Any ideas of how to fix that? Somehow map the group >> >> members to samAccountName rather than the DN? >> > >> > How is your AD structured? It sounds like it's using full names for DN's >> > rather than usernames? If so, that's not going to work. >> > >> > Our AD uses usernames for DN's; for example, I'm: >> > >> > dn: CN=henson,OU=user,DC=ad,DC=cpp,DC=edu >> > cn: henson >> > sn: Henson >> > givenName: Paul >> > initials: B. >> > distinguishedName: CN=henson,OU=user,DC=ad,DC=cpp,DC=edu >> > displayName: Paul B. Henson >> > sAMAccountName: henson >> > >> > and if you look at a group I'm in: >> > >> > dn: CN=netadmin,OU=group,DC=ad,DC=cpp,DC=edu >> > cn: netadmin >> > description: Network admins >> > member: CN=henson,OU=user,DC=ad,DC=cpp,DC=edu >> > distinguishedName: CN=netadmin,OU=group,DC=ad,DC=cpp,DC=edu >> > sAMAccountName: netadmin >> > >> > So the RDN for both users and groups is the short name that a unix box >> > expects to see, and the long name is in the displayName or description. >> > I'm guessing you're using the full name as the CN and your users look >> > like: >> > >> > dn: CN=Paul B. Henson,OU=user,DC=ad,DC=cpp,DC=edu >> > >> > so your group members look like: >> > >> > member: CN=Paul B. Henson,OU=user,DC=ad,DC=cpp,DC=edu >> > >> > If that's the case, I don't think there's any way you can get it to >> > work. The rfc2307bis group support expects the RDN to be the username, >> > there's no way to get it to look up some other attribute of the entry >> > and use it instead. >> >> ___ >> OmniOS-discuss mailing list >> OmniOS-discuss@lists.omniti.com >> http://lists.omniti.com/mailman/listinfo/omnios-discuss >> > > > > -- > Ian Kaufman > Research Systems Administrator > UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu > > > -- Ian Kaufman Research Systems Administrator UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] LDAP and Active Directory via rfc2307
Does your AD have SFU (or whatever it is called these days) set up? Ian On Fri, Apr 22, 2016 at 12:58 PM, Michael Talbott wrote: > You're exactly right. The DN in ad is the full name and if I create a user > where the DN and shortname match, then everything works great. > Unfortunately, I'm not sure if updating all the DNs to match the short name > will break other dependancies of it deployed in existing software > elsewhere. One day when I'm feeling brave and have a little downtime > scheduled, I'll batch update all the entries and see if anything breaks. > But, I suppose I'm stuck with winbind for the time being. But thank you for > all the help. > > > > > On Apr 22, 2016, at 11:27 AM, Paul B. Henson wrote: > > > > On Thu, Apr 21, 2016 at 11:35:56PM -0700, Michael Talbott wrote: > > > >> all the group members are listed as "John Doe" rather than jdoe which > >> means that when jdoe logs in, he can't access his groups due to the > >> naming disconnect. Any ideas of how to fix that? Somehow map the group > >> members to samAccountName rather than the DN? > > > > How is your AD structured? It sounds like it's using full names for DN's > > rather than usernames? If so, that's not going to work. > > > > Our AD uses usernames for DN's; for example, I'm: > > > > dn: CN=henson,OU=user,DC=ad,DC=cpp,DC=edu > > cn: henson > > sn: Henson > > givenName: Paul > > initials: B. > > distinguishedName: CN=henson,OU=user,DC=ad,DC=cpp,DC=edu > > displayName: Paul B. Henson > > sAMAccountName: henson > > > > and if you look at a group I'm in: > > > > dn: CN=netadmin,OU=group,DC=ad,DC=cpp,DC=edu > > cn: netadmin > > description: Network admins > > member: CN=henson,OU=user,DC=ad,DC=cpp,DC=edu > > distinguishedName: CN=netadmin,OU=group,DC=ad,DC=cpp,DC=edu > > sAMAccountName: netadmin > > > > So the RDN for both users and groups is the short name that a unix box > > expects to see, and the long name is in the displayName or description. > > I'm guessing you're using the full name as the CN and your users look > > like: > > > > dn: CN=Paul B. Henson,OU=user,DC=ad,DC=cpp,DC=edu > > > > so your group members look like: > > > > member: CN=Paul B. Henson,OU=user,DC=ad,DC=cpp,DC=edu > > > > If that's the case, I don't think there's any way you can get it to > > work. The rfc2307bis group support expects the RDN to be the username, > > there's no way to get it to look up some other attribute of the entry > > and use it instead. > > ___ > OmniOS-discuss mailing list > OmniOS-discuss@lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > -- Ian Kaufman Research Systems Administrator UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] What's the best way to detect OmniOS version, specifically r151014?
Examine /etc/release? Ian On Wed, Dec 2, 2015 at 11:16 AM, Chris Siebenmann wrote: > We have at least one shell script that needs to know if it's running > on a host with OmniOS r151014 versus a host with an earlier OmniOS > version (due to the change in ZFS pool reservations from 1/64th of the > pool to 1/32nd of the pool that we picked up with r151014). Is there > any particular good way for a shell script to determine this, ideally > in a lightweight way and without requiring root permissions? > > Thanks in advance. > > - cks > ___ > OmniOS-discuss mailing list > OmniOS-discuss@lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > -- Ian Kaufman Research Systems Administrator UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] big zfs storage?
Hi Linda, I have two dual-headed OmniOS system attached to two 60 bay JBODs w/ 3TB drives. Using raidz2 and hot spares, each JBOD has about 106TB usable, and i use a single filesystem per JBOD. We may or may not add another JBOD, but we will be expanding into the PB realm eventually. I use napp-it to make things easier to manage, and I will start testing some HA/Failover. I also have two ~70TB usable systems. Ian On Thu, Jul 9, 2015 at 2:21 PM, Linda Kateley wrote: > Hey is there anyone out there running big zfs on omni? > > I have been doing mostly zol and freebsd for the last year but have to build > a 300+TB box and i want to come back home to roots(solaris). Feeling kind of > hesitant :) Also, if you had to do over, is there anything you would do > different. > > Also, what is the go to HBA these days? Seems like i saw stable code for lsi > 3008? > > TIA > > linda > > > ___ > OmniOS-discuss mailing list > OmniOS-discuss@lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss -- Ian Kaufman Research Systems Administrator UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] NFS v4.1 in OmniOS stable?
Lustre is great for HPC. It lacks the "sanctity of data" that other solutions have. It is getting there - using ZFS is a huge step forward, but Lustre is by no means a general purpose solution at this point. Ian On Fri, Mar 20, 2015 at 1:09 PM, Andrew Gabriel wrote: > Dan McDonald wrote: >>> >>> On Mar 20, 2015, at 2:27 PM, Jeff Stockett wrote: >>> >>> Does OmniOS support NFS v4.1? 4.1 support is a new feature in esxi v6, >>> and I was trying to set it up as described here: >>> http://wahlnetwork.com/2015/02/02/nfs-v4-1/ >>> Things of course work fine if I use NFS v3, but if I try v4.1, I get a >>> timeout error when it tries to attach the data store. Both the omnios >>> server and the esxi client are properly joined to Active Directory so I >>> think the required Kerberos stuff should be working. >>> >> >> >> We have NFS4.0 and earlier. We do not have NFS4.1. It would be a very >> sizeable undertaking, requiring illumos community support. If anyone would >> lead the charge on that, it'd be a storage-oriented firm, like Nexenta, or >> Delphix. >> > > > Does anyone seriously use (or intend to use) 4.1 anymore? > > Lustre has the parallel file server market (with some pockets of AFS too). > The large growth of parallel storage servers now tends to be object storage, > and S3 (as a protocol) has become the standard for that. > > Kind of makes me wonder what the market for NFSv4.1 is? > > -- > Andrew > > ___ > OmniOS-discuss mailing list > OmniOS-discuss@lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss -- Ian Kaufman Research Systems Administrator UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] High Availability storage with ZFS
Hi all, I have modified and played with some simple scripts to create my own heartbeat, STONITH, failover set up. I currently have a pair of redundant systems, each one having redundant heads connected to shared JBODs, with the data replicated to the backup system via ZFS send/recv. I am going to do further testing, as this is a new set up, but I had it down to a 10 second failover without too much hassle between heads at one point, and the delta for the data on the backup system was down to about 30 minutes at one point. Ian On Tue, Jan 6, 2015 at 7:19 AM, Stephan Budach wrote: > Am 06.01.15 um 14:08 schrieb Filip Marvan: > > Hi Vincenzo, > > > > your solution is much more better, so thank you very much for your notes. I > will try that too! > > > > Filip > > > > > > From: Vincenzo Pii [mailto:p...@zhaw.ch] > Sent: Tuesday, January 06, 2015 1:54 PM > To: Filip Marvan > Cc: omnios-discuss@lists.omniti.com > Subject: Re: [OmniOS-discuss] High Availability storage with ZFS > > > > 2015-01-06 12:16 GMT+01:00 Filip Marvan : > > Hi > > > > as few guys before, I'm thinking again about High Availability storage with > ZFS. I know, that there is great commercial RSF-1, but that's quite > expensive for my needs. > > I know, that Sašo did a great job about that on his blog > http://zfs-create.blogspot.cz but I never found the way, how to successfully > configure that on current OmniOS versions. > > > > So I'm thinking about something more simple. Arrange two LUNs from two > OmniOS ZFS storages in one software mirror through fibrechannel. Arrange > that mirror in client, for example mdadm in Linux. I know, that it will have > performance affect and I will lost some ZFS advantages, but I still can use > snapshots, backups with send/receive and some other interesting ZFS things, > so it could be usable for some projects. > > Is there anyone, who tried that before? Any eperience with that? > > > > Thank you, > > > > Filip Marvan > > > > > > > Hi Filip, > > > > I am not directly answering your question, but I've gone through the > configuration of HA (with pacemaker) on OmniOS in the past months and > collected all my notes here: > http://blog.zhaw.ch/icclab/use-pacemaker-and-corosync-on-illumos-omnios-to-run-a-ha-activepassive-cluster/, > maybe it can be useful for you. > > > > In my experience, running pacemaker correctly on OmniOS is just the tip of > the iceberg, then comes the implementation/configuration of the resource > agents (and the cluster itself!). > > If this way is worth it, rather than a quicker and more custom solution, > depends on the long term plans :). > > > > Best regards, > > Vincenzo. > > > > Have you looked at the setup that Saso Kiselkov describes in his blog here: > http://zfs-create.blogspot.nl/2013/06/building-zfs-storage-appliance-part-1.html > It seems to cover most of a pacemaker setup, including the resource agents. > > > Cheers, > budy > > ___ > OmniOS-discuss mailing list > OmniOS-discuss@lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > -- Ian Kaufman Research Systems Administrator UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] NFSv4 id mapping only working on client but not server?
There has been some work on multi-domain Kerberos in NFSv4, going back to 2010. Not sure where things stand though. https://www.ietf.org/proceedings/10mar/slides/nfsv4-5.pdf Ian On Tue, Dec 16, 2014 at 4:27 PM, Paul B. Henson wrote: >> From: Schweiss, Chip >> Sent: Tuesday, December 16, 2014 6:02 AM >> >> It seems there a many ways to map ID in NFSv4, is there a way to not map >> them at all? > > I believe linux supports disabling ID mapping and using raw uid/gids on the > wire instead of strings, but I don't think illumos does? > >> All the current file systems being migrated are NFSv3 with AUTH_SYS. I'd >> consider moving them all to kerberos authentication, but something tells me >> that may be impossible with the multiple domains. > > Multiple Kerberos realms too? I don't think illumos can have more than one > kerberos realm defined for NFS... > > -- Ian Kaufman Research Systems Administrator UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] weird library issue
Thanks Dan. Alas, I get stuck here: root@massive02-b:~# pkg -R /mnt update Creating Plan - pkg update: The certificate which issued this certificate:/C=US/ST=Maryland/O=OmniTI/OU=OmniOS/CN=OmniOS r151012 Release Signing Certificate/emailAddress=omnios-supp...@omniti.com could not be found. The issuer is:/C=US/ST=Maryland/L=Fulton/O=OmniTI/CN=OmniTI Certificate Authority The package involved is:pkg://omnios/locale/gu@0.5.11,5.11-0.151012:20140913T033547Z Ian On Wed, Dec 10, 2014 at 10:55 AM, Dan McDonald wrote: > >> On Dec 10, 2014, at 1:42 PM, Ian Kaufman wrote: >> >> It looks like this system is in some weird state where there are >> packages from 151006 and 151008. > > Oh my. > > Yes, you've been bitten by the pkg upgrade problems between 006 and 008. > This was before my time here, so I'm not sure how best to extricate yourself > from things. > > If you want, you can jump forward to the current stable release (r151012). > If you want to try that, and you don't have zones, do this: > > > 1.) Create a new BE, call it 012 for this example: beadm create 012 > > 2.) Mount the BE on /mnt: beadm mount 012 /mnt > > 3.) Change the publisher on the new BE to point to r151012's repo: > pkg -R /mnt set-publisher -G http://pkg.omniti.com/omnios/release/ -g > http://pkg.omniti.com/omnios/r151012/ omnios > > 4.) Apply the update to the new BE: pkg -R /mnt update > > 5.) Unmount the BE: beadm umount 012 > > 6.) Either reboot and use the GRUB menu to try out the new 012 BE, or "beadm > activate 012" to make it go there on the next reboot. > > > Dan > -- Ian Kaufman Research Systems Administrator UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] weird library issue
It looks like this system is in some weird state where there are packages from 151006 and 151008. For example, /etc/release shows: root@massive02-b:~# cat /etc/release OmniOS v11 r151008 Copyright 2013 OmniTI Computer Consulting, Inc. All rights reserved. Use is subject to license terms. but, when I try to do a pkg update, I see this (multiple repeating lines): No suitable version of required package pkg://omnios/incorporation/jeos/omnios-userland@11,5.11-0.151008:20131204T231613Z found: Reject: pkg://omnios/incorporation/jeos/omnios-userland@11,5.11-0.151008:20131204T231613Z Reason: A version for 'incorporate' dependency on pkg:/library/security/openssl@1.0.1,5.11-0.151008 cannot be found No suitable version of required package pkg://omnios/incorporation/jeos/omnios-userland@11,5.11-0.151004:20121024T180535Z found: Reject: pkg://omnios/incorporation/jeos/omnios-userland@11,5.11-0.151004:20121024T180535Z Reason: A version for 'incorporate' dependency on pkg:/archiver/gnu-tar@1.26,5.11-0.151004 cannot be found No suitable version of required package pkg://omnios/incorporation/jeos/omnios-userland@11,5.11-0.151006:20130506T214442Z found: Reject: pkg://omnios/incorporation/jeos/omnios-userland@11,5.11-0.151006:20130506T214442Z Reason: A version for 'incorporate' dependency on pkg:/archiver/gnu-tar@1.26,5.11-0.151006 cannot be found and for openssl, I see this: root@massive02-b:~# pkg info library/security/openssl Name: library/security/openssl Summary: openssl - A toolkit for Secure Sockets Layer (SSL v2/v3) and Transport Layer (TLS v1) protocols and general purpose cryptographic library State: Installed Publisher: omnios Version: 1.0.1.10 (1.0.1j) Build Release: 5.11 Branch: 0.151006 Packaging Date: October 15, 2014 03:40:22 PM Size: 18.08 MB FMRI: pkg://omnios/library/security/openssl@1.0.1.10,5.11-0.151006:20141015T154022Z root@massive02-b:~# pkg install pkg:/library/security/openssl@1.0.1,5.11-0.151008 Creating Plan / pkg install: No matching version of library/security/openssl can be installed: Reject: pkg://omnios/library/security/openssl@1.0.1.5,5.11-0.151008:20131204T024253Z pkg://omnios/library/security/openssl@1.0.1.6,5.11-0.151008:20140110T173804Z pkg://omnios/library/security/openssl@1.0.1.7,5.11-0.151008:20140407T220403Z pkg://omnios/library/security/openssl@1.0.1.7,5.11-0.151008:20140408T142844Z pkg://omnios/library/security/openssl@1.0.1.8,5.11-0.151008:20140605T140630Z pkg://omnios/library/security/openssl@1.0.1.9,5.11-0.151008:20140807T035111Z Reason: Newer version pkg://omnios/library/security/openssl@1.0.1.10,5.11-0.151006:20141015T154022Z is already installed Ian On Wed, Dec 10, 2014 at 10:15 AM, Dan McDonald wrote: > >> On Dec 10, 2014, at 12:01 PM, Ian Kaufman wrote: >> >> Hi all, >> >> I have a situation where a bunch of libs are complaining about missing >> the following: >> >> libc.so.1 (ILLUMOS_0.7) >> >> or >> >> libc.so.1 (ILLUMOS_0.6) >> >> This is happening on one system, but I have others that are fine. A >> standard libc.so.1 exists, but various libs are looking for the tagged >> one. >> >> Any ideas? > > You'll need to provide more data. It's as if something you compiled was > compiled against a later version of libc, and then moved to an earlier > version of libc. > > Dan > -- Ian Kaufman Research Systems Administrator UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
[OmniOS-discuss] weird library issue
Hi all, I have a situation where a bunch of libs are complaining about missing the following: libc.so.1 (ILLUMOS_0.7) or libc.so.1 (ILLUMOS_0.6) This is happening on one system, but I have others that are fine. A standard libc.so.1 exists, but various libs are looking for the tagged one. Any ideas? Thanks, Ian -- Ian Kaufman Research Systems Administrator UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] NFSv4 id mapping only working on client but not server?
Yes, I oversimplified things. The issue is that AUTH_SYS/AUTH_UNIX does not support NFSv4. AUTH_SYS uses the UID, not the name, so the mapping fails. So, when RPC uses AUTH_SYS, NFSv4 is SOL. http://thread.gmane.org/gmane.linux.nfsv4/7103/focus=7105 Supposedly. at least on the client side, this has been fixed somewhere upstream. However, the server side is not. https://bugzilla.linux-nfs.org/show_bug.cgi?id=226 Regardless, my point is that this is not a Solaris/Linux issue, as a Linux server and Linux clients would be in the same boat. On Mon, Dec 8, 2014 at 12:23 PM, Paul B. Henson wrote: >> From: Ian Kaufman >> Sent: Monday, December 08, 2014 8:54 AM >> >> No, what we are saying is NFSv4 and RPC are not compatible right now, >> and thus AUTH_SYS/AUTH_UNIX will not map UIDs by name, but by number >> over RPC. If you are not going to use Kerberos or LDAP, then you need >> to sync UIDs/UIDNumber in /etc/passwd. It's not that Solaris and Linux >> are incompatible, it's that RPC does not support NFSv4. > > Well, I don't think it's technically accurate to say NFSv4 and RPC are not > compatible, nor that RPC does not support NFSv4. > > It's really more of an issue that the NFSv4 protocol failed to address ID > mapping completely for nonsecure RPC methods. For secure RPC (such as > Kerberos), identifiers are passed over the wire symbolically as strings, and > the ID mapper on each side translates them to numeric identifiers. However, > for insecure RPC using AUTH_SYS, NFSv4 continues to pass raw uid/gid > identifiers over the wire. > > This limitation of the protocol renders ID mapping somewhat useless for > NFSv4 over insecure RPC :(. Ideally they could address it in v4.1 or v4.2 by > creating a new insecure AUTH_SYS_NAMES protocol, which would simply use > symbolic string identifiers over the wire like secure NFS does. I guess > there is not much interest or concern about this in the communities that > deal with NFS protocol specifications 8-/. Perhaps the majority of > deployments for people in decision-making capacity use secure NFS and thus > are not impacted by this deficiency... > > -- Ian Kaufman Research Systems Administrator UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] NFSv4 id mapping only working on client but not server?
No, what we are saying is NFSv4 and RPC are not compatible right now, and thus AUTH_SYS/AUTH_UNIX will not map UIDs by name, but by number over RPC. If you are not going to use Kerberos or LDAP, then you need to sync UIDs/UIDNumber in /etc/passwd. It's not that Solaris and Linux are incompatible, it's that RPC does not support NFSv4. Ian On Mon, Dec 8, 2014 at 5:25 AM, John Klimek wrote: > Thanks everybody. > > Are you guys saying that Solaris and Linux are incompatible for sharing > NFSv4 ACLs (because of RPC differences, etc) ? > > On Sun, Dec 7, 2014 at 5:43 PM, Frank Pittel wrote: >> >> I've tried doing this with NFS4 a couple of years ago and what I found was >> that >> while the UID mapping worked for things like "ls" anytime you tried to >> actually >> do anything with the file RPC calls were made and since that's outside of >> NFS they >> failed. >> >> It was a couple of years ago now since I went through it but the upshot is >> that >> at least then uid mapping didn't work in any meaningful way! >> >> The Other Frank >> >> >> >> On Thu, Dec 04, 2014 at 01:55:11PM -0500, John Klimek wrote: >> > Thanks, I will give this a try. >> > >> > However, when I create a file if I check the acl on Omni, it does have >> > extended permissions and looks correct. It's only the uid and gid that >> > I'm >> > trying to get mapped correctly. >> > >> > Do you still think that noacl will solve that problem? >> > >> > (I'm migrating some virtual machines so I can't try out the setting you >> > mentioned until a few more minutes...) >> > >> > On Thu, Dec 4, 2014 at 1:37 PM, Michael Rasmussen wrote: >> > >> > > On Thu, 4 Dec 2014 13:20:04 -0500 >> > > John Klimek wrote: >> > > >> > > > >> > > > What can I have setup wrong? >> > > > >> > > > I can't find any debugging or logging options for nfsmapid... >> > > > >> > > > Also, does the uidmap and gidmap share.nfs options only work for >> > > > NFSv3? >> > > Solaris and derivatives implementation of NFS ACL is not compliant to >> > > the >> > > Linux NFS ACL. >> > > More directly it is the ACL in Linux which is not POSIX conformant so >> > > to >> > > avoid problems you >> > > should add the mount option noacl in your Linux fstab file. Noacl will >> > > instruct Omnios NFS >> > > to revert to plain old uid/gid. >> > > >> > > -- >> > > Hilsen/Regards >> > > Michael Rasmussen >> > > >> > > Get my public GnuPG keys: >> > > michael rasmussen cc >> > > http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E >> > > mir datanom net >> > > http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C >> > > mir miras org >> > > http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 >> > > -- >> > > /usr/games/fortune -es says: >> > > Don't just echo the code with comments - make every comment count. >> > > - The Elements of Programming Style (Kernighan & Plaugher) >> > > >> > > ___ >> > > OmniOS-discuss mailing list >> > > OmniOS-discuss@lists.omniti.com >> > > http://lists.omniti.com/mailman/listinfo/omnios-discuss >> > > >> > > >> >> > ___ >> > OmniOS-discuss mailing list >> > OmniOS-discuss@lists.omniti.com >> > http://lists.omniti.com/mailman/listinfo/omnios-discuss >> > > > ___ > OmniOS-discuss mailing list > OmniOS-discuss@lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > -- Ian Kaufman Research Systems Administrator UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] NFSv4 id mapping only working on client but not server?
As far as I recall, AUTH_UNIX (aka AUTH_SYS) uses RPC, and RPC has not been augmented to support NFSv4 yet. >From an old thread in the OpenSolaris boards: - Note also, that although NFSv4 uses strings for uid/gid/domain, the underlying RPC layer uses the same authentication credentials as in previous NFS versions and other RPC programs. Since it's the RPC authentication that is used to control access, etc, don't expect too much from the NFSv4 name/id mapping. It's useful for ls -l listings, etc, but not for authentication. - On Fri, Dec 5, 2014 at 5:32 AM, John Klimek wrote: > Paul, thanks! You understand exactly what I'm asking and what my problem > is. > > Something seems to be going wrong (or incorrectly configured) with the NFS > id mapping services on my Linux client or OmniOS server. > > I've setup the nfsmapid_domain and I believe it's working because before > configuring it, everything was showing as nobody/nobody, but afterwards it > shows the correct names. (on the client) I can also see in the client log > and it's correctly mapping u...@domain.com in the Linux rpc.idmapd domain. > > The problem is that when I create a file on the client, the server (OmniOS) > is showing the UID of the user that created the file instead of mapping it > using the name to the local UID. > > It looks like I can avoid the problem if I synchronize UIDs and GIDs across > all servers (it's a small home network so it's not a big problem and I'm not > using LDAP or NIS), but NFSv4 was supposed to avoid this problem by using > name mapping so you didn't have to synchronize UIDs/GIDs. > > > > On Thu, Dec 4, 2014 at 3:59 PM, Paul B. Henson wrote: >> >> > From: Michael Rasmussen >> > Sent: Thursday, December 04, 2014 11:11 AM >> > >> > Yes, because you want to avoid Omnios presents ACL which is incompatible >> > with Linux ACL. >> >> I don't believe the ACL has anything to do with NFSv4 id mapping? And a >> ZFS >> ACL presented over NFSv4 is perfectly compatible with Linux. It's not a >> Linux POSIX ACL, and cannot be manipulated with getfacl/setfacl, you need >> to >> use the nfs4 ACL tools, but it works fine. >> >> > http://forum.proxmox.com/threads/15793-CT-creation-on-NFS- >> > Share?p=81530#post81530 >> >> In that thread, the user fails to chmod via NFS: >> >> chmod: changing permissions of `/mnt/pve/proxCT/private/108.tmp': >> Operation >> not permitted >> >> The root cause of which was a setting of restricted for aclmode: >> >> vdev1/proxCT aclmode restricted >> local >> >> Per the man page "An aclmode property of restricted will cause the >> chmod(2) >> operation to return an error when used on any file or directory which has >> a >> non-trivial ACL whose entries can not be represented by a mode." >> >> The user could have set the inherited ACL on the initial filesystem to a >> trivial ACL, in which case chmod would've worked fine over NFS. >> >> In any case, I don't see anything in that thread that seems relevant to >> NFSv4 id mapping, which unless I misunderstand is the problem the OP is >> trying to resolve. >> >> On that subject, NFSv4 id mapping seems to be working fine for me between >> an >> omnios client and server. On the server, the file system is mounted as: >> >> /export/user/henson on export/user/henson >> read/write/nosetuid/nodevices/nonbmand/exec/xattr/atime/dev=2c5025c >> >> And exported as: >> >> /export/user/henson - nfs nosuid,sec=krb5i,sec=krb5p >> >> with the domain set: >> >> $ sharectl get -p nfsmapid_domain nfs >> nfsmapid_domain=csupomona.edu >> >> if I create a file on the server, it has the correct ownership: >> >> $ touch test_server >> $ ls -l test_server >> -rw-r--r--+ 1 henson csupomona 0 Dec 4 12:50 test_server >> >> on the client, the NFS export is mounted as: >> >> /mnt on files-www.csupomona.edu:/export/user/henson >> remote/read/write/setuid/devices/sec=krb5p/xattr/dev=85c0008 on Thu Dec 4 >> 12:50:01 2014 >> >> the client has the same domain: >> >> $ sharectl get -p nfsmapid_domain nfs >> nfsmapid_domain=csupomona.edu >> >> The file created on the server shows up with the correct ownership: >> >> $ ls -l test_server >> -rw-r--r--+ 1 henson csupomona 0 Dec 4 12:50 test_server >> >> A file created on the client has t
Re: [OmniOS-discuss] Ang: Re: Ang: Re: infiniband
I wonder if Nexenta has added more/better support for IB? I believe that Illumian 1.0 is finally out (after having run the Alpha for years), and I know that they did have some interest in supporting IB originally. If they have, then I would hope some of that support would make it back upstream. Mayhaps I have an email to send out ... Ian On Fri, Oct 24, 2014 at 7:58 AM, Ian Kaufman wrote: > I use a QLogic 12300 with SM built in. I haven't had any issues, > saving my cluster frontend's cycles for other things. > > Ian > > On Fri, Oct 24, 2014 at 7:55 AM, Michael Rasmussen wrote: >> On Fri, 24 Oct 2014 13:17:39 +0200 >> Johan Kragsterman wrote: >> >>> >>> Well, I can only find one person in that thread that BELIEVES that running >>> SM on a switch is preferable to run OpenSM software. He hardly comes with >>> any documentation of this, and I can't see any other that agrees... >>> >>> But of coarse, if you provide too little resources for OpenSM it will cause >>> problems, or if you have any problems with the S/W or the configuration... >>> >> I think the issue here is that OpenSM is a single threaded application >> so when the CPU which is assigned to it maxes out performance will >> drop. So I guess it comes down to a trade-off between the >> performance of the host CPU and the performance of the switch hardware. >> >> -- >> Hilsen/Regards >> Michael Rasmussen >> >> Get my public GnuPG keys: >> michael rasmussen cc >> http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E >> mir datanom net >> http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C >> mir miras org >> http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 >> -- >> /usr/games/fortune -es says: >> The default Magic Word, "Abracadabra", actually is a corruption of the >> Hebrew phrase "ha-Bracha dab'ra" which means "pronounce the blessing". >> >> ___________ >> OmniOS-discuss mailing list >> OmniOS-discuss@lists.omniti.com >> http://lists.omniti.com/mailman/listinfo/omnios-discuss >> > > > > -- > Ian Kaufman > Research Systems Administrator > UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu -- Ian Kaufman Research Systems Administrator UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] Ang: Re: Ang: Re: infiniband
I use a QLogic 12300 with SM built in. I haven't had any issues, saving my cluster frontend's cycles for other things. Ian On Fri, Oct 24, 2014 at 7:55 AM, Michael Rasmussen wrote: > On Fri, 24 Oct 2014 13:17:39 +0200 > Johan Kragsterman wrote: > >> >> Well, I can only find one person in that thread that BELIEVES that running >> SM on a switch is preferable to run OpenSM software. He hardly comes with >> any documentation of this, and I can't see any other that agrees... >> >> But of coarse, if you provide too little resources for OpenSM it will cause >> problems, or if you have any problems with the S/W or the configuration... >> > I think the issue here is that OpenSM is a single threaded application > so when the CPU which is assigned to it maxes out performance will > drop. So I guess it comes down to a trade-off between the > performance of the host CPU and the performance of the switch hardware. > > -- > Hilsen/Regards > Michael Rasmussen > > Get my public GnuPG keys: > michael rasmussen cc > http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E > mir datanom net > http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C > mir miras org > http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 > -- > /usr/games/fortune -es says: > The default Magic Word, "Abracadabra", actually is a corruption of the > Hebrew phrase "ha-Bracha dab'ra" which means "pronounce the blessing". > > ___ > OmniOS-discuss mailing list > OmniOS-discuss@lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > -- Ian Kaufman Research Systems Administrator UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] infiniband
Mellanox ConnectX-2 cards came in SDR (10Gbps) , DDR (20Gbps), and QDR (40Gpbs) configs. Ian On Thu, Oct 23, 2014 at 5:57 PM, Michael Rasmussen wrote: > On Fri, 24 Oct 2014 11:51:30 +1100 > David Bomba wrote: > >> I have the exact same issue Ian. >> >> I have multiple XenServers connected to OmniOS storage units. Under >> specific circumstances, I get iscsi disconnects which prove fatal for the >> XenServer host guests. These machines run a mixed bag of ConnectX and >> ConnectX-2 cards >> >> I have managed to get the system stable, but with severe compromises. >> >> Previously using 10Gb cards, I didnt have any problems what so ever. >> > ConnectX and ConnectX-2 is 40 gpbs cards? > > -- > Hilsen/Regards > Michael Rasmussen > > Get my public GnuPG keys: > michael rasmussen cc > http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E > mir datanom net > http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C > mir miras org > http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 > -- > /usr/games/fortune -es says: > Indifference will certainly be the downfall of mankind, but who cares? > > _______ > OmniOS-discuss mailing list > OmniOS-discuss@lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > -- Ian Kaufman Research Systems Administrator UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] infiniband
I have had an issue with the Hermon driver and Mellanox ConnectX2 QDR cards. On one system, I saw this in the log: massive02-a hermon: [ID 492207 kern.info] hermon1: CQE RNR retry counter exceeded and the card went offline. Now, the interface does not even come up. Three other identical systems are fine, as are two other systems with the same card running Illumian 1.0a. I haven't found a way to reset the RNR counter. Ian On Thu, Oct 23, 2014 at 4:49 PM, Michael Rasmussen wrote: > On Thu, 23 Oct 2014 19:25:47 -0400 > Dan McDonald wrote: > >> >> Only older cards. >> > By older cards do you the mean only 10 gbps cards? > >> > It is these cards I had in mind: >> > Voltaire 500 EX-D Dual Port Host Channel Adapter PCIe 20Gbps DDR >> > InfiniBand Card >> >> Do you know the PCI ID of that card? I can double-check, but I'm guessing >> the answer will be no. >> >> Dan >> > prtconf -v from Solaris 11.1 > https://forums.servethehome.com/index.php?threads/connectx-vpi-infiniband-card-and-solaris-11-1.3211/ > > -- > Hilsen/Regards > Michael Rasmussen > > Get my public GnuPG keys: > michael rasmussen cc > http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E > mir datanom net > http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C > mir miras org > http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 > -- > /usr/games/fortune -es says: > [Sir Stafford Cripps] has all the virtues I dislike and none of the > vices I admire. > -- Winston Churchill > > _______ > OmniOS-discuss mailing list > OmniOS-discuss@lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > -- Ian Kaufman Research Systems Administrator UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] Fault Manager component could not load
AFAIK, this has not been implemented yet in the Open Source community. Ian On Mon, Sep 22, 2014 at 5:44 PM, Matthew Lagoe wrote: > I tried to add "setprop temp-multiple 0" to > /usr/lib/fm/fmd/plugins/disk-transport.conf due to a bug in Seagate drives, > however when I start up the machine with that configuration in it I get the > following error, if anyone has any ideas what the cause could be. > > Thanks > > --- -- > - > TIMEEVENT-ID MSG-ID > SEVERITY > --- -- > - > Sep 22 17:10:10 e0719be1-fe87-6a27-cb4c-aaf2f8902a13 FMD-8000-3FMinor > > > Host: stor10 > Platform: PowerEdge-R720Chassis_id : FCNH3W1 > Product_sn : > > Fault class : defect.sunos.fmd.config > Affects : fmd:///module/disk-transport > faulted and taken out of service > FRU : None > faulty > > Description : A Solaris Fault Manager component could not load due to an > erroroneous configuration file. Refer to > http://illumos.org/msg/FMD-8000-3F for more information. > > Response: The module has been disabled. Events destined for the module > will be saved for manual diagnosis. > > Impact : Automated diagnosis and response for subsequent events > associated > with this module will not occur. > > Action : Use fmdump -v -u to locate the module. Use fmadm load >to load the module after repairing its configuration. > > > > > > _______ > OmniOS-discuss mailing list > OmniOS-discuss@lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss -- Ian Kaufman Research Systems Administrator UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] How do non-rpool ZFS filesystems get mounted?
> It doesn't. All of /fs3-test-01, /fs3-test-02, /h/281, and /h/999 > are empty before 'zfs mount -a' runs (I've verified this with ls's > immediately before the 'zfs mount -a' in /lib/svc/method/fs-local). > As a test, try renaming those "empty" directories and then reboot. We saw this issue with Solaris 10, where on reboot, the filesystems did not unmount cleanly, and failed to mount at boot. Ian -- Ian Kaufman Research Systems Administrator UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] Illumos and Infiniband
Heh - mistype. But that cease and desist we sent to IPoIB ... :) On Wed, Jan 29, 2014 at 11:42 AM, Dan Swartzendruber wrote: >> On my systems, the X-2 cards claim 32 gbps. I sued IPoIB, and have >> seen transfers NFS transfers over 6 gbps and snapshot send/receive as >> high as 1.8 gbps while the system was simultaneously utilized during >> cluster computation. >> >> Ian > > I assume you meant 'used', not 'sued'? If not, your lawyer is in the > wrong line of work :) > -- Ian Kaufman Research Systems Administrator UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] Illumos and Infiniband
On my systems, the X-2 cards claim 32 gbps. I sued IPoIB, and have seen transfers NFS transfers over 6 gbps and snapshot send/receive as high as 1.8 gbps while the system was simultaneously utilized during cluster computation. Ian On Wed, Jan 29, 2014 at 11:17 AM, David Bomba wrote: > Hi Chris, > > We have an environment running OmniOS with both ConnectX and ConnectX-2 > cards. The Hermon and Tavor drivers work very well. > > The only problem we had was that the CX-2 cards would only sync at 20gb/s. > We export Comstar iSER targets and they have been rock solid for us for over > a year. > > Dave > > > On 30 January 2014 05:38, Chris Zembower wrote: >> >> >> Hello, >> >> I'm trying to dig up some information on Illumos (or Solaris) >> compatibility with the Mellanox ConnectX-3 VPI HCA. From what I can gather, >> the Hermon driver (for 1st-gen ConnectX) may work with this hardware with >> tweaking, but I can't seem to confirm that. >> >> OmniOS is my server OS of choice (I have several systems running), but the >> necessity of a working IB fabric here has led me to implement a ZFS on Linux >> box for this particular purpose. I'm currently using LIO/targetcli to export >> block SRP targets, and I'm not happy with it at all. Really wishing I had >> Comstar for this. >> >> Anyone here using IB with OmniOS? Anyone aware of a roadmap or active >> development of new IB drivers? Why isn't there an Illumos OFED port >> happening? No interest? It's thriving on the Linux side of the world... >> >> Please don't lecture me on the merits of 10GbE/FC over Infiniband, I've >> heard it all. :-) >> >> Any insights or suggestions are very much appreciated! >> >> Best, >> Chris >> >> ___ >> OmniOS-discuss mailing list >> OmniOS-discuss@lists.omniti.com >> http://lists.omniti.com/mailman/listinfo/omnios-discuss >> > > > ___ > OmniOS-discuss mailing list > OmniOS-discuss@lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > -- Ian Kaufman Research Systems Administrator UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] Illumos and Infiniband
I use OminOS and IB, and ran into the ConnectX-3 driver issues as we were moving towards interfaces that can use QDR, FDR, 10GbE, or 40GbE, depending on drivers and fabric. We had to go back to ConnectX-2 QDR to get it all working. I was able to hack things to at least get the NIC identified, but obviously the drivers were not there. I was contemplating building OFED from source, but I don't have the time. Ian On Wed, Jan 29, 2014 at 10:50 AM, Saso Kiselkov wrote: > On 1/29/14, 6:38 PM, Chris Zembower wrote: >> Please don't lecture me on the merits of 10GbE/FC over Infiniband, I've >> heard it all. :-) > > You're not alone in your appreciation of IB's brute strength. As for why > stuff isn't happening, my guess is lack of manpower - we're a really > small community and people work on what they like, plus what they can > get their hands on (and IB gear is a little exotic). That being said, > I'd love to have a mature and up-to-date IB stack on Illumos that could > be used in production. It's sorely needed. > > Cheers, > -- > Saso > ___ > OmniOS-discuss mailing list > OmniOS-discuss@lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss -- Ian Kaufman Research Systems Administrator UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] Newbie to OmniOS
Oops, yup, my bad. For some reason, I was using 129.82.29.15 as the host ... Ian On Mon, Jan 13, 2014 at 3:35 PM, CJ Keist wrote: > Nope, >255.255.252.0/22 is 1022 host sub-net. > > > > > On 1/13/14, 4:21 PM, Ian Kaufman wrote: >> >> If the subnet mask is 255.255.252.0, the broadcast should be >> 129.82.31.255. >> >> Ian >> >> On Mon, Jan 13, 2014 at 3:02 PM, CJ Keist wrote: >>> >>> Michael, >>> Broadcast is set right (255.255.252.0) so 129.82.23.255 is correct >>> for >>> the broadcast. >>> The 129.82.20.0 129.82.20.15 in the route table is consistent with >>> other Solaris servers I have on the network (non-virtual). >>> >>> >>> >>> On 1/13/14, 3:45 PM, Michael Rasmussen wrote: >>>> >>>> >>>> On Mon, 13 Jan 2014 15:03:40 -0700 >>>> CJ Keist wrote: >>>> >>>>> Excuse the pics, but don't have cut paste ability while working through >>>>> the java console. Here is all the network info I believe. >>>>> >>>> I think you have entered wrong network data. >>>> IP 129.82.20.15 >>>> GW 129.82.20.1 >>>> BA 129.82.23.255 >>>> NW 129.82.20.0 ??? >>>> >>>> Something does not compute here. You are sure your broadcast address >>>> should not have been 129.82.20.255 instead of 129.82.23.255? >>>> >>>> >>>> >>>> ___ >>>> OmniOS-discuss mailing list >>>> OmniOS-discuss@lists.omniti.com >>>> http://lists.omniti.com/mailman/listinfo/omnios-discuss >>>> >>> >>> -- >>> C. J. Keist Email: cj.ke...@colostate.edu >>> Systems Group Manager Solaris 10 OS (SAI) >>> Engineering Network ServicesPhone: 970-491-0630 >>> College of Engineering, CSU Fax: 970-491-5569 >>> Ft. Collins, CO 80523-1301 >>> >>> All I want is a chance to prove 'Money can't buy happiness' >>> ___ >>> OmniOS-discuss mailing list >>> OmniOS-discuss@lists.omniti.com >>> http://lists.omniti.com/mailman/listinfo/omnios-discuss >> >> >> >> > > -- > C. J. Keist Email: cj.ke...@colostate.edu > Systems Group Manager Solaris 10 OS (SAI) > Engineering Network ServicesPhone: 970-491-0630 > College of Engineering, CSU Fax: 970-491-5569 > Ft. Collins, CO 80523-1301 > > All I want is a chance to prove 'Money can't buy happiness' -- Ian Kaufman Research Systems Administrator UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] Newbie to OmniOS
If the subnet mask is 255.255.252.0, the broadcast should be 129.82.31.255. Ian On Mon, Jan 13, 2014 at 3:02 PM, CJ Keist wrote: > Michael, > Broadcast is set right (255.255.252.0) so 129.82.23.255 is correct for > the broadcast. > The 129.82.20.0 129.82.20.15 in the route table is consistent with > other Solaris servers I have on the network (non-virtual). > > > > On 1/13/14, 3:45 PM, Michael Rasmussen wrote: >> >> On Mon, 13 Jan 2014 15:03:40 -0700 >> CJ Keist wrote: >> >>> Excuse the pics, but don't have cut paste ability while working through >>> the java console. Here is all the network info I believe. >>> >> I think you have entered wrong network data. >> IP 129.82.20.15 >> GW 129.82.20.1 >> BA 129.82.23.255 >> NW 129.82.20.0 ??? >> >> Something does not compute here. You are sure your broadcast address >> should not have been 129.82.20.255 instead of 129.82.23.255? >> >> >> >> ___ >> OmniOS-discuss mailing list >> OmniOS-discuss@lists.omniti.com >> http://lists.omniti.com/mailman/listinfo/omnios-discuss >> > > -- > C. J. Keist Email: cj.ke...@colostate.edu > Systems Group Manager Solaris 10 OS (SAI) > Engineering Network ServicesPhone: 970-491-0630 > College of Engineering, CSU Fax: 970-491-5569 > Ft. Collins, CO 80523-1301 > > All I want is a chance to prove 'Money can't buy happiness' > ___ > OmniOS-discuss mailing list > OmniOS-discuss@lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss -- Ian Kaufman Research Systems Administrator UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] Physical slot based disk names for LSI SAS on OmniOS?
You can also use the sas2ircu directly via the command line. However,
Günther's efforts and integration into the napp-it GUI make it so much
easier to use.
Ian
On Thu, Oct 31, 2013 at 2:37 PM, alka wrote:
> hi Felix
>
> its part of the monitor extension
> (free for less than 8 disks)
>
> see Menu disks >> SAS2 extension
>
>
> Am 31.10.2013 um 19:26 schrieb Felix Nielsen:
>
> Hi Günther,
>
> Is that feature included in napp-it? If so where and if not how do I get it
> :)
>
> Thanks
> Felix
>
> Btw. napp-it rocks :)
>
> Den 31/10/2013 kl. 19.03 skrev Günther Alka :
>
> I have added a physical slot detection that displays physical
>
> Slot, WWN and serials together with the option to switch on the red
>
> alert led on supported backplanes within napp-it with the help of
>
> sas2ircu (a LSI tool that displays slot and disk serial).
>
>
>
> On 30.10.2013 21:25, Chris Siebenmann wrote:
>
> This is a long shot and I suspect that the answer is no, but: in
>
> OmniOS, is it possible somehow to have disk device names for disks
>
> behind LSI SAS controllers that are based on the physical slot ('phy')
>
> that the disk is found in instead of the disk's reported WNN or serial
>
> number?
>
>
> (I understand that this is only easy if there are no SAS expanders
>
> involved, which is the situation that I care about.)
>
>
> I know that I can recover this information from prtconf -v with
>
> appropriate mangling, but our contemplated environment would be much
>
> more manageable if we could directly use physical slot based device
>
> naming.
>
>
> Thanks in advance. I'll post a summary if there are any private
>
> replies (for anything besides 'nope, can't do it').
>
>
> - cks
>
> ___
>
> OmniOS-discuss mailing list
>
> OmniOS-discuss@lists.omniti.com
>
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
>
>
> ___
>
> OmniOS-discuss mailing list
>
> OmniOS-discuss@lists.omniti.com
>
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
> ___
> OmniOS-discuss mailing list
> OmniOS-discuss@lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
>
> --
>
>
>
> ___
> OmniOS-discuss mailing list
> OmniOS-discuss@lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
--
Ian Kaufman
Research Systems Administrator
UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu
___
OmniOS-discuss mailing list
OmniOS-discuss@lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] rsync issues
Hi all, It appears that the issue is the default set of ACLs created when setting up the filesystems. I will dig a little deeper, but I wound up deleting one of the smaller filesystems, and then doing a ZFS send/receive to recreate it from the Illumian 1.0a5 server. The ACLs changed to match the Illumian systems, and rsync appears to behave as expected now. Ian On Tue, Sep 10, 2013 at 1:27 PM, Tim Rice wrote: > On Mon, 9 Sep 2013, Ian Kaufman wrote: > >> Hi all, >> >> I am trying to slurp data over from one IllumOS based system >> (Illumian) to another (latest stable OmniOS), and I am running into a >> problem. I keep seeing the error: >> >> rsync: failed to set permissions on "FILENAME": Not owner (1) >> >> which is leaving me with all of my dirs/files either 777 or 666. I am >> using the -a flag, both as root and as the directory/file owner, and >> the namespace/user ids are consistent across the system. My idmapd >> domain is also consistent. >> >> Has anyone else seen this problem? > > I just go through migrating the data from my UnixWare server to > OmniOS (omnios-b281e50) and saw a very similar error message. I say > similar because if I remember correctly it came from tar not rsync. > Then doing a rsync tuned up things nicely. The problem files all seemed > to be suid root. > >> >> Thanks, >> >> Ian >> > > -- > Tim RiceMultitalents > t...@multitalents.net > > -- Ian Kaufman Research Systems Administrator UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] My NFS low performance test.
Hi Muhammad, What is the output of "sharectl get nfs"? Have you done any NFS tuning? Even though the system says you have 7GB of RAM free, you are running a very slim system for ZFS. Do you have compression enabled? Deduplication? How big is your filesystem? Here are some RAM rules of thumb for ZFS (anfd other advice): For decent ZFS performance, you need at least 1GB of RAM for every TB of data, plus at least 1 extra GB for OS If deduplication is turned on, you need about 4GB of RAM per TB of data in addition to normal system and ZFS RAM if using 64K blocks on average, if using smaller 8K blocks on average, then you need about 32GB of RAM per TB of data. RAIDZ2 vdevs should have at least 5 disks and no more than 10. Some believe 6 or 8 disks per vdev for RAIDZ2 is optimal. Improve read performance with L2ARC and SSDs - improve write performance with either RAMDISK SSDs or SLC Flash Memory SSDs, and mirror the devices. Ian On Wed, Sep 11, 2013 at 11:42 PM, Muhammad Yousuf Khan wrote: > Just notice and would like to share my new finding. > > as i mentioned earlier i am using RAIDz2 and hosted a VM on NFS. so for > testing i "SCP" the whole HD image of VM to some other Linux system so that > i can learn if the issue is with Drive speed or the issue is with NFS > protocol, (of course not a bug with NFS but i have done something wrong in > configuring) > > here is the SCP result. the virtual harddisk image size is 32 GB. > i am running this command from OmniOS SSH. > > > vm-11-disk-1.qco 50% |** | 16461 MB09:42 > ETA > > > you can see it copy 16 GB and i confirm that it took 3 to 4 minutes to copy > that huge data. > > it seems fine to me. same RAIDz2 but protocol change (SCP on SSH) so i > think the issue is with NFS. not with the Hardware speed/specs limitation. > > any idea please. > > Thanks, > Myk > > > > On Thu, Sep 12, 2013 at 2:40 AM, Muhammad Yousuf Khan > wrote: >> >> first of sorry for starting a new thread for same issue explained earlier >> because. i >> mistakenly deleted the email. so my apologizes. >> >> my omniOS server hardware is as under. >> dual 3.2GHz Processors, 12 GB RAM. dell 490 workstation >> 4 sata drives set with RAIDz2 >> 1 40GB very old IDE drive for OmniOS root system >> >> i have 3 servers for testing. >> 1. KVM virtualization server (Proxmox) >> 2. OmniOS (NFS) >> 3. Desktop machine >> >> all three are connected via 1GB LAN broadcast domain. >> >> i have hosted a test VM (windows 2003 server) on NFS and when i am trying >> to copy the data (2gb File) from VM via Microsoft sharing to a desktop >> machine. i doing something very abnormal which i can not understand why >> >> just FYI : all the virtual machines hosted on KVM (proxmox server) local >> harddrives are on mdadm linux raid. and working great. and i see no issue in >> copy and pasting data from or to the local VM machines. >> >> In attached graphics you will see 2 test of sending 2GB file. >> in "snap a.1" for few seconds file transfer stuck at 2 percent and then >> due to unknown reasons bumps to 15%. and it continues for few seconds and >> when file reaches the end it again goes down to 3 Percent as you can see in >> "snap a.2" >> >> in "snap b.1" you can see it starts at 3% max and for 2 minutes it >> continues and speed only changes btw 1% to 3%. see in "snap b.2" >> >> some says disable sync for testing and see if performance increases, sorry >> it didnt help me. same thing happens, whether i disable or enable the sync >> it makes no difference for me >> >> and Some says to upgrade the RAM which i am already looking but just a >> part of my confusion when i see vmstat it shows me i have 7GB free RAM >> >> is there anyone can please explain me why and what is happening behind all >> this, the workstation hosting OmniOS is a good machine. why it is doing some >> thing very unexpected. >> >> Any advice, suggestion, tip etc. will be highly appreciated. >> >> Thanks >> Myk >> > > > ___ > OmniOS-discuss mailing list > OmniOS-discuss@lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > -- Ian Kaufman Research Systems Administrator UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] rsync issues
Sadly, no, nothing is immutable as I was able to rsync between Illumian boxes without any problems. I think something is wrong with rsync in OmniOS. Ian On Mon, Sep 9, 2013 at 2:00 PM, Jim Klimov wrote: > On 2013-09-09 20:40, Ian Kaufman wrote: >> >> Hi all, >> >> I am trying to slurp data over from one IllumOS based system >> (Illumian) to another (latest stable OmniOS), and I am running into a >> problem. I keep seeing the error: >> >> rsync: failed to set permissions on "FILENAME": Not owner (1) >> >> which is leaving me with all of my dirs/files either 777 or 666. I am >> using the -a flag, both as root and as the directory/file owner, and >> the namespace/user ids are consistent across the system. My idmapd >> domain is also consistent. > > > One wild guess, especially if you are root on local system, is that > maybe the original FS objects had extended attributes like immutable, > and if that is set on a directory - that might somehow forbid further > receiving of files into it? > > This is a "wild guess" again, but if it is right - you might want to > retry, by receiving data and hard/symlinks first (without much care > for attributes), and then pass over this with a more complete rsync > option set, which would set FS attributes as well. > > HTH, > //Jim > > _______ > OmniOS-discuss mailing list > OmniOS-discuss@lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss -- Ian Kaufman Research Systems Administrator UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
[OmniOS-discuss] rsync issues
Hi all, I am trying to slurp data over from one IllumOS based system (Illumian) to another (latest stable OmniOS), and I am running into a problem. I keep seeing the error: rsync: failed to set permissions on "FILENAME": Not owner (1) which is leaving me with all of my dirs/files either 777 or 666. I am using the -a flag, both as root and as the directory/file owner, and the namespace/user ids are consistent across the system. My idmapd domain is also consistent. Has anyone else seen this problem? Thanks, Ian -- Ian Kaufman Research Systems Administrator UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] ldap auth
Quick question - are you restricting it to ONLY TLS/SSL LDAP over port 636, essentially shutting down port 389 communication? I beat my head against the wall back in Solaris 10 as well. Apparently, the LDAP cache manager needs to communicate over 389. What I finally resorted to was installing OpenLDAP and setting up my Solaris 10 systems as read only LDAP slaves, and then used the native LDAP client to talk to the local OpenLDAP server over port 389 using the loopback interface. Ian On Thu, Sep 5, 2013 at 2:44 AM, Thierry Bingen wrote: > On Tue, 03 Sep 2013 12:17:07 -0700, Paul B. Henson kindly answered: > >> On 9/2/2013 7:17 AM, Thierry Bingen wrote: >> >>> Suffering from exactly the same problem (LDAP bind failing after upgrading >>> from r151004 to r151006), I tried your recipe; my /etc/default/init now >>> contains: >>> >>> TZ="Europe/Brussels" >>> CMASK=022 >>> NSS_HASH_ALG_SUPPORT=+MD5 >>> >>> but it did not make any difference after reboot, e.g.: >>> >>> # ldapsearch -h ldap.xxx.net -p 636 -Z -v -P /var/ldap/cert8.db -D >>> "cn=Directory Manager" -b "dc=xxx,dc=net" "cn=Thierry Bingen" >>> ldapsearch: started Mon Sep 2 15:29:40 2013 ldap_init( ldap.xxx.net, 636 ) >>> ldap_simple_bind: Can't contact LDAP server >> >> >> If you run "echo $NSS_HASH_ALG_SUPPORT", is the environment variable set in >> the shell from which you are initiating the ldapsearch? > > > Oops, I should have checked this and, indeed: > > root@lataie:~# echo $NSS_HASH_ALG_SUPPORT > [nothing] > > hence I did > root@lataie:~# export NSS_HASH_ALG_SUPPORT=+MD5 > root@lataie:~# echo $NSS_HASH_ALG_SUPPORT > +MD5 > > However, my ldapsearch command still fails just the same... > > By the way, I forgot to mention that I snooped the packets arriving on the > LDAP server and they get there without any problem. > >> If you run "pargs -e " on the LDAP cache manager or name service cache >> process, does the environment variable show up? > > The ldap_cachemgr daemon fails to start for the same reason. (The truth is > that this failure is my REAL problem; I used the ldapsearch example to > shorten the explanation of the situation...) > > T. > ___ > OmniOS-discuss mailing list > OmniOS-discuss@lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss -- Ian Kaufman Research Systems Administrator UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
