Re: [OpenIndiana-discuss] Need a PCI e-sata card for OI151a

[Jason Matthews]

e-SATA shares the same electrical specification as SATA. The primary
difference is in the cable shielding specification. I would consider using a
LSI card and an adapter to go from the non/lightly shielded internal SATA
cable to the heavily shielded e-SATA cable. I am not aware of anyone who
makes one with a repeater to maintain super high signal quality but it
should work w/o such a device (supermicro does it all the time ;-], where as
Intel doesn't).

If you get an 8-port card you could dump your onboard ports altogether and
get the hot-plug you deserve. For 1068e based products you'll want the
SAS3801-R -- while it comes with the RAID firmware it is designed for
internal ports. The plan SAS3081 has two four port external plugs which you
don't want.  

You should be able to flash the SAS3081-R with the IT firmware or simply not
configure RAID on it, in which case the drives are exposed as JBODs by
default.

j.

PS - you'll need an 8x PCI-e slot.

http://www.ebay.com/sch/?_nkw=sas3081&clk_rvr_id=327631751749

if you have prime, these adapters ship for free.
http://www.amazon.com/s/ref=nb_sb_noss?url=search-alias%3Delectronics&field-
keywords=sata+to+e-sata+bracket#/ref=sr_nr_p_85_0?rh=n%3A172282%2Ck%3Asata+t
o+e-sata+bracket%2Cp_85%3A2470955011&bbn=172282&keywords=sata+to+e-sata+brac
ket&ie=UTF8&qid=1332871247&rnid=2470954011




-Original Message-
From: Dan Swartzendruber [mailto:dswa...@druber.com] 
Sent: Tuesday, March 27, 2012 6:11 AM
To: 'Discussion list for OpenIndiana'
Subject: [OpenIndiana-discuss] Need a PCI e-sata card for OI151a

Here's my situation: m1015 with 6 sata drives for pool tank.  7th port has
15K 73GB SAS drive as cache device.  8th port currently connected to e-sata
connector on front panel of case for monthly backups.  160GB sata drive on
one of the 4 motherboard sata ports (supermicro pdsmi+).  I have a 64GB
crucial m4 I want to use as a log device, but plugging it into the
motherboard seems to only yield sata1 speed (in addition to the fact that OI
apparently refuses to even go to the grub menu and just hangs - sigh...)
Even when I didn't have that issue (e.g. just switched from nexenta back to
OI), I discovered the motherboard ports apparently do NOT support hot-plug,
so switching the M4 and the e-sata connector is a no-go (unless I want to
have to boot with the e-sata drive plugged in and turned on - LOL).  My
motherboard only has one pcie slot (x8) which is where the m1015 resides.
So the plan is to get a pci card with e-sata connector.  I've found a couple
of cheap rosewill cards on newegg, that indicate the sil3512 chipset, but
I'm having trouble finding out if that is supported or not.  Any help
appreciated!

___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss



___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] Need a PCI e-sata card for OI151a

[Jason Matthews]

 

> Jason, I appreciate the input, but as I said, I'm already slot limited. 

> My mobo only has one pci-e slot, and that is occupied by the 8-port 

> m1015 HBA :(

 

 

My apologies. I just skimmed the big block of text. 

 

So, the M1015 is a rebadged LSI 9240-8i, likely with the IR firmware.

 

You can do what you want to do with a 16 port card. You may have to dump and
restore your zpool however since you have differences in firmware, IR versus
IT. The following card only offers IT firmware.

 

http://www.provantage.com/lsi-logic-lsi00244~7LSIG0J0.htm

$380

 

Also, I discourage you from using the crucial drives as log devices. They
lack super-caps to guarantee proper write operation in the event of a power
failure. I use DDRdrive X1s in production environment and Crucial 128M4s for
L2ARC. If the Intel 510s were available at the time of purchase I would have
used those instead.

 

Life might be easier for you if you got a more reasonable motherboard for a
server.

 

j.

 

 

___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] Need a PCI e-sata card for OI151a

[Jason Matthews]

Sorry I wasn't more helpful.

btw, "log device" implies ZIL, as in intent log.

j.

-Original Message-
From: Dan Swartzendruber [mailto:dswa...@druber.com] 
Sent: Tuesday, March 27, 2012 12:03 PM
To: Discussion list for OpenIndiana
Cc: Jason Matthews
Subject: Re: [OpenIndiana-discuss] Need a PCI e-sata card for OI151a


Already flashed to IT firmware and working fine.  Unfortunately, I am on 
a budget constraint (soho setup), and the mobo+cpu+8GB was already 
sitting around handy.  My plan for the M4 was L2ARC, not ZIL.  I think 
I'm going to get the cheap rosewill and use that for the esata drive - 
as I said, this is only running one consumer quality sata drive 
occasionally, so if it's not great, I can live with that.  I may keep 
trolling ebay looking for a used 16-port card like the one you 
referenced.  Thanks!



___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] spurious degraded multipath messages?

[Jason Matthews]

you plugged a consumer sata disk into an enterprise sas controller and it told 
you your sata disk doesnt have two data ports connected to the controller. i 
suspect you knew this already :-)

Sent from Jasons' hand held

On Mar 28, 2012, at 12:35 PM, Dan Swartzendruber  wrote:

> 
> Cool thanks.
> 
> ___
> OpenIndiana-discuss mailing list
> OpenIndiana-discuss@openindiana.org
> http://openindiana.org/mailman/listinfo/openindiana-discuss

___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

[OpenIndiana-discuss] we'll pay a bounty to fix three annoying problems in illumos

[Jason Matthews]

My company is interested in having two driver issues and a software issue
resolved and we are willing to pay a bounty.

Saturday, we were hit by something that looked similar to the mpt/mpt_sas
i/o hang issue https://www.illumos.org/issues/1069. This is a serious bug to
any "enterprise" or "carrier" grade solution and yet doesn't appear from the
bug db that any progress has been made on it. I realize this problem may be
"hard." Fortunately, I have a disk that can create the problem ;-) and we
would like to pay to have the problem solved.

The other driver issue is the lack of a bmc driver. 
https://www.illumos.org/issues/370. The test platform I can provide is on
Intel S5520UR motherboards with the server processor module.


The third issue is to turn on fault lights for newisys nds-2240 disk shelves
when a disk has been identified as failed. The support for non-Sun JBOD
shelves needs to be expanded. After the sale of LSI's disk shelf business to
NTAP options became quite limited. Today I use output from fmtopo and
sas2irc util to flip on the locator led but this does not make for an
enterprise solution. I understand that Nexenta has such a solution but they
are not sharing, which is too bad.

Feel free to work on one or all three. In the end, we will donate the code
to the project. If you are qualified and interested in being paid to solve
these problems please send me email off list.


Thanks,
j.




___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] we'll pay a bounty to fix three annoying problems in illumos

[Jason Matthews]

> As an observation, there are cleverer solutions than manually 
> calling sas2ircu [card] locate [backplane] [id] [on/off].

Thanks but it is not manual, it is scripted and keyed off the disk name.

> napp-it has a paid option, as does Nexenta, for this - I know a number
> of people in the community have written similar things or are doing
> so, so if you look in the mailing list archives, you'll probably find
> some useful things. 

Napp-it isn't a good fit as I am not running storage appliances. I have no
interest in installing their frame work. I have no interest in paying per
node for something that is mostly implemented by former engineers at Sun and
then gifted to us. Paying Napp-it or Nexenta for that matter doesn't help
IllumOS.

Oh, and the Nexenta solution, well, runs on Nexenta.

The goal is to enhance IllumOS.

How many man hours have collectively been wasted by highly paid individuals
implementing their own solutions outside of the framework that Sun provided?
I want a kernel hacker solution, not a sysadmin solution. 

In a better world the project would have the resources to identify at least
four external storage boxes (some 2.5", some 3.5", some SATA, some SAS) and
get the SES/FMD frameworks working for them.

We are willing to pay for the Newisys NDS-2240. 



j.



On Mon, Apr 9, 2012 at 3:55 PM, Jason Matthews  wrote:
>
>
> My company is interested in having two driver issues and a software issue
> resolved and we are willing to pay a bounty.
>
> Saturday, we were hit by something that looked similar to the mpt/mpt_sas
> i/o hang issue https://www.illumos.org/issues/1069. This is a serious bug
to
> any "enterprise" or "carrier" grade solution and yet doesn't appear from
the
> bug db that any progress has been made on it. I realize this problem may
be
> "hard." Fortunately, I have a disk that can create the problem ;-) and we
> would like to pay to have the problem solved.
>
> The other driver issue is the lack of a bmc driver.
> https://www.illumos.org/issues/370. The test platform I can provide is on
> Intel S5520UR motherboards with the server processor module.
>
>
> The third issue is to turn on fault lights for newisys nds-2240 disk
shelves
> when a disk has been identified as failed. The support for non-Sun JBOD
> shelves needs to be expanded. After the sale of LSI's disk shelf business
to
> NTAP options became quite limited. Today I use output from fmtopo and
> sas2irc util to flip on the locator led but this does not make for an
> enterprise solution. I understand that Nexenta has such a solution but
they
> are not sharing, which is too bad.
>
> Feel free to work on one or all three. In the end, we will donate the code
> to the project. If you are qualified and interested in being paid to solve
> these problems please send me email off list.
>
>
> Thanks,
> j.
>
>
>
>
> ___
> OpenIndiana-discuss mailing list
> OpenIndiana-discuss@openindiana.org
> http://openindiana.org/mailman/listinfo/openindiana-discuss

___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss


___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] Building in zones

[Jason Matthews]

> My network interface is set up as as a bridged interface to 
> my pc's wireless card.

If you are using a bridged interface then you need to match the mac address
in virtualbox to the burned in address on your hardware.

You can get your mac from: ipconfig /all in a command shell in windows

Let me know if that helps -- or well, if it doesn't. ;-)

j.



___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] no external connectivity on link

[Jason Matthews]

> ifconfig -a shows the aggregated link as well and all indications 
> are that it's fine

Ifconfig doesn't do that. Try using the LACP related commands instead. Your
output should look like like the following

root@db031:~# dladm show-aggr -L
LINKPORT AGGREGATABLE SYNC COLL DIST DEFAULTED EXPIRED
aggr0   igb0 yes  yes  yes  yes  nono
--  igb1 yes  yes  yes  yes  nono


root@db031:~# dladm show-aggr -x
LINKPORT   SPEED DUPLEX   STATE ADDRESS
PORTSTATE
aggr0   -- 1000Mb fullup0:1e:67:21:95:20   --
igb0   1000Mb fullup0:1e:67:21:95:20
attached
igb1   1000Mb fullup0:1e:67:21:95:21
attached


notice how the ports are attached, up, in sync, not defaulted, and not
expired.

> I can ping it.

Do you mean it can ping itself?

> However I cannot ping anything else on or off the network 
> (like 192.168.1.1 or yahoo.com)

This is likely because the LAG is down. What does the switch say?

> once I get this working I plan to enable link trunking on my 
> switch to take advantage of the faster speeds possible with this setup.

Umm, you have to configure both sides, the host and switch if you are going
to have network connectivity.

I wrote this article about PXE booting systems on switches configured for
LACP but it a part of it walks you through how to setup LACP on the host and
the switch (assuming you have juniper) --
http://broken.net/openindiana/how-to-pxe-boot-systems-on-lacp-using-juniper-
switches/

Does that help?

j.


___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] no external connectivity on link

[Jason Matthews]

> Are you prepared for no-improvement? Ethernet link aggregation often 
> disappoints people looking for more performance using IP.


It may a reasonable effect if:
1) you are already saturating the link
2) you have sufficient number of partners and they you doing large block
transfers

If you have a single machine just talking to one other machine, LACP will
not help by definition. You likely need a faster link.

Given the above, LACP can help if you tune your stack and your apps know how
to leverage the tuning. I have spent a number of years working in search and
lacp has been run of the mil. Typically I would see 1.2 Gb/s from boxes
whose job it was to forward search indexes to searchers even, if in some
cases, going over a router. This of course was on an optimized stack. I seem
to recall Hadoop clusters going has high as 1.4Gb/s in our environment.

LACP is good for link redundancy on servers connected to virtualized
chassis.  I use this instead of IPMP. LACP is even better for distributing
load on switches, which was the original intent of etherchannel from which
LACP rose. 

j.




___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] no external connectivity on link

[Jason Matthews]

Please verify that your host and switch have compatible activity settings.
Compatible meaning, they are not the same ;-)

I set the switch for active and the hosts for passive. You will receive
inconsistent results like the host being reachable at times and unreachable
at other times if they are not properly configured.

root@jobs004:~# dladm show-aggr
LINKPOLICY   ADDRPOLICY   LACPACTIVITY  LACPTIMER
FLAGS
aggr0   L4   auto passive   short
-

In juniper speak, you can check with something like this. Just replace your
AE number:

root@brdr0.sf0> show configuration interfaces ae1 aggregated-ether-options
lacp {
active;
}

or view the config with this:

root@brdr0.sf0> show lacp interfaces ae1
Aggregated interface: ae1
LACP state:   Role   Exp   Def  Dist  Col  Syn  Aggr  Timeout
Activity
  ge-0/0/5   ActorNoNo   Yes  Yes  Yes   Yes Fast
Active
  ge-0/0/5 PartnerNoNo   Yes  Yes  Yes   Yes Fast
Passive
  ge-1/0/5 FUPActor   NoNo   Yes  Yes  Yes   Yes Fast
Active
  ge-1/0/5 FUP  Partner   NoNo   Yes  Yes  Yes   Yes Fast
Passive
LACP protocol:Receive State  Transmit State  Mux State
  ge-0/0/5  Current   Fast periodic Collecting
distributing
  ge-1/0/5  Current   Fast periodic Collecting
distributing

I don’t have any IOS boxes with LACP at the moment or I would dump those
processes as well

j.

-Original Message-
From: Tim Dunphy [mailto:bluethu...@gmail.com] 
Sent: Wednesday, May 02, 2012 4:27 AM
To: Discussion list for OpenIndiana
Subject: Re: [OpenIndiana-discuss] no external connectivity on link

>Ifconfig doesn't do that. Try using the LACP related commands instead.
>Your output should look like like the following…notice how the ports are
>attached, up, in sync, not defaulted, and not expired.

Ok, thanks for that. Here's what those commands you mentioned show:

root@openindiana:/export/home/bluethundr# dladm show-aggr -L
LINKPORT AGGREGATABLE SYNC COLL DIST DEFAULTED EXPIRED
aggr1   e1000g0  yes  no   no   no   nono
--  e1000g1  yes  no   no   no   nono

root@openindiana:/export/home/bluethundr# dladm show-aggr -x
LINKPORT   SPEED DUPLEX   STATE ADDRESS
PORTSTATE
aggr1   -- 1000Mb fullup0:25:90:b:34:0 --
e1000g01000Mb fullup0:25:90:b:34:0
attached
e1000g11000Mb fullup0:25:90:b:34:1
attached

so, it looks like the ports are attached, up, NOT in sync, not
defaulted and not expired. Apparently I need to be able to get them in
sync, but I'm not sure  how to do that.

>> I can ping it.

>Do you mean it can ping itself?

Yep, that's what I was trying to say. The host can ping itself and as
of now I can ping the aggregated address from the host itself and from
another host on the network.

I basically left it alone overnight and when I went back to it this
morning I could ping external hosts from this machine and ssh into it
on that aggregated address whereas last night I could not which is why
I emailed the list! :)

Maybe it just takes a little while to work after it's setup?

> once I get this working I plan to enable link trunking on my
> switch to take advantage of the faster speeds possible with this setup.

>Umm, you have to configure both sides, the host and switch if you are going
>to have network connectivity.

Yep!

I'll have a look at your LACP article and see what I can do. Thanks
for your input!

Thanks and best regards,
Tim





On Wed, May 2, 2012 at 3:01 AM, Mark  wrote:
>
> And don't forget to configure or disable the firewall.
> That's what my last issue with lack of connectivity turned out to be.
> It may be on and blocking by default.
>
>
> On 2/05/2012 6:13 p.m., Richard Elling wrote:
>>
>> On May 1, 2012, at 8:41 PM, Tim Dunphy wrote:
>>
>>> hello list
>>>
>>> I have attempted to enable link aggregation on my oi 151 box using the
>>> command dladm create-aggr -d e1000g0 -d e1000g1 1 then I plumbed it
>>> with an address of 192.168.1.200 and echoed 192.168.1.1>
>>> defaultrouter
>>>
>>> I noticed that my /etc directory did not have the traditional solaris
>>> hostname.e1000g0 and hostname.e1000g1 files representing my links that
>>> you would typically expect. But I created a file with the command echo
>>> 192.168.1.200>  hostname.aggr1
>>>
>>> ifconfig -a shows the aggregated link as well and all indications are
>>> that it's fine
>>>
>>> dladm show-aggr shows the link aggregation and I can ping it..however
>>> I cannot ping anything else on or off the network (like 192.168.1.1 or
>>> yahoo.com)
>>>
>>> can someone offer a tip on what I'm doing wrong and how to resolve the
>>> issue?
>>
>>
>> Typical error is an incorrect netmask. Default netmask for a class-B
>> network is 255.255.0.0.
>>
>>>
>>> 

Re: [OpenIndiana-discuss] can't delete aggregate (dladm)

[Jason Matthews]

-t is temporary. Try rebooting. There is some chance you'll be back to where
you started ;-)

j.

-Original Message-
From: Tim Dunphy [mailto:bluethu...@gmail.com] 
Sent: Monday, May 14, 2012 12:14 PM
To: Discussion list for OpenIndiana
Subject: Re: [OpenIndiana-discuss] can't delete aggregate (dladm)

ok, disaster averted !

I tried a different flag the command I was using;
dladm delete-aggr -t aggr0 1

that allowed me to delete the virtual interface. Then, at that point I
was able to plum and unplumb the interfaces as per normal and recreate
the aggregate. It's all working now. Sorry for freaking out and
mailing the list! :) Don't want to burn up any good karma!

[root@openindiana:~] #dladm show-link
LINKCLASS MTUSTATEBRIDGE OVER
e1000g0 phys  9000   up   -- --
e1000g1 phys  9000   up   -- --
aggr1   aggr  9000   up   -- e1000g0 e1000g1

[root@openindiana:~] #ifconfig -a
lo0: flags=2001000849 mtu
8232 index 1
inet 127.0.0.1 netmask ff00
aggr1: flags=1000843 mtu 9000 index 13
inet 192.168.1.250 netmask ff00 broadcast 192.168.1.255
ether 0:25:90:b:34:0
lo0: flags=2002000849 mtu
8252 index 1
inet6 ::1/128

[root@openindiana:~] #ping yahoo.com
yahoo.com is alive

This list is a very congenial/valuable resource and I don't want to
squander it.

:)

tim

On Mon, May 14, 2012 at 2:50 PM, Tim Dunphy  wrote:
> Hello list,
>
>  I was trying to fix a problem I was having in being able to ping
> external hosts. For some reason I was able to ping other hosts on the
> network but had to reboot after installing a package. When the host
> (an OI 151 box) came back up it wasn't able to ping anything except
> itself.
>
>  The links were setup in an active/passive link aggregate called
> aggr0. I decided to attempt to recreate the link pool with the dladm
> remove-aggr command.
>
>  But when I typed dladm remove-aggr -d e1000g1 1
>
>  I see the message :
>
>  dladm: remove operation failed: object not found.
>
> Which is weird because if I do a dladm show-aggr  I see the aggregate
> in question.
>
> I was able to unplumb the aggregate link with
>
>  ifconfig aggr0 unplumb.
>
> But I cannot plumb or unplumb the individual physical links at this
> point. If I unplumb them I get
>
> ifconfig unplumb e1000g0
> ifconfig: cannot unplumb e1000g0: Interface does not exist
>
> When I plumb it I see:
>
> ifconfig e1000g0 plumb
> ifconfig: cannot plumb e1000g0: Interface already in use
>
> This is a very confusing situation! Does anyone have any solution for
> this? I need to be able to delete the link aggregation, plumb the
> individual interfaces and verify they are working before recreating
> the aggregate. I have good instructions for creating an aggregate and
> it was working well at one recent point. But I decided to try another
> tutorial that lead me down this path, and now I know which one to
> avoid!
>
> thanks for any advice you may have
>
>
> --
> GPG me!!
>
> gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B



-- 
GPG me!!

gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B

___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss


___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] sata hot plug problem (ahci?) + openindianacrash

[Jason Matthews]

Let me get this straight...

You installed the OS on the disk with the BIOS set to IDE. Later, you
changed the BIOS to AHCI and the system crashes when booting. Is that about
right?



j.


___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] packages for oracle db

[Jason Matthews]

This will get you most of the way there...

pkg install SUNWi1cs SUNWi15cs  compatibility/packages/SUNWxwplr  SUNWlibm

but I don't see SUNWi1of or SUNWxwplr in my copy of the repo. Perhaps you
can think of some interesting ways to obtain those packages you need.


j.

-Original Message-
From: Tim Dunphy [mailto:bluethu...@gmail.com] 
Sent: Tuesday, May 22, 2012 2:52 PM
To: Discussion list for OpenIndiana
Subject: [OpenIndiana-discuss] packages for oracle db

hey list,

 I'm trying to install oracle 11g on my oi 151 box. It'll need the
following packages:

ERROR: information for "SUNWi1cs" was not found
ERROR: information for "SUNWi15cs" was not found
ERROR: information for "SUNWxwfnt" was not found
ERROR: information for "SUNWxwplr" was not found
ERROR: information for "SUNWi1of" was not found
ERROR: information for "SUNWlibm" was not found

Where can I find these packages?

Thanks
Tim
-- 
GPG me!!

gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B

___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss


___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] installing oracle 11g

[Jason Matthews]

I don't want to sound insulting, but is X running?
Are you on console or logged in over ssh? If ssh, is xforwarding enabled? If
so, do you have some sort of X client on your side? Do other X clients work
(eg. xload)?

If console, who are youlogged in as? What user are you running the installer
as?

Why don't you logout, log back in, and show us the value of $DISPLAY (after
describing how you interact with X).

Thanks,

j.

-Original Message-
From: Tim Dunphy [mailto:bluethu...@gmail.com] 
Sent: Wednesday, May 23, 2012 1:09 PM
To: Discussion list for OpenIndiana
Subject: [OpenIndiana-discuss] installing oracle 11g

hey again list,

 I'm trying to surmount a small hurdle in installing Oracle 11g on my
oi 151 box.




When I run the ./runInstaller command from the database directory I
get this message

Checking monitor: must be configured to display at least 256 colors
>>> Could not execute auto check for display colors using command
/usr/openwin/bin/xdpyinfo. Check if the DISPLAY variable is set.
Failed 

And it proceeds to throw a java exception if I continue

>>> Ignoring required pre-requisite failures. Continuing...
Preparing to launch Oracle Universal Installer from
/tmp/OraInstall2012-05-23_04-01-36PM. Please wait ...-bash-4.0$
Exception in thread "main" java.lang.NoClassDefFoundError
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:164)
at java.awt.Toolkit$2.run(Toolkit.java:821)
at java.security.AccessController.doPrivileged(Native Method)
at java.awt.Toolkit.getDefaultToolkit(Toolkit.java:804)
at com.jgoodies.looks.LookUtils.isLowResolution(Unknown Source)
at com.jgoodies.looks.LookUtils.(Unknown Source)
at
com.jgoodies.looks.plastic.PlasticLookAndFeel.(PlasticLookAndFeel.ja
va:122)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:242)
at
javax.swing.SwingUtilities.loadSystemClass(SwingUtilities.java:1783)
at javax.swing.UIManager.setLookAndFeel(UIManager.java:480)
at
oracle.install.commons.util.Application.startup(Application.java:758)
at
oracle.install.commons.flow.FlowApplication.startup(FlowApplication.java:164
)
at
oracle.install.commons.flow.FlowApplication.startup(FlowApplication.java:181
)
at
oracle.install.commons.base.driver.common.Installer.startup(Installer.java:2
65)
at
oracle.install.ivw.db.driver.DBInstaller.startup(DBInstaller.java:114)
at
oracle.install.ivw.db.driver.DBInstaller.main(DBInstaller.java:132)

I think the key value I need to set is this one:

-bash-4.0$ export DISPLAY='localhost:0.0'


And I've tried a couple of variations including the actual hostname of
the host in place of localhost.

Any thoughts on how I can get past this point?

Thanks
Tim




-- 
GPG me!!

gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B

___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss


___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] installing oracle 11g

[Jason Matthews]

What Alan wrote.

j.

-Original Message-
From: Alan Coopersmith [mailto:alan.coopersm...@oracle.com] 
Sent: Wednesday, May 23, 2012 3:21 PM
To: Discussion list for OpenIndiana
Subject: Re: [OpenIndiana-discuss] installing oracle 11g

On 05/23/12 03:16 PM, Tim Dunphy wrote:
>> I don't want to sound insulting, but is X running?
>> Are you on console or logged in over ssh? If ssh, is xforwarding enabled?
If
>> so, do you have some sort of X client on your side? Do other X clients
work
>> (eg. xload)?
> 
> That's fine. I'm logged in at the console, not ssh'd in and yes x is
> working fine.
> 
> 
> I'm running as the oracle user
> 
> [root@openindiana:/tank/iso] #su - oracle
> 
> 
> And interesting that xload gives an error when I try to run it as the
> oracle user.
> 
>  I've tried setting a few variables for DISPLAY:
> 
> -bash-4.0$ export DISPLAY=":0.0"
> -bash-4.0$ xload
> Xlib: connection to ":0.0" refused by server
> Xlib: No protocol specified
> 
> Error: Can't open display: :0.0

The default X setup, on pretty much every OS made in the past twenty years,
is
that only the user who logs into X can open windows on the display.   Some
Linux
distros do include a PAM module to forward this authorization on su, but OI
does
not.

The simplest workaround is to grant the Oracle user permission before you su
to
it:
xhost +si:localuser:oracle

A variety of other solutions are possible as well, as a google search for
terms
such as "xauthority" and "xauth" will turn up.

-- 
-Alan Coopersmith-  alan.coopersm...@oracle.com
 Oracle Solaris Engineering - http://blogs.oracle.com/alanc

___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss


___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

[OpenIndiana-discuss] quick boots are a pain in my arse

[Jason Matthews]

So, my systems wont quick boot at all. Any quick boot results in the
following message:

NOTICE: unsupported 32-bit IO address on pci-pci bridge [7/0/0]
NOTICE: unsupported 32-bit IO address on pci-pci bridge [7/0/0]

Then system then hangs. Initial boots are always fine, as are full reboots.

I have been able to work around this problem by disabling quick boots in the
repo. However, I have noticed that whenever the system goes through a
"transient reboot" it seems to attempt a quick boot. This consequently keeps
the system from booting. This is a "bad thing"

I am seeking help in two regards. First, I would prefer to resolve the error
and in lieu of an actual fix, I would like to learn how I can alter the
transient boot environment to do a full reboot.

The hardware configuration for these systems are as follows
Intel S5520UR motherboard (SR2525URLXR)
1 LSI 9205-8e
1 LSI 9211-8i
96 GB of RAM
1 DDRdrive X1
2 L5630 CPUs
2 Newisys NDS-2241 disk shelves shared across two systems
4 Crucial M4 SSDs as boot disks and L2ARC

I am standardized on 151a1, but I can upgrade some test units if we think
there is a solution in the newer revisions.

Thanks,
j.
___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] quick boots are a pain in my arse

[Jason Matthews]

I typo'd this.

It should have read:
NOTICE: unsupported 32-bit IO address on pci-pci bridge [7/0/0]
NOTICE: unsupported 32-bit IO address on pci-pci bridge [8/2/0]


Thanks,
j.

-Original Message-
From: Jason Matthews [mailto:ja...@broken.net] 
Sent: Thursday, July 05, 2012 5:34 PM
To: 'Discussion list for OpenIndiana'
Subject: [OpenIndiana-discuss] quick boots are a pain in my arse



So, my systems wont quick boot at all. Any quick boot results in the
following message:

NOTICE: unsupported 32-bit IO address on pci-pci bridge [7/0/0]
NOTICE: unsupported 32-bit IO address on pci-pci bridge [7/0/0]

Then system then hangs. Initial boots are always fine, as are full reboots.

I have been able to work around this problem by disabling quick boots in the
repo. However, I have noticed that whenever the system goes through a
"transient reboot" it seems to attempt a quick boot. This consequently keeps
the system from booting. This is a "bad thing"

I am seeking help in two regards. First, I would prefer to resolve the error
and in lieu of an actual fix, I would like to learn how I can alter the
transient boot environment to do a full reboot.

The hardware configuration for these systems are as follows
Intel S5520UR motherboard (SR2525URLXR)
1 LSI 9205-8e
1 LSI 9211-8i
96 GB of RAM
1 DDRdrive X1
2 L5630 CPUs
2 Newisys NDS-2241 disk shelves shared across two systems
4 Crucial M4 SSDs as boot disks and L2ARC

I am standardized on 151a1, but I can upgrade some test units if we think
there is a solution in the newer revisions.

Thanks,
j.
___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] quick boots are a pain in my arse

[Jason Matthews]

> -Original Message-
> From: Marion Hakanson [mailto:hakan...@ohsu.edu] 
> Sent: Thursday, July 05, 2012 6:19 PM
> To: Discussion list for OpenIndiana
> Subject: Re: [OpenIndiana-discuss] quick boots are a pain in my arse
> 
> 
> svccfg -s "system/boot-config:default" setprop
config/fastreboot_default=false
>
> Seems to take care of it here, anyway.

Thanks for the effort, but as I pointed out in my original email I have
already disabled fast boot in the smf repo. 


j.
___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] ZFS and AVS guru $500

[Jason Matthews]

are you missing a zero to the left of the decimal place?

j. 

Sent from Jasons' hand held

On Jul 23, 2012, at 8:57 PM, "John T. Bittner"  wrote:

> Subject: ZFS and AVS guru
> 
> I am working on setting up 2 SAN's to replicate via AVS. 
> The 2 sans I build have 15 drives SAS drives + 2 Cache SSD's and 2 Log SSD's.
> OS drives are also SSD's and are mirrored.
> Units are running current version of Openindiana with AVS installed.
> Our environment we run comstar fiber channel targets with ISCSI backup.
> I have conflicting reports on if active / active is possible but if not 
> active / passive will do.
> 
> I need someone that has done this before and is familiar with this type of 
> setup.
> $500.00 for the work, to include 1 hour of training on the system, how to 
> monitor the replication, failover and failback.
> 
> The need to get this done is a rush, so you must be available in the next day 
> or so.
> 
> Anyone interested please email me direct at j...@xaccel.net
> 
> Thanks
> 
> John Bittner
> Xaccel Networks.
> 
> 
> 
> 
> 
> ___
> OpenIndiana-discuss mailing list
> OpenIndiana-discuss@openindiana.org
> http://openindiana.org/mailman/listinfo/openindiana-discuss

___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] LSI SAS2008 based Raid card not found

[Jason Matthews]

i have a large number of systems with 9211-8i with IR firmware running in jbod 
mode with now problems. 

infact 2003, 2008, and 2308 based cards all work for me in either IT mode or IR 
running as jbod (until i add a intel 910 but that is a work in progress)

i have havent tried a megaraid card in several years, probably since zfs was 
released. 

the successor to the 9211 is the 9207. there is one on my desk and while i 
havent done extensive validation on it yet, it appears to work normally in my 
std setup. 

Sent from Jasons' hand held

On Jul 27, 2012, at 8:15 AM, Richard Skelton  wrote:

> Hi,
> I am running OpenIndiana on my Sun Ultra 24 and the driver is not being
> loaded for the Raid card below.
> cat /etc/release
> OpenIndiana Development oi_151.1.5 X86 (powered by illumos)
>Copyright 2011 Oracle and/or its affiliates. All rights reserved.
>Use is subject to license terms.
>   Assembled 26 June 2012
> /usr/X11/bin/scanpci -v
> pci bus 0x0001 cardnum 0x00 function 0x00: vendor 0x1000 device 0x0073
> LSI Logic / Symbios Logic MegaRAID SAS 9240
> CardVendor 0x1734 card 0x1177 (Fujitsu Technology Solutions RAID Ctrl
> SAS 6G 0/1 (D2607))
>  STATUS0x0010  COMMAND 0x0047
>  CLASS 0x01 0x04 0x00  REVISION 0x03
>  BIST  0x00  HEADER 0x00  LATENCY 0x00  CACHE 0x10
>  BASE0 0x1001  addr 0x1000  I/O
>  BASE1 0xf0840004  addr 0xf084  MEM 64BIT
>  BASE3 0xf084  addr 0xf080  MEM 64BIT
>  MAX_LAT   0x00  MIN_GNT 0x00  INT_PIN 0x01  INT_LINE 0x05
> 
> 
> Looking around on the internet it seems the Fujitsu Technology Solutions
> (D2607) is  based on the LSI SAS2008 and may be a LSI 9211-8i
> 
> This card works in the Ultra 24 running Solaris 10 8/11 s10x_u10wos_17b
> X86 and uses the imraid_sas driver:-
> 
> modinfo|grep sas
> 165 f7fdd000  11ff8  69   1  imraid_sas (03.03.00)
> 
> 
> 
> Thanks for any advice on how to get this card up and running.
> 
> Cheers
> Richard.
> 
> ___
> OpenIndiana-discuss mailing list
> OpenIndiana-discuss@openindiana.org
> http://openindiana.org/mailman/listinfo/openindiana-discuss

___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] [solarisx86] For those nostalgic fellas

[Jason Matthews]

har har har! it is a three hump camel for sure. 

Sent from Jasons' hand held

On Aug 6, 2012, at 9:15 PM, Alan Coopersmith  
wrote:

> Holy water ... check.
> Wooden stakes ... check.
> Silver bullets ... check.
> Chainsaw ... check.
> 
> Okay, all ready here, we'll make sure it can't get back in.
> 
>-alan-
> 
> On 08/ 6/12 08:47 PM, Jerry Kemp wrote:
>> reposted from the Sun rescue mailing list.
>> 
>> Jerry
>> 
>> 
>> .
>> So.
>> 
>> Is no one going to comment on the new CDE news?
>> 
>> http://cdesktopenv.sourceforge.net/
>> 
>> It has now been open sourced under the LGPL.
>> 
>> More info here:
>> http://sourceforge.net/p/cdesktopenv/wiki/Home/
>> .
>> 
>> On 01/ 2/11 12:15 PM, Alan Coopersmith wrote:
>>> On 01/ 2/11 05:55 AM, Apostolos Syropoulos wrote:
 http://devio.us/~kpedersen/
>>> 
>>> Nooo!   I just killed CDE!   It can't be back from the dead!
>>> Run before the zombie gets us!
>>> 
>>> [Unfortunately, since CDE is a closed gate, the bug data isn't published
>>> on bugs.opensolaris.org, so I can't link to the bug report for the "fix"
>>> in snv_156 that deleted 12937 source files for the CDE applications,
>>> leaving just the libraries like Motif for binary compatibility.
>>> Unfortunately, OpenMotif isn't binary compatible with the Motif 2.1 in
>>> Solaris.]
>>> 
> 
> 
> -- 
>-Alan Coopersmith-  alan.coopersm...@oracle.com
> Oracle Solaris Engineering - http://blogs.oracle.com/alanc
> 
> ___
> OpenIndiana-discuss mailing list
> OpenIndiana-discuss@openindiana.org
> http://openindiana.org/mailman/listinfo/openindiana-discuss

___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] [solarisx86] For those nostalgic fellas

[Jason Matthews]

welp, build it, package it up, and add it to a repo (or pay some one to do the 
same)

i would rather see  resources put to work on a hundred different things (or 
more, likely) before CDE. 

i would pay $ for fmd integration with newisys storage chassis so get in line 
behind me. ... 

for the greater good an OI that didnt barf on its clothes when running on sandy 
bridge would be awesome. or a respin of igb for the new chip would be helpful 
(i may be dated on these but you get the point, right?)

j. 

Sent from Jasons' hand held

On Aug 7, 2012, at 8:56 AM, "Richard L. Hamilton"  wrote:

> That's ok for you, you had to suffer to get rid of it.  For those that suffer 
> where it's gone, this is great!
> 
> On Aug 7, 2012, at 12:15 AM, Alan Coopersmith wrote:
> 
>> Holy water ... check.
>> Wooden stakes ... check.
>> Silver bullets ... check.
>> Chainsaw ... check.
>> 
>> Okay, all ready here, we'll make sure it can't get back in.
>> 
>>-alan-
>> 
>> On 08/ 6/12 08:47 PM, Jerry Kemp wrote:
>>> reposted from the Sun rescue mailing list.
>>> 
>>> Jerry
>>> 
>>> 
>>> .
>>> So.
>>> 
>>> Is no one going to comment on the new CDE news?
>>> 
>>> http://cdesktopenv.sourceforge.net/
>>> 
>>> It has now been open sourced under the LGPL.
>>> 
>>> More info here:
>>> http://sourceforge.net/p/cdesktopenv/wiki/Home/
>>> .
>>> 
>>> On 01/ 2/11 12:15 PM, Alan Coopersmith wrote:
 On 01/ 2/11 05:55 AM, Apostolos Syropoulos wrote:
> http://devio.us/~kpedersen/
 
 Nooo!   I just killed CDE!   It can't be back from the dead!
 Run before the zombie gets us!
 
 [Unfortunately, since CDE is a closed gate, the bug data isn't published
 on bugs.opensolaris.org, so I can't link to the bug report for the "fix"
 in snv_156 that deleted 12937 source files for the CDE applications,
 leaving just the libraries like Motif for binary compatibility.
 Unfortunately, OpenMotif isn't binary compatible with the Motif 2.1 in
 Solaris.]
 
>> 
>> 
>> -- 
>>-Alan Coopersmith-  alan.coopersm...@oracle.com
>> Oracle Solaris Engineering - http://blogs.oracle.com/alanc
>> 
>> ___
>> OpenIndiana-discuss mailing list
>> OpenIndiana-discuss@openindiana.org
>> http://openindiana.org/mailman/listinfo/openindiana-discuss
>> 
> 
> -- 
> eMail:mailto:rlha...@smart.net
> Home page:http://www.smart.net/~rlhamil/
> Facebook, MySpace,
> AIM, Yahoo, etc:ask
> 
> 
> 
> 
> 
> ___
> OpenIndiana-discuss mailing list
> OpenIndiana-discuss@openindiana.org
> http://openindiana.org/mailman/listinfo/openindiana-discuss

___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] SSD swap space performance

[Jason Matthews]

many ssds carry 3-year warranties and report useable life via smart. if you are 
concerned about wearing them then buy a spare or two and rma them as they 
approach burn out. 

that is what i do :)

Sent from Jasons' hand held

On Aug 8, 2012, at 9:24 AM, Reginald Beardsley  wrote:

> Order of magnitude I already knew.  I was hoping for measured data.  I'm 
> running Solaris 10 on this particular system.  The OI box is a nettop for 
> Internet access.
> 
> I'm concerned w/ occasionally running jobs which exceed physical memory.  At 
> most I can only go up to 24 GB by replacing all the 2 GB DIMMS w/ 4 GB DIMMS. 
> 
> However, I'd not considered the problem of pathological paging patterns.  On 
> a long job one might well wear out an SSD in a single run which would not be 
> helpful.  I don't think I want to try to figure out what the paging patterns 
> of a large solver like glpk are.  Particularly because it almost certainly 
> problem and option dependent.
> 
> A dedicated pair of small, fast disks for swap is probably the best option. 
> To the best of my knowledge Solaris will still interleave paging among 
> multiple swap partitions on separate drives. This was something I routinely 
> did on my systems for many years.
> 
> IIRC two matched drives were almost twice as fast as one. It was essential 
> that the drives have identical performance, otherwise it actually slowed the 
> system down.  I vaguely recall that more than two didn't improve performance, 
> but it was a long time ago.  I may not have had 4 identical drives.
> 
> Have Fun!
> Reg
> 
> --- On Wed, 8/8/12, Jan Owoc  wrote:
> 
>> From: Jan Owoc 
>> Subject: Re: [OpenIndiana-discuss] SSD swap space performance
>> To: "Discussion list for OpenIndiana" 
>> Date: Wednesday, August 8, 2012, 10:39 AM
>> On Wed, Aug 8, 2012 at 8:49 AM, Udo
>> Grabowski (IMK)
>> 
>> wrote:
>>> On 08/08/2012 16:47, Reginald Beardsley wrote:
 
 Has anyone done any testing on the performance of
>> an SSD for swap vs disk?
 I'm currently fully populated at 12 GB in my Z400.
 
>>> That should work, but if you really need a fast SSD
>> for
>>> regularily swapping, you may should start thinking
>> about
>>> more memory...
>> 
>> I haven't done testing on OpenIndiana, but in my experience
>> swap on an
>> SSD is an order of magnitude faster than swap on HDD. Note
>> that, as
>> Udo pointed out, this is still an order of magnitude (or
>> two) slower
>> than additional RAM.
>> 
>> If your workload is such that you have large chunks of
>> memory that can
>> be swapped out for extended periods of time, then an SSD
>> will do this
>> several times faster. However, if you have a program that is
>> actively
>> using more than the 12GB of RAM you have, you will kill your
>> SSD (SSDs
>> have a limited number of write cycles - each time you'd run
>> the
>> program, loads of data would hit the SSD) and won't get
>> anywhere near
>> the performance of actually having adequate memory.
>> 
>> Jan
>> 
>> ___
>> OpenIndiana-discuss mailing list
>> OpenIndiana-discuss@openindiana.org
>> http://openindiana.org/mailman/listinfo/openindiana-discuss
>> 
> 
> ___
> OpenIndiana-discuss mailing list
> OpenIndiana-discuss@openindiana.org
> http://openindiana.org/mailman/listinfo/openindiana-discuss

___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] Using a SAN on OpenIndiana

[Jason Matthews]

i recommend 40Gb infiniband. oi has no support for management so you'll need 
switch. used fear is cheap on ebay. 

infiniband is generally cheaper than 10Gb ether on a per port basis and from a 
latency perspective there is comparison. 

stock exchanges dont use 10Gb ether, they use infiniband. 

Sent from Jasons' hand held

On Aug 8, 2012, at 3:54 PM, "John McQuay"  wrote:

> Hi All
> 
> 
> 
> I'm using OI 151_a5 and have a query relating to SANs.  There's a website
> called smallnetbuilder  that I've stumbled across giving advice on many
> things, one of the articles is how to build a SAN.  I've thought about
> building a smaller version of this for kicks more than anything but was
> wondering if anyone has mounted a fibre-channel SAN into OI, and if so, how
> easy is it?
> 
> 
> 
> Many thanks and best regards, John.
> 
> ___
> OpenIndiana-discuss mailing list
> OpenIndiana-discuss@openindiana.org
> http://openindiana.org/mailman/listinfo/openindiana-discuss

___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] sata hba choice

[Jason Matthews]

I only have one gripe with the mpt2 based cards (2008, 2308, etc) is that
they cant currently be used as boot devices in the presence of an Intel 910
SSD. I have defect 490258-218190922 logged at LSI but there is seemingly no
movement after several weeks. The status of this can be tracked at
http://broken.net/intel-910-ssd/intel-910-ssd-compatibility-issues-with-othe
r-lsi-mpt2-based-products-part-i/ Ironically, the Intel 910 also uses an
mpt2 chip.

My next buy of servers are going out with the EOL'd SAS3081E-R cards which
are 1068e based so I use 800GB Intel 910s. I prefer to boot from LSI cards
as sort of the devil I know as opposed to the devil I don't in the case of
an onboard controller.


j.

___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] SSDs for ZFS Pool?

[Jason Matthews]

I have several hundred SSDs in use as L2ARC. I have the monitoring system
querying SMART data a number of times per day looking at the percentage of
rated life used in effort to replace disks before we burn out the cells.

Each time this query runs, it dirties the logs with some messages in regards
to this error but it doesn't actually cause me any grief. I don't believe
this causes any data corruption issues and SMART still works.

My recommendation is to ignore the messages for now and worry about real
problems. 


j.

-Original Message-
From: Bryan N Iotti [mailto:ironsides.med...@gmail.com] 
Sent: Friday, August 24, 2012 8:53 AM
To: Discussion list for OpenIndiana
Subject: Re: [OpenIndiana-discuss] SSDs for ZFS Pool?

Sorry to reopen an old post, but I have some questions I can't seem to 
find an answer to myself...

I recently caved in and bought an LSI 3041E-R HBA, which I reflashed to 
Initiator Target firmware, the latest version available. I have my usual 
two 2TB SATA drives attached directly via SATA. I have encountered 
Illumos bug #1787 as described here:
https://www.illumos.org/issues/1787

Would installing a SAS-To-SATA interposer card on each disk clear this 
error, or would it still act the same way? I mean, if I understand 
right, this bug affect only SATA drives on SAS controllers.

Should I just cave in once more and buy some SAS disks, if SMART is so 
important to me?


Any thoughts welcome.

Bryan

On 08/20/12 05:08 PM, Sašo Kiselkov wrote:
> On 08/20/2012 04:55 PM, Sebastian Gabler wrote:
>> Hi,
>> Questions:
>>
>> - I understand that TRIM will not work in any stripe set, be it parity
>> based or not, correct?
> Correct, Illumos has no TRIM support (yet).
>
>> - will heavy writes (such as transferring data from dumps, which will
>> happen relatively often) lead to contention?
> That's more a question of the quality of SATA drives you plan to
> purchase. Read
>
http://gdamore.blogspot.com/2010/08/why-sas-sata-is-not-such-great-idea.html
> for why SATA in a SAS environment isn't such a great idea. If you have
> to use SATA, I recommend getting SAS interposers for them, e.g.
>
http://dataonstorage.com/dataon-products/6g-sas-to-sata-interposer-card-and-
kit.html
> (for example all SATA drives shipped in Sun JBODs had interposers)
>
>> - Should I upgrade the HBAs to something newer like LSI2008, for
>> compatibility reasons (no, I probably don't need the extra bandwidth SAS
>> 6g offers!)
> The LSI SAS1068 are very high quality chips, so if you feel you don't
> need 6G SAS, there's probably no need to upgrade. LSI SAS2008, however,
> can be had at extremely competitive prices (<100USD for a new
> Dell-branded external HBA card with the chips is an extremely attractive
> proposition) and are also very stable.
>
>> - Are there any SSDs that you could recommend?
> OCZ Talos R, though these are quite pricey. You might want to look into
> stuffing it with normal 7k2 rpm drives and getting a suitably-large
> L2ARC device (these can do wonders, my crappy little 60GB Vertex 3 can
> push >90MB/s and >10k iops in 8k random reads).
>
>> - with some desktop HDs, I experienced that they will be kicked out of
>> the array for no obvious reason - does that happen with consumer SSDs as
>> well, i.e. while the controller re-balances free cells for writing?
> It's more a problem of the SATA circuitry which can't cope with the
> constant battery of commands when your pool is taking a bad beating. See
> above for my opinion on SATA vs. SAS.
>
>> - should I better use 10 K SAS drives w/ flash-based ZIL?
> Drive speed isn't all that important if you get a suitably large L2ARC.
> If your databases are going to be used for testing only (so data loss in
> case the machine goes down isn't that big an issue), you might want to
> turn the ZIL entirely off. It's quite a performance killer unless you
> choose to dump a bunch of money into an expensive slog device (e.g.
> ddrdrive, zeusram, etc.).
>
> --
> Saso
>
> ___
> OpenIndiana-discuss mailing list
> OpenIndiana-discuss@openindiana.org
> http://openindiana.org/mailman/listinfo/openindiana-discuss


___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] SSDs for ZFS Pool?

[Jason Matthews]

> Do you have any real-life stats about SSD breakage (lifetime
> expectancy) in L2ARC usage modes - did any of your SSDs
> already reach their end-of-life, and if yes - what percentage
> and how old were they? What model(s) do you use?

The oldest round of gear uses 128GB Crucial M4s. I have seen two failures on
the M4s and neither was due to cell wear. The first failure was infant
mortality with spontaneous failure to complete writes. The second failure
had an electrical issue on the bus.

Typical l2arc wear on my databases is 10% on each L2ARC device after ten
months of operation or at present 7061 power on hours. I use three devices
or more per pool. My L2ARC exceeds the size of the live data set in all
cases. There are hundreds (and going to thousands) of databases running in
zones.

My database I/O patterns are write heavy with a large batch job which runs
essentially back to back. I have l2arc_noprefetch set to zero which in
theory would wear the l2arc devices faster. I have found it to be the right
call as it improved the l2arc hit cache ratio significantly for my i/o
pattern.

My monitoring system uses cacti, zabbix, and graphite. The write i/o
requirements are much more intense as a data pool than l2arc on the
databases. At this moment, I am at 40% utilization on each of the SSDS for
roughly the same time period (7599 power on hours).

The warranty on the devices is three years. It is clear the monitoring
system is going to burn through the disks before the warranty runs up. By
the time l2arc disks wear out on the databases I will be off of them, off
the spinning rust, and on to Intel 910s or similar devices.

I haven't had enough failures to have any meaningful statistics.


j.




___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] sata hba choice

[Jason Matthews]

> I can say from experience chaining supermicro sas chassis with any type of
> disk can be a recipe for disaster. Lsiutil reported a couple million
> errors in a couple months time which eventually led to some sort of
> contention and corruption on both sides of our mirrors. These were 
> also SAS and not sata disks.


I use newisys nds-2240 chassis. They were designed to be replacements for
the 620j, which left a vaccum in the market when LSI sold off its chassis
division to netapp. 

The devices work flawlessly. I use them in an all sas environment. I put the
sata based l2arc disks on a separate controller, on a passive midplane, in
the server chassis.

If you buy the disks from newisys/sanima-sc, they'll even burn the drives in
so you can avoid infant mortality. 

It has been a very smooth ride.

j.


___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] HW sensors for OI?

[Jason Matthews]

Wow you are ambitious. We don't even have a bmc driver.

Today, you would have to get the values from ipmi over the network or
through SMASH if you have it.

j.

-Original Message-
From: Jim Klimov [mailto:jimkli...@cos.ru] 
Sent: Saturday, August 25, 2012 4:20 PM
To: Discussion list for OpenIndiana
Subject: [OpenIndiana-discuss] HW sensors for OI?

Hello all,

   Got myself wondering: is there a generic hardware-monitoring
framework for illumos or OpenIndiana, similar to Linux lm-sensors
with its pluggable drivers to query different monitoring devices
and buses?

   Something that works on nearly any motherboard, that I could
attach MRTG to and have nice graphs of temperatures, voltages
and other physical variables?

Thanks,
//jim

___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss
___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] HW sensors for OI?

[Jason Matthews]

> there was a chance of porting *BSD BMC drivers as a legal 
> solution, or finding and using driver binaries from older 
> OpenSolaris as a questionable one.

The bmc driver has been closed source from the start. I have been unable to
get the bmc driver from SXC or even S11 to to work in 151a1, at least with
the gear available to me. However, the S11 bmc driver works fabulously using
S11. Like most people, my gear uses a rebranded AMI service processor.

j.
___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

[OpenIndiana-discuss] is there a need for a US mirror?

[Jason Matthews]

I saw the thread about a new mirror in Europe.

Does OI need an US mirrors in San Francisco or NoVA? I may be able to help,
but I would like to get a better understanding of the current bandwidth
usage.

Thanks,
j.
___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] Mitigating the performance impact of scrub

[Jason Matthews]

Reginald Beardsley Wrote:

> This is new territory. I've never had the ability to 
> check the disk integrity before.  So what is a sensible 
> scrub schedule?  Monthly?  

Well, I do it by the book (I also service my car on the manufacturers
intervals [but not the dealers]). I scrub once per month. I have an
extremely write heavy batch work load. The batch runs every minute and if
left to the defaults, resilvers and scrubs take days and weeks,
respectively.

I had to tune the other way to speed them up... Now resilvers and scrubs
complete in a few hours.

https://broken.net/uncategorized/zfs-performance-tuning-for-scrubs-and-resil
vers/


___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] Mitigating the performance impact of scrub

[Jason Matthews]

> Not sure I see an effective difference between a scrub that is run 
> quickly once a month and a scrub that runs for a month w/ little
> performance impact on other usage.

The problem was, the once a month batch job after running for day reported
it was going to take six weeks. I suspect it would have reported three
months after day two, and six months after day three. The numbers were
getting worse over time.

j.
  
___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

[OpenIndiana-discuss] beadm broken - multiple active environments

[Jason Matthews]

Okay, I some how managed to break beadm on one of my systems and I am hoping
for a little help.

The box has three zpools, rpool, data, and archive. If I export data, beadm
works without errors. However, if data is imported then beadm gives me this
lovely descriptive blurb:

root@heimdall:~# beadm create test
Unable to create test.
Multiple active zone roots

So it seems clear, that some how, beadm thinks there is an 'active zone
root' on the data pool. So, how can I find it, remove it, etc?

Also, is scanning all the attached pools for active root zones a sane thing
to do? Do they even matter if they aren't bootable or on the rpool/syspool?

Thanks,
j.
___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] beadm broken - multiple activeenvironments

[Jason Matthews]

Richard Lowe wrote:

> Is there any possibility that you've cloned a zone onto that data pool?


Sorry about the belated response. You are a genius. This was exactly the
problem.

I keep getting bitten by zfs send -R. It is convenient as it copies all the
descendent file systems, but by also cloning all the properties it causes me
more issues than it solves. I generally realize my mistake at reboot when
zfs mount -a fails, which might be months after the mistake is made.

I wish there was zfs send -r, which would copy all the descendents but leave
the properties behind. I have since scripted this, but seems like a natural
option.

Thanks for your help. The error message was puzzling my puzzler.

j.
___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] Zfs stability "Scrubs"

[Jason Matthews]

From: heinrich.vanr...@gmail.com [mailto:heinrich.vanr...@gmail.com] 


> My point is most high end storage units has some form of data 
> verification process that is active all the time. 

As does ZFS. The blocks are checksumed on each read. Assuming you have
mirrors or parity redundancy, the misbehaving block is corrected,
reallocated, etc.

> In my opinion scrubs should be considered depending on the importance 
> of data and the frequency based on what type of raidz, change rates 
> and disk type used. 

One point of scrubs is to verify the data that you don't normally read.
Otherwise, the errors would be found in real time upon the next read.

> Perhaps in future ZFS will have the ability to limit resource 
> allocation when scrubbing like with BV where it can be set. Rebuild 
> priory can also be set.

There are tunables for this.


> Also some high end controllers have "port" verify for each 
> disk (media read) when using their integrated raid that runs 
> periodically. Since in the world of ZFS it is recommended to use 
> JBOD I see it as more than just the filesystem. I have never deployed 
> a system containing mission critical data using filesystem raid 
> protection other than with ZFS since there is no protection in them an 
> I would much rather bank on the controller.

 
Unfortunately my parser was unable to grok this. Seems like you would prefer
a raid controller.

j.
___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] Memory usage concern

[Jason Matthews]

> It seems the memory is going to the kernel.


I have this problem on my web proxy tier, but not on my backend systems.

I have checked kernel core dumps for memory leaks, switched the Ethernet
cards/drivers, tested exclusive versus shared interfaces on the zones, and
tweaked everything I could think of. I haven't found a solution. I run the
same motherboard every where and I only have this issue on the proxy tier.

I put 96gb of ram in one box and watched the system consume it all.
Ultimately, I increased the RAM across the board to push the reboot cycle
out a couple of weeks. My longer term plan is to develop the software stack
on Linux and migrate the proxy tier to CentOS. 

It sickens me that we can't get this right, but if Linux is the best tool
for the job then that's what I will use. Fortunately, the data tier doesn't
suffer from this problem and that's where we really need the data handling
capabilities of ZFS.

My hardware platform on the web tier is Intel SR2625URLX, LSI 9211-8i,
Crucial M4s for storage. The primary application running is nginx. I have
nearly no writes to disk. We have such a fire hose of http traffic that we
cannot possibly write logs on the proxy tier.

By the way, it is nice to find you. James Carlson insisted I was the only
one having this problem ;-)

j.

root@www003:~# for i in `seq 1 5`; do date; echo ::memstat |mdb -k |egrep
"Kernel|MB"; sleep 120; done
October 15, 2012 03:32:37 PM PDT
Page SummaryPagesMB  %Tot
Kernel3659602 14295   44%
October 15, 2012 03:34:39 PM PDT
Page SummaryPagesMB  %Tot
Kernel3660353 14298   44%
October 15, 2012 03:36:41 PM PDT
Page SummaryPagesMB  %Tot
Kernel3661008 14300   44%
October 15, 2012 03:38:43 PM PDT
Page SummaryPagesMB  %Tot
Kernel3661708 14303   44%
October 15, 2012 03:40:45 PM PDT
Page SummaryPagesMB  %Tot
Kernel3662362 14306   44%


___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] Memory usage concern

[Jason Matthews]

> I can't tell you much about continuus use of VirtualBox and 
> its stability over long periods of time, I used it with success on 
> daily basis for few years now, with OI (and previously OpenSolaris) 
> being host. I've seen no problems with stability, however I only run 
> it for a day and then whole system was powered down for night.

I have this problem without virtual box. The system consumes memory until it
gets down to a couple of hundred megabytes. By then, network response times
shoot up. Since I collect this data, I can also tell you that there are
periodic "resets" where the response time improves for some period of time.
The cycle continues until the system crashes due to memory starvation.

j.
___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] Memory usage concern

[Jason Matthews]

I should also mention that over all network performance degrades over time
and the only fix I have sound so far is to shut down the zones, destroy the
vnics, recreate them, and then reboot the zones.

I do that about every four hours to keep the response times reasonable. It
really sucks.

j.
___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] Memory usage concern

[Jason Matthews]

-Original Message-
From: Cedric Blancher [mailto:cedric.blanc...@googlemail.com] 

> IMO you blame the wrong people. You can have the same kind of 
> problems with any Illumos-based distribution if you activate a 
> zone and let the machine just sit there for a week or so or have 
> a lot filesystem activity using mmap(). Either way the machines 
> will choke themselves to memory starvation. The only workaround 
> we found are regular reboots (every 24h), or limit the 
> ZFS ARC to an absolute minimum.

I don't think you understand. My proxy tier does almost no reads from the
file system. There is no content on the server. The ARC is irrelevant. ZFS
isn't consuming the memory and if it did, it would give it up, given enough
memory pressure.  The proxy tier is simply a proxy to another system in the
back end which also runs nginx.

The backend systems, run nginx, Postgres, and some other utility scripts to
push bits where they need to go. The backend systems do, as reported by
zpool iostat, about 10k writes/second (up to about 50MB/s). There are way
more back end systems than proxy tier systems and the backend runs just
fine. Most will achieve a year of uptime in November.

I have two radically different behaviors, on the same hardware platform,
running essentially the same software stack where the major difference is
the fire hose of network activity.

In addition, to say that a zone needs to be rebooted every twenty-four hours
is nuts. If this is what you are doing, then I fully do not understand your
position.

j.



___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] [developer] Re: Memory usage concern

[Jason Matthews]

-Original Message-
From: Brendan Gregg [mailto:brendan.gr...@joyent.com] 


> A full "echo ::kmastat | mdb -k" before reboot may show where the 
> memory is or isn't.

I am not a point where I would normally reboot it, but we are pretty well
deep into memory utilization for the size of the apps that run...

On the proxy tier, it seems that the DCE cache is way out of out of line.
What are the tunables here? 


Thanks,
j.


Here is a dump from a box on the proxy tier...

root@www003:~# echo ::kmastat |mdb -k |sort -nrk 5 |head
kmem_va_40964096 2964721 2964736 12143558656B   3023570
0
dce_cache152 73389856 73389862 11561725952B  73389938
0
zio_data_buf_131072   131072  43616  43644 5720506368B893451 0
kmem_va_16384  16384  51737  51744  847773696B 51969 0
zio_buf_16384  16384  51386  51475  843366400B   5324046 0
tcp_conn_cache  1808  17352  88512  181272576B 1597914226 0
kmem_va_81928192  22097  22112  181141504B 37356 0
streams_dblk_1040   1152 42 106253  124346368B 1582704292 0
kmem_bufctl_cache 24 3527424 3527541   86519808B   3750227 0
arc_buf_hdr_t176 345516 345576   64339968B   1199529 0
root@www003:~# echo ::memstat |mdb -k
Page SummaryPagesMB  %Tot
     
Kernel3721822 14538   44%
ZFS File Data 1448069  5656   17%
Anon84512   3301%
Exec and libs5085190%
Page cache  15805610%
Free (cachelist)22861890%
Free (freelist)   3072717 12002   37%

Total 8370871 32698
Physical  8370870 32698

And here is one from the database tier...

root@db001:~# echo ::kmastat |mdb -k |sort -nrk 5 |head
zfs_file_data_81928192 2991810 7365904 60341485568B 3495907153 0
kmem_va_4096  4096 4965999 6615520 27097169920B  73888636 0
zio_data_buf_8192 8192 2991120 2991810 24508907520B 3108123433 0
kmem_va_1638416384 216265 1442656 23636475904B 224812598 0
zfs_file_data_buf 24814600192B 24814600192B 24814600192B 15532060657
arc_buf_hdr_t   176 56809623 61605434 11469811712B 489897048 0
zio_buf_131072131072  83335  83370 10927472640B 624255183 0
zio_buf_16384  16384 213952 214677 3517267968B 866096686 0
kmem_alloc_3232 54596040 95296625 3122679808B 4035614613 0
kmem_va_81928192  61443 213888 1752170496B 483211721 0
root@db001:~# echo ::memstat |mdb -k
Page SummaryPagesMB  %Tot
     
Kernel   10971334 42856   44%
ZFS File Data 6059601 23670   24%
Anon  5686726 22213   23%
Exec and libs9304360%
Page cache  53574   2090%
Free (cachelist)   140715   5491%
Free (freelist)   2225873  86949%

Total25147127 98230
Physical 25147125 98230
___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] [developer] Re: Memory usage concern

[Jason Matthews]

DCE usage continued...


I am guessing this is likely my problem...


root@web006:~# netstat -nd |wc -l
  704075

It is really twice as bad as that, as there are two proxy zones per physical
server.

I found this which outlines the fundamental problem.

http://smartos.org/2012/02/28/using-flamegraph-to-solve-ip-scaling-issue-dce
/

this also explains the 'reset' effect I see when low memory conditions are
reached.

Have any controls been installed in the kernel since 151a1?


Thanks,
j.
___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] Memory usage concern

[Jason Matthews]

> OK, sorry then. Same symptoms, different cause, albeit it's so 
> bad that it makes the OS virtually unusable for any serious work.

Something does seem wrong. I process billions (and billions to channel Carl
Sagan) of inserts into Postgres per day on the database tier. I consider
that to be serious work. Something definitely seems out of sorts with your
system.



j.

___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] ZFS remote receive

[Jason Matthews]

From: Michael Stapleton [mailto:michael.staple...@techsologic.com] 

> You could try to set the crypo algorithm to none if you do not 
> need encryption.
>
> ssh -c none 

That wont work with the shipped ssh. You could use netcat

target# nc -l -p 31337 | zfs recv data/path/etc

source# zfs send data/path/etc@snap | nc target 31337

that will do it without encryption.

If you want to use the shipped ssh with the least overhead consider using
arcfour. ssh -c arcfour

Also, if you are going over the WAN I strongly recommend using the HPN ssh.
I have tutorial
(https://broken.net/zfs/using-hpn-openssh-to-flee-the-roach-motel-aws/)on
how to use it. I routinely get 80 megabits/sec transferring data out of a
east coast availability zones at AWS to san Francisco.

In my view the HPN patches should just be incorporated into not only ssh,
but all network application stacks as a configuration option.

j.

___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] "bad mutex" crash?

[Jason Matthews]

I will take a shot in the dark here. I imagine the request will time out and
be deleted in a few days. Then you could try again -- but I am just
speculating. If Jesus was a web developer, that's how he would do it.

j.

-Original Message-
From: Dan Swartzendruber [mailto:dswa...@druber.com] 
Sent: Tuesday, October 30, 2012 9:56 PM
To: 'Discussion list for OpenIndiana'
Subject: Re: [OpenIndiana-discuss] "bad mutex" crash?


Wonderful fail on the illumos site.  I went to the page to sign up to submit
issues.  When the activation mail arrived, somehow I deleted it by mistake.
Guess what?  Apparently there is no way to request a new one.  I *can* click
on the link to change my password, but that doesn't help because the account
has not been activated, so I'm stuck.  Fail 2 is the apparent lack of any
email address ("contact us") anywhere I can see on the top-level page.
Seriously?  I guess I will now create an email alias and a different user
name and see if *that* works.  Sigh...



___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss
___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] hardware specs for illumos storage/virtualization server

[Jason Matthews]

Sent from Jasons' hand held

On Nov 17, 2012, at 1:06 PM, Jim Klimov  wrote:
> 
> My box did have uncorrected raidz2 errors and couldn't even point
> to a failed drive - it's as if they all scratched the same region
> at once...


> which may have been caused by power jerks or something,
> with all drives accessing same areas in unison. 

western digital drives on an adaptec controller by chance?

when i was at aol a few years back we had several hundred, may a thousand 24 
bay supermicro boxes with adaptec controllers. about once a month a box would 
turn up for service with a hand full of corrupted files. this happened on 
mirrors, raid 5, and raid 6. 

this even happened to one of my osol thumbnail servers (about 1b images on the 
pool).  we always suspect a wd-adaptec problem but the vendors just pointed 
fingers at each other. 

it did seem to occur on the hardware platforms and our sample size was in the 
tens of thousands. 

j. 
___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] Future of OI

[Jason Matthews]

> I wonder if CheapBytes is still around

Cheaper bandwidth probably delivered cheap bytes cheaper than CheapBytes.

j. 
___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] Finding pools after reinstall

[Jason Matthews]

> BUT, when I try to find the zpools on the old disk with zpool
> import, it says nada. And format only lists one disk besides the boot
> disk. So, ideas?

Which motherboard are you using?
Do you have the drives connected to an HBA or the motherboard? If HBA, which
HBA?

If they are connected to the motherboard, and the ports are sata, your best
chance of making them work is to first try the ports in 'AHCI' mode. If that
doesn't work, select 'legacy' or some synonym there of. If that doesn't
work, go back to AHCI and let's find you a driver if possible.

Some HBAs have drivers distributed directly by the manufacturer -- such as
adaptec. If you have an adaptec card or embedded adaptec, you'll need to
download the solaris driver from adaptec and install it.

I look forward to you describing your system.

Thanks,
j.
___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] Knowledge Base for Compiling and InstallingSoftware Not Available as Packages.

[Jason Matthews]

> Perhaps some of you all might like to contribute to it, too?

This is an interesting topic.

I am also fond of reproducible processes. I learned from a colleague long
ago to write Makefiles which in turn run other make files to boot strap an
entire software stack -- sort of a lord of the rings, one ring to rule them
all sort of setup.  I was introduced to this concept about twelve years ago
and I have been using it ever since.

I haven't bridged the gap between my "/usr/local" build and pkg formatted
packages but I can't help but wonder if it is better to supply an automated
build system rather than manual instructions -- any automated build system
that worked would be preferable to doing something manually.

A few years ago I thought about releasing the whole thing in a VirtualBox
image because then it is fully plug and play but never got around to it. One
down side is the image would be rather large to accommodate the large number
of packages now included in my build system.

I am not poopooing your idea in any way. I am just wondering if we can take
it up a level or two.

j.
___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] Knowledge Base for Compiling andInstallingSoftware Not Available as Packages.

[Jason Matthews]

> For most (?) of us, we build a server once or twice a year.  

Some of us, build them hundreds at time, multiple times per year -- then
multiply zones on top of that. Some might do it thousands of times per year.


We may not be trying to solve the types of problems. 

For what it is worth, Makefiles are a nice compromise. Since you are
compiling software already you probably already have a working knowledge of
make. 

Thanks anyway.

j.
___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] Finding pools after reinstall

[Jason Matthews]

Sent from Jasons' hand held

On Dec 20, 2012, at 7:05 AM, "Edward Ned Harvey (openindiana)" 
 wrote:

> But he didn't change the hardware - in the OP he said he replaced one disk 
> (the boot disk) and installed openindiana where he was previously running 
> opensolaris.

sure. he also didnt say that he didnt turn any knobs. perhaps the cmos battery 
is dead and settings were lost. maybe he didnt perceive the loss of accurate 
time to be a big deal. 

maybe he didnt know the clock was off or settings were lost because as you 
suggest he never went into the bios to check. 

when troubleshooting i prefer to start on step one, not step five. how about a 
little patience while we work through this?

j. 
___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] ssh root login

[Jason Matthews]

# rolemod -K type=normal root
# vi /etc/ssh/sshd_config
PermitRootLogin yes
# svcadm restart ssh

Tada.

j.

-Original Message-
From: Ray Arachelian [mailto:r...@arachelian.com] 
Sent: Monday, January 14, 2013 12:32 PM
To: Discussion list for OpenIndiana
Subject: Re: [OpenIndiana-discuss] ssh root login

On 01/12/2013 06:01 PM, Bob Friesenhahn wrote:
> I am trying to accomplish ssh root login with a forced command via an
> entry in /root/.ssh/authorized_keys.  This is to support my home-made
> backup system.  The strategy is already working for Solaris 10, Apple
> OS X, Linux, and FreeBSD hosts.  However, it is failing for
> OpenIndiana and I am having difficulty determining why.
>

Check root's entry in /etc/user_attr, if it's set to role, you'll never
be able to login as root.
You can run "rolemod -K type=normal root" to turn it back into a user.


___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss
___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] Are there any known problems with OI and jumbo frames MTU 9000

[Jason Matthews]

Flo schreef:
>
> My question:
> Do I get a performance gain with a MTU size of 9000 and are there 
> known issues with OpenIndiana, link-aggregation and a MTU size of 9000?


I use this configuration with no issues. For an MTU of 9000 with VLANs  on
the host, make sure the switch MTU is set to 9014 or greater.

Thanks,
j.
___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

[OpenIndiana-discuss] zpool iostat

[Jason Matthews]

I am confused. Zpool iostat -v worn 5 1000 reports this...

The three disks with goose eggs are offline. I am burning out the last few
cells on all four SSDs so I can RMA.  However, I am confused why the only
disk that works reports 149 write operations while the pool reports 465.

How does that work? The system is running 151a

Thanks,
j.

  capacity operationsbandwidth
pool   alloc   free   read  write   read  write
-  -  -  -  -  -  -
worn100G  18.9G 86465  10.9M  10.4M
  mirror100G  18.9G 86465  10.9M  10.4M
c2t500A07510316E93Fd0  -  -  0  0  0  0
c2t500A07510316E915d0  -  -  0  0  0  0
c2t500A07510316E92Dd0  -  -  0  0  0  0
c2t500A07510316E8C8d0  -  - 86149  10.9M  10.4M
-  -  -  -  -  -  -

  capacity operationsbandwidth
pool   alloc   free   read  write   read  write
-  -  -  -  -  -  -
worn100G  18.9G 32368  4.01M  8.37M
  mirror100G  18.9G 32368  4.01M  8.37M
c2t500A07510316E93Fd0  -  -  0  0  0  0
c2t500A07510316E915d0  -  -  0  0  0  0
c2t500A07510316E92Dd0  -  -  0  0  0  0
c2t500A07510316E8C8d0  -  - 32139  4.01M  8.37M
-  -  -  -  -  -  -

  capacity operationsbandwidth
pool   alloc   free   read  write   read  write
-  -  -  -  -  -  -
worn100G  18.9G 67600  8.45M  11.1M
  mirror100G  18.9G 67600  8.45M  11.1M
c2t500A07510316E93Fd0  -  -  0  0  0  0
c2t500A07510316E915d0  -  -  0  0  0  0
c2t500A07510316E92Dd0  -  -  0  0  0  0
c2t500A07510316E8C8d0  -  - 67206  8.45M  11.1M
-  -  -  -  -  -  -

  capacity operationsbandwidth
pool   alloc   free   read  write   read  write
-  -  -  -  -  -  -
worn100G  18.9G 67462  8.37M  7.82M
  mirror100G  18.9G 67462  8.37M  7.82M
c2t500A07510316E93Fd0  -  -  0  0  0  0
c2t500A07510316E915d0  -  -  0  0  0  0
c2t500A07510316E92Dd0  -  -  0  0  0  0
c2t500A07510316E8C8d0  -  - 67174  8.37M  7.82M
-  -  -  -  -  -  -
___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] OI Crash

[Jason Matthews]

On Jan 15, 2013, at 7:10 PM, Ian Collins  wrote:
> 
> If there isn't any evidence of a crash, there's a fair chance you have a 
> hardware problem.

i have decent number of identical production boxes. 

about once per quarter one of them spontaneously reboots leaving no trace as to 
why. it is never the same box twice. 

the first couple of times i offlined the systems and ran diagnostics on them. i 
ran memtest for two weeks. i checked SEL, etc found butt-kiss. 

i came to the conclusion it is just something that happens. it is likely a 
driver issue. since my srchitecture can absorb such failures i havent spent 
slot of time on it. 

 i am still on 151a so perhaps there is a fix that i just dont have it yet. 
whatever my reboot problem is, it is not a hardware problem. 


j. 
___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] OI Crash

[Jason Matthews]

to this end, redirect your console to a serial port and put a serial recorder 
on it. they cost maybe 60$ but can be handy to catch output from panics. 

j. 

Sent from Jasons' hand held

On Jan 19, 2013, at 11:04 AM, Reginald Beardsley  wrote:

> Having a console window open and checking it periodically can be very 
> helpful.   Such events will get logged to the console. I recently had a 
> correctable event show up in mine. There's probably a way to have the events 
> trigger an email if desired.
> 
> Have Fun!
> Reg
> 
> --- On Sat, 1/19/13, Aurélien Larcher  wrote:
> 
>> From: Aurélien Larcher 
>> Subject: Re: [OpenIndiana-discuss] OI Crash
>> To: "Discussion list for OpenIndiana" 
>> Date: Saturday, January 19, 2013, 12:30 PM
>> Hi,
>> Has someone mentioned using 'fmdump' ?
>> 
>> With this tool I discovered that I had issues with an
>> unreliable disk
>> controller on my workstation with the consequence of OI
>> freezing approx.
>> every 2months.
>> In my case ZFS is getting the fault and standby until
>> resolution of the
>> issue, thus yielding an indefinite wait for disk I/O to
>> resume.
>> Best
>> 
>> Aurelien
>> 
>> 
>> On Sat, Jan 19, 2013 at 3:19 PM, Reginald Beardsley 
>> wrote:
>> 
>>> One time when I happened to look, I saw that the Ultra
>> 60 I used at work
>>> had been up for over 18 months.
>>> 
>>> If a sys admin told me he wanted to reboot a system
>> once a week, "just in
>>> case" he'd be looking for a new job very soon or else
>> sent back to the PC
>>> support pool.
>>> 
>>> BTW The reason that 11/780 era admins did not want to
>> shut machines down
>>> was primarily the problems posed by hundreds, if not
>> thousands of
>>> mechanical connectors some of which if allowed to cool
>> would lose contact.
>>>   The cure was simple, but tedious, you went around
>> reseating circuit boards
>>> and cabling and powered up again. There are a lot of
>> boards and cables in a
>>> well populated 11/780 especially if its got an
>> FPS-120B, Gould-DeAnza
>>> graphics processor and a Versatec plotter attached
>> along w/ the usual disk
>>> and tape drives.
>>> 
>>> One summer weekend in Dallas, my group moved across
>> town.  So our
>>> workstations spent the day in a moving van probably at
>> 130+ F.  Monday
>>> morning several would not boot until I went around and
>> reseated the disk
>>> drive cables.
>>> 
>>> Voodoo has no place in computing.
>>> 
>>> Have Fun!
>>> Reg
>>> 
>>> ___
>>> OpenIndiana-discuss mailing list
>>> OpenIndiana-discuss@openindiana.org
>>> http://openindiana.org/mailman/listinfo/openindiana-discuss
>> 
>> 
>> 
>> -- 
>> ---
>> LARCHER Aurélien|
>> KTH, School of Computer Science and
>> Communication
>> Work: +46 (0) 8 790 71 42   |
>> Lindstedtsvägen 5, Plan 5
>> Mob.: +46 (0) 7 09 46 40 17 | 100 44 Stockholm, SWEDEN
>> ---
>> Praise the Caffeine embeddings ...
>> ___
>> OpenIndiana-discuss mailing list
>> OpenIndiana-discuss@openindiana.org
>> http://openindiana.org/mailman/listinfo/openindiana-discuss
> 
> ___
> OpenIndiana-discuss mailing list
> OpenIndiana-discuss@openindiana.org
> http://openindiana.org/mailman/listinfo/openindiana-discuss

___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] Mapping devices in OI to physical devices

[Jason Matthews]

i wrote my own leveraging some lsi tools, fmtopo, and 100 ugly lines of perl. 

in some smaller systems where i dont have lsi cards i just label the drive 
carrier with the last five chars of the wwn. 

i have offered to pay a bounty gir someone solve this problem for 
newisys/sanmina-sc disk chassis. alas, no one responded. 

nexenta has a solution but they are not sharing. 

j. 

Sent from Jasons' hand held

On Jan 22, 2013, at 9:26 AM, Len Zaifman  wrote:

> We have just had a major system meltdown and it took several days to fix.
> 
> What we would have liked is 2 things we had on thumpers (Old SUN ZFS systems)
> 
> 1) A tool to show the mapping of a solaris device name to a physical location
> 2) A tool to turn on the light on a disk via its solaris device name.
> 
> The process below is too painful, and we have other devices whose disks may 
> go bad. Does either 1 or 2 above exist in openindiana? I could not find it, 
> if it does.
> 
> Thanks.
> 
> The issue was:
> 
> OI (OpenIndiana Development oi_151a X86) reported:
> 
> 
> Jan 22 10:57:43 archivea scsi: [ID 107833 kern.warning] WARNING: 
> /pci@7a,0/pci8086,3408@1/pci1000,3040@0 (mpt_sas10):
> Jan 22 10:57:43 archiveaDisconnected command timeout for Target 18
> Jan 22 10:57:43 archivea scsi: [ID 365881 kern.info] 
> /pci@7a,0/pci8086,3408@1/pci1000,3040@0 (mpt_sas10):
> Jan 22 10:57:43 archiveaLog info 0x3114 received for target 18.
> Jan 22 10:57:43 archiveascsi_status=0x0, ioc_status=0x8048, 
> scsi_state=0xc
> 
> zfs performance went through the floor  and was intolerable(< 1 mb/sec where 
> we had hundreds of MB/sec for resilver/scrubs and 100 MB/sec through the 
> filesystem).
> 
> The defective disk was one of 45 disks in a Supermicro Jbod system 
> (SC847E26-RJBOD1)
> 
> We finally found which disk it was by comparing serial numbers reported by 
> iostat, disks that reported errors and the actual disk serial number (we 
> pulled all 45 disks out to do this mapping). we do not want to repeat this 
> process for our other devices.
> 
> 
> Len Zaifman
> Systems Manager, High Performance Systems
> The Centre for Computational Medicine
> The Hospital for Sick Children
> 555 University Ave.
> Toronto, Ont M5G 1X8
> 
> tel: 416-813-5513
> email: leona...@sickkids.ca
> 
> 
> 
> This e-mail may contain confidential, personal and/or health 
> information(information which may be subject to legal restrictions on use, 
> retention and/or disclosure) for the sole use of the intended recipient. Any 
> review or distribution by anyone other than the person for whom it was 
> originally intended is strictly prohibited. If you have received this e-mail 
> in error, please contact the sender and delete all copies.
> 
> ___
> OpenIndiana-discuss mailing list
> OpenIndiana-discuss@openindiana.org
> http://openindiana.org/mailman/listinfo/openindiana-discuss

___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] HA or keepalived in OI / solaris

[Jason Matthews]

be advised that i have only been able to get vrrp to work in global zones. 

i found references to people thinking it could work in zones with the right 
incantation but alas i couldnt find it. perhaps my jarred dragon breath was to 
dated. 

otherwise vrrp more or less works as expected. 

j. 

Sent from Jasons' hand held

On Jan 28, 2013, at 7:30 AM, James Carlson  wrote:

> On 01/28/13 09:11, Ram Chander wrote:
>> Hi,
>> 
>> I want a floating IP that can switch between two servers for HA. Since
>> keepalived is only for linux, is there anyother alternative in OI   ?
> 
> The VRRP (Virtual Router Redundancy Protocol) and ILB (integrated load
> balancer) features should be able to do this.  See this for a possible
> starting point:
> 
> http://docs.oracle.com/cd/E23824_01/html/821-1453/gkkky.html
> 
> I have personal reservations against using VRRP in this manner, but
> there are obviously quite a few who disagree.
> 
> -- 
> James Carlson 42.703N 71.076W 
> 
> ___
> OpenIndiana-discuss mailing list
> OpenIndiana-discuss@openindiana.org
> http://openindiana.org/mailman/listinfo/openindiana-discuss

___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] Auto snapshots

[Jason Matthews]

i am rev locked for now on 151a1 and have never had oroblems with snapshots 
once i started the service. 

i dont use the gui so i cant speak to those functions but the snapshots work. 

j. 

Sent from Jasons' hand held

On Feb 1, 2013, at 8:29 AM, Harry Putnam  wrote:

> Probably well trodden ground, but looking back into the past list
> discussions I'm not noticing a straight answer about this.
> 
> I quit working with OI some 1.5 yrs ago and have forgotten most
> of what I ever learned about using it.
> 
> I do remember there was a problem with auto snapshots in the early
> builds available shortly after the break from opensolaris to
> openindiana.  I think build 148 to 150 were the last builds I used.
> 
> Has that issue been completely resolved... auto snapshots now work
> with no problems?
> 
> 
> ___
> OpenIndiana-discuss mailing list
> OpenIndiana-discuss@openindiana.org
> http://openindiana.org/mailman/listinfo/openindiana-discuss

___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] sas multipath

[Jason Matthews]

randy,

if you do absolutely nothing special - meaning no stmsboot or anything else - 
the mpt2 driver will default to round robin. failover is sub-sub second. 

my configuration works like this and is based on 151a1.  if you undo all the 
old school fancy stuff it will "just work"

thanks,
j. 

Sent from Jasons' hand held

On Feb 6, 2013, at 4:03 AM, Randy S  wrote:

> 
> Hi all,
> 
> I have a test system running with OIa7, very much ram, 2 LSI SAS 9207-4i4e 
> hba's, 1 sm jbod containing seagate 3Tb sas drives. 
> 
> I have enabled the multipathing with stmsboot and set it to use 
> logical-blocksize, since round-robin symmetric-option isn't functioning.
> 
> During tests I can see that the system sees the two path's  with mpathadm and 
> simple dd tests show that all disks are all accessed at almost maximum speed 
> with minimum latency according to iostat. 
> We started to test with 4 sas disks in 2 striped mirrors. 
> 
> All good, but shouldn't it be possible to pull one of the sas cables without 
> any disruption to the storage, forcing it to use only one path? (this way 
> simulating a broken hba).
> When we do this, the whole system grinds to a screeching halt, being almost 
> unresponsive.
> 
> What am I doing wrong. Maybe somebody can point in in the right direction to 
> troubleshoot.
> 
> Rgrds,
> 
> Randy
> 
> 
> 
> 
> 
> 
> ___
> OpenIndiana-discuss mailing list
> OpenIndiana-discuss@openindiana.org
> http://openindiana.org/mailman/listinfo/openindiana-discuss

___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] zlogin question

[Jason Matthews]

Hi Jim,

> # for Z in `zoneadm list`; do zlogin "$Z" "zonename"; don
>
>   I don't see any apparent errors in the scripting side, except that
> maybe zlogin somehow acts like exec and ceases the subshell?
>
> Any ideas? Is it a bug or my mistake?

Not to be an ass, but did you mean, done, instead of don ;-)

I am sorry to say, I have to put this in the, it works for me category. 

Infact, all of your examples work for me. I generally use the last form.

root@db032:~# for z in `zoneadm list |grep -v global`; do zlogin $z
hostname; done
shard029b
shard032a
shard064a
shard061b
___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] sas multipath

[Jason Matthews]

> Which sas drives do you use?

In terms of spinning rust, my environment is uniform with ST9146852SS. I
have a significant number >10^3 of them running. These drives take a
pounding every day and I have only experienced about six drive failures in
one year of time. I am not sure if Seagate gets the credit for it or
Newisys/Sanmina-SC who burns the drives in. As far as failures go one or two
of them have been electrical in nature, most failures are caught with CRC
errors during the monthly scrub.

I am sure these drvies are EOL by now due to their small size. I am starting
a migration to Intel 910s. Retro fitting them in my chassis with the
DDRdrive X1 and two LSI cards has proven challenging but I finally found a
way to make all devices happy.

I suspect this time next year, I may have no spinning rust at all as I can
replace disk shelves with PCI-E cards. This allows for more disk I/O and the
same (or technically smaller) foot print in the data center assuming my CPUs
can keep up with it.

j.
___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] zlogin question

[Jason Matthews]

> shouldn't you use zoneadm list -cv ? so that only realy running zones are 
> listed and inspected? Always expect the uninspected

Sure, but wasn't the focus on the loop and not how to generate an accurate
list?

j.
___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] Zfs import fails (OpenIndiana-discuss Digest, Vol 31, Issue 28)

[Jason Matthews]

wow. this would be my initial hale mary play. 

rm the zfs cache file in etc/zfs
devfsadm -C
does touch /reconfigure still work? do it
reboot -p (fast boot is not my favorite)

might be worth checking the cabling. 

then try the import. after that i'd come to this list. 

were there any firmware upgrades involved?

j. 

Sent from Jasons' hand held

On Feb 7, 2013, at 6:09 PM, Ong Yu-Phing  wrote:

> I had a similar situation where after export, my disk paths were wrongly 
> picked up by ZFS, even though they were recognised by the system.
> 
> ===
> The zpool import fails with this sort of message:
>pool1   UNAVAIL  insufficient replicas
>  mirror-0  UNAVAIL  corrupted data
>c3t0d0  ONLINE
>c3t1d0  ONLINE
> 
> When I check the disk labels on the disks:
> 
> zdb -l /dev/dsk/c3t0d0s0
> 
> LABEL 0
> 
>version: 28
>name: 'pool1'
>state: 1
>txg: 7173951
>pool_guid: 8370873525947507187
>hostid: 13162267
>hostname: 'openindiana'
>top_guid: 15064987019855796782
>guid: 4751459059166773513
>vdev_children: 3
>vdev_tree:
>type: 'mirror'
>id: 0
>guid: 15064987019855796782
>metaslab_array: 30
>metaslab_shift: 34
>ashift: 9
>asize: 1998985625600
>is_log: 0
>create_txg: 4
>children[0]:
>type: 'disk'
>id: 0
>guid: 4751459059166773513
>path: '/dev/dsk/c2t0d0s0'
>devid: 'id1,sd@n600605b002e26410183d783b0e56515a/a'
>phys_path: '/pci@0,0/pci8086,3410@9/pci1014,3c7@0/sd@0,0:a'
>whole_disk: 1
>DTL: 4119
>create_txg: 4
>children[1]:
>type: 'disk'
>id: 1
>guid: 7277976899319815787
>path: '/dev/dsk/c2t1d0s0'
>devid: 'id1,sd@n600605b002e264101839b39f123c1210/a'
>phys_path: '/pci@0,0/pci8086,3410@9/pci1014,3c7@0/sd@1,0:a'
>whole_disk: 1
>DTL: 4118
>create_txg: 4
> 
> So I noticed that even though the disk is c3t0d0, the path shows in the disk 
> label as c2t0d0s0 (and similarly for the "child"/partner mirrored disk, 
> c3t1d0, yet path shows as c2t1d0), is this what is causing the really strange 
> behaviour, when even though the devices are considered ONLINE, the pool 
> itself is UAVAIL?
> ===
> 
> Maybe you can check if you are hitting the same situation? Unfortunately, I 
> never was able to fix this (caused by a live upgrade of OI from 148 to 
> 151a3); instead, I booted from a live USB, then did a ZFS send to transfer 
> the filesystems to another server.
> 
> On 07/02/2013 21:47, Ram Chander wrote:
>> "format" detects all 108 disks . I suspect it could be zfs issue. I ran out
>> of options now.
>> 
>> root@host1:~# format
>> Searching for disks...done
>> AVAILABLE DISK SELECTIONS:
>>0. c1t0d0 
>>   /pci@0,0/pci8086,340b@4/pci1028,1f17@0/sd@0,0
>>1. c5t1d0 
>>   /ethdrv/sd@1,0
>>2. c5t1d1 
>>   /ethdrv/sd@1,1
>>3. c5t1d2 
>>   /ethdrv/sd@1,2
>>4. c5t1d3 
>>   /ethdrv/sd@1,3
>>5. c5t1d4 
>>   /ethdrv/sd@1,4
>>   etc
>> 
>> root@host1~# devfsadm -Cv
>> root@host1:~#
>> 
>> 
>> 
>> On Thu, Feb 7, 2013 at 6:44 PM, Sa?o Kiselkov wrote:
>> 
>>> You have an issue with conectivity to your drives on the Coraid HBA
>>> card. I suggest querying your HBA via its management tools to make sure
>>> you can discover all the drives on your network. Chances are, they're
>>> not all visible, which is why your pool is having trouble.
>>> 
>>> --
>>> Saso
>>> 
>>> On 02/07/2013 01:49 PM, Ram Chander wrote:
 The drives are in Coraid  connected to the server via Coraid 10G HBA
>>> card.
 Its exported and imported on the same host but after OS upgrade ( format
>>> OS
 disk and install again ). Before OS upgrade, zpool export was issued and
 when tried to import it faults as below. The pool is not functional in
>>> any
 system now.  Tried -d /dev/dsk option but no luck.
 
 
 root@storage1:~# zpool import
pool: pool1
  id: 10136140719439709374
   state: FAULTED
  status: The pool was last accessed by another system.
  action: The pool cannot be imported due to damaged devices or data.
 The pool may be active on another system, but can be imported
>>> using
 the '-f' flag.
see: http://illumos.org/msg/ZFS-8000-EY
  config:
 
 pool1  FAULTED  corrupted data
   raidz1-0   ONLINE
 c5t1d0   UNAVAIL  corrupted data
 c5t1d1   UNAVAIL  corrupted data
 c5t1d2   UNAVAIL  corrupted data
 c5t1d3   UNAVAIL  corrupted data
 c5t1d4   UNAVAIL  corrupted data
   raidz1-1   ONLI

Re: [OpenIndiana-discuss] how to make illumos kernal in to openindinaboot cd

[Jason Matthews]

Say what?

j.

-Original Message-
From: carl brunning [mailto:ca...@flamewarestudios.com] 
Sent: Friday, February 08, 2013 11:54 AM
To: openindiana-discuss@openindiana.org
Subject: [OpenIndiana-discuss] how to make illumos kernal in to
openindinaboot cd

Hi
Am able to make the illumos-gate  source code
Am able to make openindiana boot cd

What I need to know is what do I copy from the illumos stuff after build and
were to for  the cd making part

This bit of instruction I cannot find (I may be blind :) )

This way I can boot and install


Thanks and keep up the good work

Carl
___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss
___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] Slow zfs writes

[Jason Matthews]

I am going to offer to obvious advice...

How full is your pool?  Zpool performance degrades as the pool fills up and
the tools don't tell you how close you are to the cliff -- you find the
cliff on your own by falling off of it. As a rule of thumb, I keep
production system less than 70% utilized.

Here is a real life example. On a 14.5TB (configured) pool, I found the
cliff with 250+GB still reported as free. The system continued to write to
the pool but throughput dismal.

Is your pool full?

j.

-Original Message-
From: Ram Chander [mailto:ramqu...@gmail.com] 
Sent: Monday, February 11, 2013 4:48 AM
To: Discussion list for OpenIndiana
Subject: [OpenIndiana-discuss] Slow zfs writes

Hi,

My OI box is expreiencing slow zfs writes ( around 30 times slower ).
iostat reports below error though pool is healthy. This is happening in
past 4 days though no change was done to system. Is the hard disks faulty ?
Please help.


# zpool status -v
root@host:~# zpool status -v
  pool: test
 state: ONLINE
status: The pool is formatted using a legacy on-disk format.  The pool can
still be used, but some features are unavailable.
action: Upgrade the pool using 'zpool upgrade'.  Once this is done, the
pool will no longer be accessible on software that does not support
feature flags.
config:

NAME STATE READ WRITE CKSUM
test   ONLINE   0 0 0
  raidz1-0   ONLINE   0 0 0
c2t0d0   ONLINE   0 0 0
c2t1d0   ONLINE   0 0 0
c2t2d0   ONLINE   0 0 0
c2t3d0   ONLINE   0 0 0
c2t4d0   ONLINE   0 0 0
  raidz1-1   ONLINE   0 0 0
c2t5d0   ONLINE   0 0 0
c2t6d0   ONLINE   0 0 0
c2t7d0   ONLINE   0 0 0
c2t8d0   ONLINE   0 0 0
c2t9d0   ONLINE   0 0 0
  raidz1-3   ONLINE   0 0 0
c2t12d0  ONLINE   0 0 0
c2t13d0  ONLINE   0 0 0
c2t14d0  ONLINE   0 0 0
c2t15d0  ONLINE   0 0 0
c2t16d0  ONLINE   0 0 0
c2t17d0  ONLINE   0 0 0
c2t18d0  ONLINE   0 0 0
c2t19d0  ONLINE   0 0 0
c2t20d0  ONLINE   0 0 0
c2t21d0  ONLINE   0 0 0
c2t22d0  ONLINE   0 0 0
c2t23d0  ONLINE   0 0 0
  raidz1-4   ONLINE   0 0 0
c2t24d0  ONLINE   0 0 0
c2t25d0  ONLINE   0 0 0
c2t26d0  ONLINE   0 0 0
c2t27d0  ONLINE   0 0 0
c2t28d0  ONLINE   0 0 0
c2t29d0  ONLINE   0 0 0
c2t30d0  ONLINE   0 0 0
  raidz1-5   ONLINE   0 0 0
c2t31d0  ONLINE   0 0 0
c2t32d0  ONLINE   0 0 0
c2t33d0  ONLINE   0 0 0
c2t34d0  ONLINE   0 0 0
c2t35d0  ONLINE   0 0 0
c2t36d0  ONLINE   0 0 0
c2t37d0  ONLINE   0 0 0
  raidz1-6   ONLINE   0 0 0
c2t38d0  ONLINE   0 0 0
c2t39d0  ONLINE   0 0 0
c2t40d0  ONLINE   0 0 0
c2t41d0  ONLINE   0 0 0
c2t42d0  ONLINE   0 0 0
c2t43d0  ONLINE   0 0 0
c2t44d0  ONLINE   0 0 0
spares
  c5t10d0AVAIL
  c5t11d0AVAIL
  c2t45d0AVAIL
  c2t46d0AVAIL
  c2t47d0AVAIL



# iostat -En

c4t0d0   Soft Errors: 0 Hard Errors: 5 Transport Errors: 0
Vendor: iDRACProduct: Virtual CD   Revision: 0323 Serial No:
Size: 0.00GB <0 bytes>
Media Error: 0 Device Not Ready: 5 No Device: 0 Recoverable: 0
Illegal Request: 1 Predictive Failure Analysis: 0
c3t0d0   Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: iDRACProduct: LCDRIVE  Revision: 0323 Serial No:
Size: 0.00GB <0 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c4t0d1   Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: iDRACProduct: Virtual Floppy   Revision: 0323 Serial No:
Size: 0.00GB <0 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0


root@host:~# fmadm faulty
---   --
-
TIMEEVENT-ID  MSG-ID
SEVERITY
---   --
-
Jan 05 08:21:09 7af1ab3c-83c2-602d-d4b9-f9

Re: [OpenIndiana-discuss] sas multipath

[Jason Matthews]

Randy,

Can you tell me more about your set up? Is it dual domain/dual controller?

 | disk shelf |
|--- |I/O controller 0|
   LSI Port0||+   |
--| --->||+   |
Server|  |I/O controller 1|
  |  |+   |
__|>||+   |
   LSI Port1|--- |I/O controller 2|
 ||

Sorry for the ascii graphics. 

Does your setup look something like this?

So, do you have two sff-8088 connecting to two different I/O controllers
which in turn either connect to each other or one or more other I/O
controllers?

If not, can you show me what it looks like. Also, did you actually test
failover or just view the results of mpathadm list lu for the path count?
When the system boots, I am pretty sure  a message is written to the console
about using round robin. Did you ever see this message?

Also, are your disks dual attached or are you only redundant to your I/O
modules?


Thanks,
j.

___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] rge0 flip-flop up and down

[Jason Matthews]

another instance of no good deed goes unpunished :)

Sent from Jasons' hand held

On Feb 21, 2013, at 9:54 AM, Maurizio Boriani  wrote:

> On Thu, 2013-02-21 at 17:15 +0100, Maurizio Boriani wrote:
>> Hi all,
>>  on a fresh openindiana 151a7 installation, my realtek flapping up and
>> down, but for nwam is always up. The host is a laptop hp probook 4540s.
>> There are some known issues, acpi or similar? 
>> 
>> tia,
> 
> found. By setting ip:dohwcksum=0 all seems ok.
> 
> bye
> 
> ___
> OpenIndiana-discuss mailing list
> OpenIndiana-discuss@openindiana.org
> http://openindiana.org/mailman/listinfo/openindiana-discuss

___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] Unable to load RAID driver

[Jason Matthews]

The latest phase is 15. I would try that, before loading an older version.

http://www.lsi.com/downloads/Public/Host%20Bus%20Adapters/Host%20Bus%20Adapt
ers%20Common%20Files/SAS_SATA_6G_P15/9211_8i_Package_P15_IR_IT_Firmware_BIOS
_for_MSDOS_Windows.zip


j.





-Original Message-
From: Lawrence [mailto:lawren_g...@yahoo.com] 
Sent: Monday, February 25, 2013 7:28 PM
To: Discussion list for OpenIndiana
Subject: Re: [OpenIndiana-discuss] Unable to load RAID driver

Hi Saso,

Would you be able to show me where to download that particular firmware?
I can only find this version on the LSI website
9211_8i_Package_P13.5_IR_IT_Firmware_BIOS_for_MSDOS_Windows 
Description: Package for Firmware and BIOS Upgrade on MSDOS and Windows 
Version:  13.00.66.00 
Size:  1.7M 
Regards.



 From: Sašo Kiselkov 
To: openindiana-discuss@openindiana.org 
Sent: Tuesday, February 26, 2013 1:29 AM
Subject: Re: [OpenIndiana-discuss] Unable to load RAID driver
 
On 02/25/2013 05:25 PM, Lawrence Giam wrote:
> Hi Saso,
> 
> The system is not loading the driver correctly. After booting into OI,
doing a prtconf | less shows the mpt_sas drivers not loaded. When I tried to
manually add, it show the error below.

What does "sas2flash -listall" report?

I'm currently running these firmware revisions without any trouble:

# ./sas2flash -listall
..[snip]..
Ctlr          FW Ver         NVDATA        x86-BIOS
---
SAS2008(B2)   13.00.57.00    0d.43.00.02    07.25.00.00
SAS2008(B2)   07.15.08.00    07.00.00.19    07.11.10.00

Perhaps you can try and reflash to these firmware revisions and retest
to see if the trouble isn't only caused by a "too new" firmware issue.

Cheers,
--
Saso

___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss
___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss
___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] Unable to load RAID driver

[Jason Matthews]

you can probably d/l older versions using the search feature on lsi site. it is 
painful for sure. 

alternately, i can check what i have on hand. i want to say i have 9, 11, 13.5 
and 15. i will check and reach out to you off list. 

j. 

Sent from Jasons' hand held

On Feb 26, 2013, at 8:00 AM, Lawrence Giam  wrote:

> Hi Jason,
> 
> I have already tried the P15 version, does work for me.
> 
> Sent from my iPad
> 
> On 26 Feb, 2013, at 12:39 PM, "Jason Matthews"  wrote:
> 
>> 
>> 
>> 
>> 
>> The latest phase is 15. I would try that, before loading an older version.
>> 
>> http://www.lsi.com/downloads/Public/Host%20Bus%20Adapters/Host%20Bus%20Adapt
>> ers%20Common%20Files/SAS_SATA_6G_P15/9211_8i_Package_P15_IR_IT_Firmware_BIOS
>> _for_MSDOS_Windows.zip
>> 
>> 
>> j.
>> 
>> 
>> 
>> 
>> 
>> -Original Message-
>> From: Lawrence [mailto:lawren_g...@yahoo.com] 
>> Sent: Monday, February 25, 2013 7:28 PM
>> To: Discussion list for OpenIndiana
>> Subject: Re: [OpenIndiana-discuss] Unable to load RAID driver
>> 
>> Hi Saso,
>> 
>> Would you be able to show me where to download that particular firmware?
>> I can only find this version on the LSI website
>> 9211_8i_Package_P13.5_IR_IT_Firmware_BIOS_for_MSDOS_Windows 
>> Description: Package for Firmware and BIOS Upgrade on MSDOS and Windows 
>> Version:  13.00.66.00 
>> Size:  1.7M 
>> Regards.
>> 
>> 
>> 
>> From: Sašo Kiselkov 
>> To: openindiana-discuss@openindiana.org 
>> Sent: Tuesday, February 26, 2013 1:29 AM
>> Subject: Re: [OpenIndiana-discuss] Unable to load RAID driver
>> 
>> On 02/25/2013 05:25 PM, Lawrence Giam wrote:
>>> Hi Saso,
>>> 
>>> The system is not loading the driver correctly. After booting into OI,
>> doing a prtconf | less shows the mpt_sas drivers not loaded. When I tried to
>> manually add, it show the error below.
>> 
>> What does "sas2flash -listall" report?
>> 
>> I'm currently running these firmware revisions without any trouble:
>> 
>> # ./sas2flash -listall
>> ..[snip]..
>> Ctlr  FW Ver NVDATAx86-BIOS
>> ---
>> SAS2008(B2)   13.00.57.000d.43.00.0207.25.00.00
>> SAS2008(B2)   07.15.08.0007.00.00.1907.11.10.00
>> 
>> Perhaps you can try and reflash to these firmware revisions and retest
>> to see if the trouble isn't only caused by a "too new" firmware issue.
>> 
>> Cheers,
>> --
>> Saso
>> 
>> ___
>> OpenIndiana-discuss mailing list
>> OpenIndiana-discuss@openindiana.org
>> http://openindiana.org/mailman/listinfo/openindiana-discuss
>> ___
>> OpenIndiana-discuss mailing list
>> OpenIndiana-discuss@openindiana.org
>> http://openindiana.org/mailman/listinfo/openindiana-discuss
>> 
>> 
>> ___
>> OpenIndiana-discuss mailing list
>> OpenIndiana-discuss@openindiana.org
>> http://openindiana.org/mailman/listinfo/openindiana-discuss
> 
> ___
> OpenIndiana-discuss mailing list
> OpenIndiana-discuss@openindiana.org
> http://openindiana.org/mailman/listinfo/openindiana-discuss

___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] Unable to load RAID driver

[Jason Matthews]

after reviewing the original request i am a bit confused. 

your output says you have a megaraid card. the 9211-11 isnt megaraid. 

i know some of those ibm cards can be cross flashed but are you sure you have 
an appropriate model?

j. 

Sent from Jasons' hand held

On Feb 25, 2013, at 5:48 AM, Lawrence  wrote:

> Hi All,
> 
> I have a IBM ServeRAID M1015 RAID controller which I tried to cross-flash to 
> LSA 9211-8i IT mode as mentioned in this website: 
> http://www.servethehome.com/ibm-serveraid-m1015-part-4/.
> 
> I have a IBM x3650 M3 server which I then use with the ServeRAID M1015 raid 
> card to test if my card is working in the IT mode as mentioned. I have tried 
> booting using OpenIndiana 151a and 151a7 LiveCD/USB to try and detect the 
> card. The system is able to detect the raid card but when I tried to add the 
> drivers "mpt_sas" to the system, I got the following errors:
> 
> root@openindiana:~# add_drv -i
>  "pci1014,3b1" mpt_sas 
> devfsadm: driver failed to attach:
>  mpt_sas 
> Warning: Driver (mpt_sas) successfully
>  added to system but failed to attach 
> 
> Feb 21 17:14:22 openindiana scsi: [ID
>  107833 kern.warning] WARNING:
>  /pci@0,0/pci8086,3410@9/pci1014,3b1@0
>  (mpt_sas0): 
> Feb 21 17:14:22 openindiana hard reset
>  failed! 
> Feb 21 17:14:22 openindiana scsi: [ID
>  107833 kern.warning] WARNING:
>  /pci@0,0/pci8086,3410@9/pci1014,3b1@0
>  (mpt_sas0): 
> Feb 21 17:14:22 openindiana mptsas
>  chip initialization failed 
> Feb 21 17:14:22 openindiana scsi: [ID
>  107833 kern.warning] WARNING:
>  /pci@0,0/pci8086,3410@9/pci1014,3b1@0
>  (mpt_sas0): 
> Feb 21 17:14:22 openindiana attach
>  failed
> 
> Boot to the RAID controller, I gathered the following information:
> Firmware Version: 2.70.74-1038
> FW Package Version: 20.5.1-0014
> FW Time: Oct 13 2010, 13:52:10
> WebBIOS Version: 4.0-e_24-Rel
> SubVendor ID: 0x1014
> SubDevice ID: 0x3b1
> 
> Doing a scanpci:
> sudo scanpci | less #search for LSI
> pci bus 0x0010 cardnum 0x00 function 0x00: vendor 0x1000 device
>0x0073
>  LSI Logic / Symbios Logic MegaRAID SAS 2008 [Falcon]
> ===
> 
> So it does find the card.  And then prtconf:
> =
> jack@openindiana:~$ prtconf -Dv | less #search for 73
> pci1014,3b1 (driver name: mpt_sas)
> Hardware properties:
> name='pci-msix-capid-pointer' type=int items=1
> value=00c0
> name='pci-msi-capid-pointer' type=int items=1
> value=00a8
> name='assigned-addresses' type=int items=15
>
> value=81100010..1000..0100.83100014..9294..4000.8310001c.000
> 0.9290..0004
> name='reg' type=int items=20
>
> value=0010.....01100010....0100.03100014.000
> 0...4000.0310001c....0004
> name='compatible' type=string items=13
> value='pciex1000,73.1014.3b1.3' +
>'pciex1000,73.1014.3b1' + 'pciex1000,73.3' + 'pciex1000,73' +
>'pciexclass,010400' + 'pciexclass,0104' + 'pci1000,73.1014.3b1.3' +
>'pci1000,73.1014.3b1' + 'pci1014,3b1' + 'pci1000,73.3' +
>'pci1000,73' + 'pciclass,010400' + 'pciclass,0104'
> name='model' type=string items=1
> value='RAID controller'
> name='power-consumption' type=int items=2
> value=0001.0001
> name='devsel-speed' type=int items=1
> value=
> name='interrupts' type=int items=1
> value=0001
> name='subsystem-vendor-id' type=int items=1
> value=1014
> name='subsystem-id' type=int items=1
> value=03b1
> name='unit-address' type=string items=1
> value='0'
> name='class-code' type=int items=1
> value=00010400
> name='revision-id' type=int items=1
> value=0003
> name='vendor-id' type=int items=1
> value=1000
> name='device-id' type=int items=1

Re: [OpenIndiana-discuss] 3737 days of uptime

[Jason Matthews]

long uptimes are not hard to achieve. i bet if i got my old sparc ipx out from 
college days and booted it the system would report about thirteen years if 
uptime. i think i last used it in the summer of 2000. 

i "shut it down" last time by putting it to sleep.  when it boots, assuming 
battery is good shape, it should have all that elapsed time as uptime. 

but you right, i didnt apply any patches. something tells me its as secure as 
the day i packed it away :)

j. 

Sent from Jasons' hand held

On Mar 20, 2013, at 4:55 AM, Sašo Kiselkov  wrote:

> On 03/20/2013 12:32 PM, Edward Ned Harvey (openindiana) wrote:
>> It would only bring a tear to my eye, because of how foolishly
>> irresponsible that is.  3737 days of uptime means 10 years of
>> never applying security patches and bugfixes.  Whenever people
>> are proud of a really long uptime, it's a sign of a bad sysadmin.
> 
> 1) Reboot is only required when applying kernel patches or big
>   clusters which affect core services (and the admin is using BEs).
> 
> 2) Not all machines are web-facing and thus don't necessarily need
>   regular security patching.
> 
> 3) If it ain't broken, don't fix it.
> 
> Not everybody can afford the luxury of periodic maintenance downtime and
> certain systems are required to be 100% available, though I will admit
> that these are few and far between (and in most cases a good hot-spare
> policy will take care of this).
> 
> --
> Saso
> 
> ___
> OpenIndiana-discuss mailing list
> OpenIndiana-discuss@openindiana.org
> http://openindiana.org/mailman/listinfo/openindiana-discuss

___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] Multiple devices appearing as one behind SAS expander

[Jason Matthews]

neat problem. 
generally the wwn is stamped on the drive. are they all marked with the same 
wwn?

who makes the drives? ocz? sandisk?

Sent from Jasons' hand held

On Mar 21, 2013, at 6:13 AM, "Schweiss, Chip"  wrote:

> I have four 480GB SSDs I'm trying to re-add to my pool as L2ARC.   They
> were all connected internally on the server, but now I have a dedicated SAS
> JBOD for the 2 1/2 SSDs in the system.   The JBOD is a SuperMicro 2U with
> LSI SAS2X36 expander.
> 
> The SSDs are are all showing up as a single device: c11t1200d0s0
> 
> They are incorrectly identified as the same device by the multipathadm:
> 
> mutipathadm list lu:
> 
>/dev/rdsk/c11t1200d0s2
>Total Path Count: 4
>Operational Path Count: 4
> 
> Not sure where napp-it gets its information for it's controller view but
> this is how it reports them:
> 
> c14::w5003048001d91b4e,0   connected   configured   unknown   Client
> Device: /dev/dsk/c11t1200d0s0(sd75) disk-path n /devices/pci@0
> ,0/pci8086,340e@7/pci1000,30d0@0/iport@f000:scsi::w5003048001d91b4e,0
> c14::w5003048001d91b4f,0   connected   configured   unknown   Client
> Device: /dev/dsk/c11t1200d0s0(sd75) disk-path n /devices/pci@0
> ,0/pci8086,340e@7/pci1000,30d0@0/iport@f000:scsi::w5003048001d91b4f,0
> c14::w5003048001d91b5a,0   connected   configured   unknown   Client
> Device: /dev/dsk/c11t1200d0s0(sd75) disk-path n /devices/pci@0
> ,0/pci8086,340e@7/pci1000,30d0@0/iport@f000:scsi::w5003048001d91b5a,0
> c14::w5003048001d91b5b,0   connected   configured   unknown   Client
> Device: /dev/dsk/c11t1200d0s0(sd75) disk-path n /devices/pci@0
> ,0/pci8086,340e@7/pci1000,30d0@0/iport@f000:scsi::w5003048001d91b5b,0
> 
> How do I break this and get OI to see these as 4 independent devices?
> 
> Thank you,
> -Chip
> ___
> OpenIndiana-discuss mailing list
> OpenIndiana-discuss@openindiana.org
> http://openindiana.org/mailman/listinfo/openindiana-discuss

___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] Anyone using OpenIndiana in production?

[Jason Matthews]

> Would anyone be so kind as to share their experiences using 
> OpenIndiana in production?

The platform is generally stable but there are some gotchas in my
application:

- I have lost entire zones when migrating from 151a1 to 151a7.
- There is no unencumbered bmc driver
- Support for disk fault light identification in disk shelves is limited to
a handful of devices so you are responsible for finding a work around. Work
arounds are available.
- USB disk support is shaky on active drives - solaris 11 probably isn't any
better
- active web servers need work arounds to avoid DCE cache from consuming all
system memory
- I have never gotten VRRP on zones with exclusive interfaces to work
despite some suggesions that it could work with some sort of work around.
VRRP on the host OS works as expected.
- some bug can make systems/zones slow to login via ssh. I only experience
this on zones. A few zones have the issue all the time. Others are
intermittent. All the zones are configured the same in terms of dns. I use
zlogin in a pinch.  https://www.illumos.org/issues/1983
- device driver support is limited. Specifying equipment can be painful for
the uninitiated.

That said, I probably use more "enterprise" features at scale than most, but
certainly not all.
- LACP, VNICs, VLAN trunking, jumbo frames
- zones, extensive use snapshots, zone migration
- NFS
- iSCSI 
- ZFS - mirrors, 3-way mirrors, raidz, L2ARC, ZIL, clones
- SAS disk enclosures with lots of disks

It all just works - at scale. The platform does more than a billion inserts
to Postgres databases per day and it runs like clock work.

My web tier process processes about half million packets per second and
aside from some initial (painful) tuning, it just works.

I don't use virtual box in my production environment. 

Does this help?









___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] How to configure OI for proper daylight savings adjustments.

[Jason Matthews]

set your time zone for arizona :-)

Sent from Jasons' hand held

On Mar 31, 2013, at 8:29 AM, "Hans J. Albertsson" 
 wrote:

> I have exactly the same setup here: My machine has been on this setup since 
> b134, and I have the same line in init and the same definition file, and NO 
> problem whatsoever.
> 
> And I'm in Stockholm. You are, however, in Gothenburg, and that is a well 
> known contrary city.
> 
> Maybe it's the sprits of Kålle and/or Ada playing havoc with your sanity?? :-)
> 
> 
> On 2013-03-31 19:47, Robin Axelsson wrote:
>> -BEGIN PGP SIGNED MESSAGE-
>> Hash: SHA1
>>  Hi,
>> my server has been quite troublesome when it comes to adjusting the time
>> for daylight savings. Last time it adjusted the system time by moving it
>> forward by no less than two hours and now it is one hour behind. It
>> never gets it right. The config file "/etc/default/init" has a
>> "TZ=Europe/Stockholm" line in it, and there is a
>> "/usr/share/lib/zoneinfo/Europe/Stockholm" file in existence.
>> 
>> I've had problems with the scheduled DST adjustments since the good old
>> OpenSolaris days and it is a true pita. Is there anyone who has got this
>> working properly?
>> 
>> Regards
>> 
>> Robin.
>> -BEGIN PGP SIGNATURE-
>> Version: GnuPG v2.0.17 (MingW32)
>> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>>  iQEcBAEBAgAGBQJRWHbBAAoJEHrMcsL/irW8BM4H/jyLiqEdcPJLyn7SbszyzU8N
>> wzIAfzYHI4ApJEKCvbzLLhRfpJNvefS1M7owUM2OQqrXHBa1U2OY3rWd4IOjg2o4
>> nM0WpSmCSPG3GRh4j1a8/qpoQPu38k8uxNXXV/k8MiQ0RkRCU+JoMu+7b1aBJczc
>> LKb7ikm0jQuD03t7biaS/44O7GKLzgQnsUu1J9+Nmbd3uwtQgrCVvnwYK9/7JWPR
>> +/JKSWHUPMPn5vIVLSaxombCiKxGQrQ2AjKLW2JtZvDrsbC7u0HT4Kr7EOEiNdJ6
>> WkqkBpgjOLRNJusmjCWLglnM81xgMvld79kj3z15zi3YsIMmT7daK4N2OYjH/Lk=
>> =w3mp
>> -END PGP SIGNATURE-
>> 
>> 
>> 
>> ___
>> OpenIndiana-discuss mailing list
>> OpenIndiana-discuss@openindiana.org
>> http://openindiana.org/mailman/listinfo/openindiana-discuss
> 
> 
> ___
> OpenIndiana-discuss mailing list
> OpenIndiana-discuss@openindiana.org
> http://openindiana.org/mailman/listinfo/openindiana-discuss

___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] hardware recommendations for OI desktop system

[jason matthews]

I don't see one one the wiki.

i am not trying to be overly caddy, but why don't you order one of each and let 
us know ;-)


I noticed something on the hcl wiki today. Ken Mays marked some systems as 
'OpenIndiana Certified' - what is the process for achieving that distinction?

j.



___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] [UNSUBSCRIBE]

[jason matthews]

On Jun 6, 2013, at 5:00 PM, Christopher Chan  
wrote:

> That's great Dave. Over here, the move has been to not Linux but to Unix in 
> the form of Mac OS X.


Same in the SF Bay area. Even the linux ditto heads are willing to run OSX.




___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

[OpenIndiana-discuss] OI DCE cache (dce_ip)

[jason matthews]

Has the DCE cache problem been fixed in OI? the smartos folks a proposed 
solution but even their work appears to have stopped.

i am still running this bit of wisdom every three minutes on 151a1 but i am 
reaching traffic levels where the cache still reaches up to three  hundred 
megabytes. this causes system performance to drag and the usefulness of this 
work around is degrading.

#!/bin/bash
/usr/sbin/ndd -set /dev/ip ip_dce_reclaim_fraction 1
{
echo '%/l/lotsfree >oldlotsfree'
echo 'lotsfree/Z ((%/l/freemem)*4)'
sleep 2
echo 'lotsfree/Z https://gist.github.com/postwait/2597292/raw/d8fd94aaac6da21830ba3249ad438bd33cf53b97/gistfile1.txt
"OS-1082 dce_reclaim_thread stops making forward progress"
http://wiki.smartos.org/pages/viewpage.action?pageId=755416&navigatingVersions=true
 OS-1056 ip_dce_reclaim_stack() cannot be made perfectly safe

both of these entries do no instill confidence.

thanks,
j.





___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] OI 151_a7 crashing when trying to import ZFS pool

[jason matthews]

i would start with smartos but any modern live boot image will due.

j.


On Jul 8, 2013, at 11:36 AM, CJ Keist  wrote:

> Thank you.  It is ver 28 ZFS.  this is a supermicro chasis with 36 interal 
> drives so i cannot move another system to mount the zfs pools. I could try 
> install OpenSolaris or illuminos?  Is there a preference.


___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] Distro identify

[jason matthews]

> What's the best way to identify that a given system is running
> OpenIndiana? uname -v ?

You have lots of options.

uname -v works
jason@caprica:/data/home/jason$ uname -v
oi_148

jason@caprica:/data/home/jason$ cat /etc/release 
  OpenIndiana Development oi_148 X86
Copyright 2010 Oracle and/or its affiliates. All rights reserved.
Use is subject to license terms.
   Assembled 29 November 2010


___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

[OpenIndiana-discuss] how does one disable zil in OI?

[jason matthews]

The evil tuning guide indicates the ZIL can be turned on/off with:
echo zil_disable/W0t1 | mdb -kw
echo zil_disable/W0t0 | mdb -kw

this does not appear to be valid in OI. Is there a new procedure to disable
ZIL?


Thanks,
j.
___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

[OpenIndiana-discuss] jbod disk shelves with fmd fault led support

[jason matthews]

i am looking for oi friendly disk chassis that support managing the disk fault 
lights from fmd. 

i need approximately 40 units and while i have found some lsi 620js i havent 
found enough for my needs. 

excluding over priced sun gear, which shelves meet these requirements and work 
well?

i know dataonstorage is trying to certify their dns-1640 model but are there 
more mature products available?

please advise,
j.  


Sent from Jasons' hand held
___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] zpool create hangs

[jason matthews]

i have used osol134b with the adaptec supplied s10 driver as osol didnt have a 
working driver in the distribution

i also used oi148b with the bundled driver with great success. 

you could try the adaptec driver, it wouldnt hurt, but i dont believe it will 
help. seems like your problem is somewhere else. 

to be clear do you have the 5805 or 5805z? the last time i looked in late 2009 
the 5805z wasnt on the hcl

Sent from Jasons' hand held

On Jul 21, 2011, at 6:01 AM, Gabriele Bulfon  wrote:

> ...sorry, it's 5805, not 3805
> --
> Da: Gabriele Bulfon
> A: Discussion list for OpenIndiana
> Data: 21 luglio 2011 14.41.24 CEST
> Oggetto: Re: [OpenIndiana-discuss] zpool create hangs
> Looks like the controller is not responding.not even format responds once 
> I select one of the disks...
> Any problem with Adaptec 3805? I found a driver at adaptec site but it's for 
> Solaris 10...
> Should I try to replace the original one with this?
> --
> Da: Gabriele Bulfon
> A: Discussion list for OpenIndiana
> Data: 21 luglio 2011 11.11.33 CEST
> Oggetto: [OpenIndiana-discuss] zpool create hangs
> Hi, I was experimenting with 5 sas disks.
> Created a raidz2 pool with :
> zpool create data raidz2 c8t0d0 c8t1d0 c8t2d0 c8t3d0 c8t4d0
> worked.
> Then destroyed it, and tried again a raidz1:
> zpool create data raidz1 c8t0d0 c8t1d0 c8t2d0 c8t3d0 c8t4d0
> the command did not return for 10 minutes, disks all light up, no work at all.
> I killed the zpool create process, but it did not kill.
> After almost 20 minutes the process was killed, and no pool was created.
> Zpool status was fine, with just the original pools.
> I thought that the problem was raidz1 with 5 disks, so I did the raidz2 again:
> zpool create data raidz2 c8t0d0 c8t1d0 c8t2d0 c8t3d0 c8t4d0
> and it's still there not returning.
> What's happening??
> ___
> OpenIndiana-discuss mailing list
> OpenIndiana-discuss@openindiana.org
> http://openindiana.org/mailman/listinfo/openindiana-discuss
> ___
> OpenIndiana-discuss mailing list
> OpenIndiana-discuss@openindiana.org
> http://openindiana.org/mailman/listinfo/openindiana-discuss
> ___
> OpenIndiana-discuss mailing list
> OpenIndiana-discuss@openindiana.org
> http://openindiana.org/mailman/listinfo/openindiana-discuss

___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] jbod disk shelves with fmd fault led support

[jason matthews]

yes my bad. i am looking for SAS chassis. 

what is the process for mapping drives?

Sent from Jasons' hand held

On Jul 20, 2011, at 5:58 PM, Gary Driggs  wrote:

> You didn't say whether you're looking for FC or SAS but the Promise series 
> offers both; 
> http://www.promise.com/storage/raid_category.aspx?region=en-global&m=17&sub_m=sub_m_2&rsn=1&statistic=VTrak
> 
> I've built one out with LSI SAS cards & Nexentastor but IIRC it took some 
> work to get the drives mapped properly for identification purposes.
> 
> -Gary
> ___
> OpenIndiana-discuss mailing list
> OpenIndiana-discuss@openindiana.org
> http://openindiana.org/mailman/listinfo/openindiana-discuss

___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] ZFS with 5 SAS best config

[jason matthews]

> Hi, I'm configuring a machine with 5 SAS disks dedicated to data.
> I was thinking to make a raidz2 pool on these 5 disks.
> What is the best option?
> What if I want to do it 4 disks + 1 for spare?


It is not clear to me what your requirements are.

if you need speed and redundancy, use mirrors.
If you need maximum storage use raidz
If you need more redundancy while maximizing storage than use raidz2
If you want more redundancy than raidz2 use a raidz3
If you want maximum redundancy use a 5-way mirror on five controllers ;-)

Your choice should be driven by performance and storage requirements. My
personal belief is unless the data is cold/archived/etc, the only proper
choice is mirrors.

j.
___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] jbod disk shelves with fmd fault ledsupport

[jason matthews]

At the risk of beating a dead horse...

Has anyone given the newisys NDS shelves a whirl? The nds-2241 looks awful
lot like the 24 bay 2.5 inch shelves netapp pushes. 

It my hope that netapp drives it fault lights in the same fashion to how
solaris does it. Is there anyone on the list with experience using netapp or
newisys shelves? Can fmd drive the fault lights?


___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

[OpenIndiana-discuss] nexenta dsm

[jason matthews]

So our friends at DataOn offer a software package, marketed towards Nexenta,
named DSM. It reportedly has a command line toolto identify and flip on
fault lights in SES equipped chassis. They report that the CLI will work
with OI.

Does anyone have an experience with it, either good or bad?


Thanks,
j.
___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] nexenta dsm

[jason matthews]

Rocky,

What is the name of that package?

Thanks,
j.

-Original Message-
From: Rocky Shek [mailto:roc...@dataonstorage.com] 
Sent: Monday, July 25, 2011 11:10 PM
To: 'Discussion list for OpenIndiana'
Subject: Re: [OpenIndiana-discuss] nexenta dsm

Gary,

The package you download is only for Nexenta platform.

We do have other package for openindiana/solaris 10 users.

It help users to identify fail disk and monitor FRU of the JBOD

Rocky 

-Original Message-
From: Gary [mailto:gdri...@gmail.com] 
Sent: Monday, July 25, 2011 5:33 PM
To: openindiana-discuss@openindiana.org
Subject: Re: [OpenIndiana-discuss] nexenta dsm

On Mon, Jul 25, 2011, jason matthews wrote:

>
> Does anyone have an experience with it, either good or bad?
>

Curious -- I only see the debian package for Nexentastor or NCP;
http://apt.dataonstorage.com/dists/hardy-stable/main/binary-solaris-i386/bas
e

I've extracted the binaries  but it's unclear whether this will work outside
of Nexentastor's plug-in architecture. Also, the documentation and web site
suggests that it's only for their hardware. That's not surprising but if
they're also letting others use the binaries on mainstream (or forked)
Solaris distributions I don't see how they've provided it in a way that
makes that possible -- or at the very least uncomplicated.

-Gary
___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss


___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss
___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

[OpenIndiana-discuss] BMC driver

[jason matthews]

Does the old trick of copying the BMC driver from NV134 or Solaris Express
151 work on OI148?

I have tried to load both versions of that driver they both bombed out. Has
anyone tried with OI151? Or am I just beating my head against the wall...
Was there some sort of change in the kernel that breaks this trick?


Thanks,
j.
___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] BAD WD drives - defective by design?

[Jason Matthews]

Would tuning the kernel to deal with the super slow time-outs of consumer
grade drives help? It is not going to make your consumer drive an enterprise
product, or keep your performance high, but it might keep FMD from failing
out your entire pool and/or corrupting the pool.

These are guestimates. Your mileage may vary. 

First, let's give the piece-of-frugle SATA drive a 45 second time. If that
doesn’t work, double it until you find a time out that works and then start
lowering it. It really depends on how badly WD is trying to screw the pouch
on this. It would be helpful for everyone to report your results so others
know how to configure their systems for WD (if they are unable to avoid
buying WD in the first place).

In /etc/system:
set sd:sd_io_time=45

Now that we are hitting the timeout, let's give it three retries. Yes, that
is 135 seconds to time out -- outch -- buy hey, you saved a few bucks when
you bought sata right? And it works most of the time?

in /kernel/drv/sd.conf
sd-config-list "   ", retries-timeout:3

This string where you define the disk is a fixed length field and must be
padded by spaces. To properly format this, it appears that 8 characters are
for the manufacturer, 16 characters for the product id, separated by a
single space. I think iostat -En will reveal the proper strings to put in
sd.conf

Let me know if this helps... Or if you just dismiss it outright ;-)

j.







-Original Message-
From: Roy Sigurd Karlsbakk [mailto:r...@karlsbakk.net] 
Sent: Monday, August 29, 2011 2:24 PM
To: Discussion list for OpenIndiana
Subject: Re: [OpenIndiana-discuss] BAD WD drives - defective by design?

file systems don't have sectors, they have blocks. and zfs normally uses
512b-128k blocks, which is fine. this issue won't be fixed by reducing the
recordsize in zfs, it's a hardware/driver thing

- Original Message -
> I think Rich was suggesting using 4k sectors on the file system, not
> asking
> what the drives sector size was.
> 
> On Mon, Aug 29, 2011 at 2:15 PM, Roy Sigurd Karlsbakk
> wrote:
> 
> > All drives have 512b sector sizes, WD FASS (blacks) and WD EADS
> > (greens)
> > both use plain old 512 sectors.
> >
> > - Original Message -
> > > The drives are attached to a backplane?
> > >
> > > Try using 4k sector sizes and see if that improves it - I've seen
> > > and
> > > been part of a number of discussions which involved this - and
> > > you, I
> > > think, actually.
> > >
> > > - Rich
> > >
> > > On Mon, Aug 29, 2011 at 5:07 PM, Roy Sigurd Karlsbakk
> > >  wrote:
> > > > Hi all
> > > >
> > > > It seems recent WD drives that aren't "Raid edition" can cause
> > > > rather a lot of problems on RAID systems. We have a few machines
> > > > with LSI controllers (6801/6081/9201) and we're seeing massive
> > > > errors occuring. The usual pattern is a drive failing or even a
> > > > resilver/scrub starting and then, suddenly, most drives on the
> > > > whole
> > > > backplane report errors. These are usually No Device (as
> > > > reported by
> > > > iostat -En), but the result is that we may see data corruption
> > > > at
> > > > the end. We also have a system setup with Hitachi Deskstars,
> > > > which
> > > > has been running for almost a year without issues. One system
> > > > with a
> > > > mixture of WD Blacks and greens showed the same errors as
> > > > described,
> > > > but has been working well after the WD drives were replaced by
> > > > deskstars.
> > > >
> > > > Now, it seems WD has changed their firmware to inhibit people
> > > > from
> > > > using them for other things than toys (read: PCs etc). Since
> > > > we've
> > > > seen this issue on different controllers and different drives,
> > > > and
> > > > can't reproduce it with Hitachi Deskstars, I would guess the
> > > > firmware "upgrade" from WD is the issue.
> > > >
> > > > Would it be possible to fix this in ZFS somehow? The drives seem
> > > > to
> > > > work well except for those "No Device" errors
> > > >
> > > > Vennlige hilsener / Best regards
> > > >
> > > > roy
> > > > --
> > > > Roy Sigurd Karlsbakk
> > > > (+47) 97542685
> > > > r...@karlsbakk.net
> > > > http://blogg.karlsbakk.net/
> > > > --
> > > > I all pedagogikk er det essensielt at pensum presenteres
> > > > intelligibelt. Det er et elementært imperativ for alle pedagoger
> > > > å
> > > > unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I
> > > > de
> > > > fleste tilfeller eksisterer adekvate og relevante synonymer på
> > > > norsk.
> > > >
> > > > ___
> > > > OpenIndiana-discuss mailing list
> > > > OpenIndiana-discuss@openindiana.org
> > > > http://openindiana.org/mailman/listinfo/openindiana-discuss
> > > >
> > >
> > > ___
> > > OpenIndiana-discuss mailing list
> > > OpenIndiana-discuss@openindiana.org
> > > http://openindiana.org/mailman/listinfo/openindiana-discuss
> >
> > --
> > Vennlige hilsener / Best regards

Re: [OpenIndiana-discuss] BAD WD drives - defective by design?

[Jason Matthews]

TLER was removed from the consumer drives in one of the rounds of firmware
upgrades. That tunable is now only available in the enterprise versions of
the disk -- at least that is my understanding.

If you have fix it in the OS, it is going to be more expensive. See my last
email on this.


j.

-Original Message-
From: John McEntee [mailto:jmcen...@stirling-dynamics.com] 
Sent: Monday, August 29, 2011 2:35 PM
To: openindiana-discuss@openindiana.org
Subject: Re: [OpenIndiana-discuss] BAD WD drives - defective by design?

the WD drives have a what WD call TLER, that is a setting that defines 
how long the drive trys to read a bad sector before giving up at 
returning an error. The raid edition does this almost instantly and the 
RAID hardware / ZFS can then decide what to do. The non RAID edition 
retries for a long time before returning and error which result in RAID 
hardware and ZFS thinking the drive is dead an not responding at all.

Search google for WDTLER and download the windows program and see if it 
allows your drive to be changed. I believe WD has stopped it working on 
the latest cheap non RAID drives. For this resaon I by Hitachi, soon the 
only choice will be the WD RAID or Seagate. I am trying to squeeze in 
one last Hitachi upgrade to 3TB in at the moment.

John


On 29/08/2011 22:22, Rennie Allen wrote:
> I think Rich was suggesting using 4k sectors on the file system, not
asking
> what the drives sector size was.
>
> On Mon, Aug 29, 2011 at 2:15 PM, Roy Sigurd
Karlsbakkwrote:
>
>> All drives have 512b sector sizes, WD FASS (blacks) and WD EADS (greens)
>> both use plain old 512 sectors.
>>
>> - Original Message -
>>> The drives are attached to a backplane?
>>>
>>> Try using 4k sector sizes and see if that improves it - I've seen and
>>> been part of a number of discussions which involved this - and you, I
>>> think, actually.
>>>
>>> - Rich
>>>
>>> On Mon, Aug 29, 2011 at 5:07 PM, Roy Sigurd Karlsbakk
>>>   wrote:
 Hi all

 It seems recent WD drives that aren't "Raid edition" can cause
 rather a lot of problems on RAID systems. We have a few machines
 with LSI controllers (6801/6081/9201) and we're seeing massive
 errors occuring. The usual pattern is a drive failing or even a
 resilver/scrub starting and then, suddenly, most drives on the whole
 backplane report errors. These are usually No Device (as reported by
 iostat -En), but the result is that we may see data corruption at
 the end. We also have a system setup with Hitachi Deskstars, which
 has been running for almost a year without issues. One system with a
 mixture of WD Blacks and greens showed the same errors as described,
 but has been working well after the WD drives were replaced by
 deskstars.

 Now, it seems WD has changed their firmware to inhibit people from
 using them for other things than toys (read: PCs etc). Since we've
 seen this issue on different controllers and different drives, and
 can't reproduce it with Hitachi Deskstars, I would guess the
 firmware "upgrade" from WD is the issue.

 Would it be possible to fix this in ZFS somehow? The drives seem to
 work well except for those "No Device" errors

 Vennlige hilsener / Best regards

 roy
 --
 Roy Sigurd Karlsbakk
 (+47) 97542685
 r...@karlsbakk.net
 http://blogg.karlsbakk.net/
 --
 I all pedagogikk er det essensielt at pensum presenteres
 intelligibelt. Det er et elementært imperativ for alle pedagoger å
 unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de
 fleste tilfeller eksisterer adekvate og relevante synonymer på
 norsk.

 ___
 OpenIndiana-discuss mailing list
 OpenIndiana-discuss@openindiana.org
 http://openindiana.org/mailman/listinfo/openindiana-discuss

>>> ___
>>> OpenIndiana-discuss mailing list
>>> OpenIndiana-discuss@openindiana.org
>>> http://openindiana.org/mailman/listinfo/openindiana-discuss
>> --
>> Vennlige hilsener / Best regards
>>
>> roy
>> --
>> Roy Sigurd Karlsbakk
>> (+47) 97542685
>> r...@karlsbakk.net
>> http://blogg.karlsbakk.net/
>> --
>> I all pedagogikk er det essensielt at pensum presenteres intelligibelt.
Det
>> er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse
av
>> idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer
adekvate
>> og relevante synonymer på norsk.
>>
>> ___
>> OpenIndiana-discuss mailing list
>> OpenIndiana-discuss@openindiana.org
>> http://openindiana.org/mailman/listinfo/openindiana-discuss
>>
>
>


___

The contents of this e-mail and any attachment(s) are strictly confidential
and are solely for the person(s) at the e-mail address(es) above. If you are
not an addr

Re: [OpenIndiana-discuss] server hangs

[Jason Matthews]

perhaps  checking fmd would be a good start but an overloaded ps is possible. 

fmdump -eV

for your storage utilization some handy commands to add to your arsenal are:

zpool list
zfs get -r used 

cheers

Sent from Jasons' hand held

On Aug 31, 2011, at 6:43 PM, Daniel Kjar  wrote:

> Careful... are you overtaxing your power supply?  My 148 system was behaving 
> like that when I put too many drives in an ultra 20.
> 
> On 8/31/2011 7:48 PM, Roman Naumenko wrote:
>> Hi,
>> 
>> I have SunOS 5.11 oi_148 installed on my storage server with 8 disks in 
>> raidz2 pool.
>> It hangs about once in a week and I had to restart it.
>> Can you help me troubleshoot it?
>> 
>> It has some zfs volumes shared over nfs and afpd. (afpd is unfortunately a 
>> development version to satisfy OSX Lion).
>> 
>> roks@data:~$ afpd -V
>> afpd 2.2.0 - Apple Filing Protocol (AFP) daemon of Netatalk
>> 
>> afpd has been compiled with support for these features:
>> 
>> AFP3.x support: Yes
>> TCP/IP Support: Yes
>> DDP(AppleTalk) Support: No
>> CNID backends: dbd last tdb
>> SLP support: No
>> Zeroconf support: Yes
>> TCP wrappers support: Yes
>> Quota support: Yes
>> Admin group support: Yes
>> Valid shell checks: Yes
>> cracklib support: No
>> Dropbox kludge: No
>> Force volume uid/gid: No
>> ACL support: Yes
>> EA support: ad | sys
>> LDAP support: Yes
>> 
>> It also has time-slider enabled, which is pretty buggy peace of hmmm 
>> software, but it shouldn't cause server to crash or hang.
>> 
>> So the problems start with nfs and/or afpd timeouts on clients, but I still 
>> can ssh to the server. Can't read any files or logs though.
>> Then network service disappears in a minute or few minutes, console becomes 
>> frozen and I have to do hard restart at that point.
>> 
>> Where should I look to understand what causing this?
>> Since I can't reproduce the problem, I'd like to get prepared when it 
>> happens next time.
>> I couldn't find anything unusual in the logs after restart.
>> 
>> time-slider complains for some reason about space on rpool
>> Aug 31 19:41:36 data time-sliderd: [ID 702911 daemon.notice] No more hourly 
>> snapshots left
>> Aug 31 19:41:36 data time-sliderd: [ID 702911 daemon.warning] rpool exceeded 
>> 80% capacity. Hourly and daily automatic snapshots were destroyed
>> 
>> Where does it see 80%?
>> 
>> $ df -h
>> 
>> FilesystemSize  Used Avail Use% Mounted on
>> rpool/ROOT/solaris5.5G  3.0G  2.6G  54% /
>> swap  1.4G  396K  1.4G   1% /etc/svc/volatile
>> /usr/lib/libc/libc_hwcap1.so.1 5.5G  3.0G  2.6G  54% /lib/libc.so.1
>> swap  1.4G  8.0K  1.4G   1% /tmp
>> swap  1.4G   52K  1.4G   1% /var/run
>> rpool/export  2.6G   32K  2.6G   1% /export
>> rpool/export/home 2.6G   33K  2.6G   1% /export/home
>> rpool/export/home/usr1 2.6G   38K  2.6G   1% /export/home/usr1
>> rpool/export/home/usr2 3.0G  385M  2.6G  13% /export/home/usr2
>> rpool 2.6G   48K  2.6G   1% /rpool
>> 
>> 
>> --Roman
>> 
>> ___
>> OpenIndiana-discuss mailing list
>> OpenIndiana-discuss@openindiana.org
>> http://openindiana.org/mailman/listinfo/openindiana-discuss
> 
> -- 
> Dr. Daniel Kjar
> Assistant Professor of Biology
> Division of Mathematics and Natural Sciences
> Elmira College
> 1 Park Place
> Elmira, NY 14901
> 607-735-1826
> http://faculty.elmira.edu/dkjar
> 
> "...humans send their young men to war; ants send their old ladies"
>-E. O. Wilson
> 
> 
> ___
> OpenIndiana-discuss mailing list
> OpenIndiana-discuss@openindiana.org
> http://openindiana.org/mailman/listinfo/openindiana-discuss

___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] server hangs

[Jason Matthews]

what about hw event logs? if you have power flucuations it might show ip there. 

you can probably pull those out from your service processor or boot to bios and 
read them there. 

Sent from Jasons' hand held

On Sep 1, 2011, at 8:37 AM, Roman Naumenko  wrote:

> Costly troubleshooting you had. 
> All right then, I will wait for the next failure to look through it once 
> again and maybe swap psu if nothing again found. 
> 
> --Roman N 
> 
> - Original Message -
> 
>> I burned through about 3 disks before I figured it out. Nothing in
>> the
>> logs made me think this but the eventual failure of the disks alerted
>> me
>> that something hardwarish was happening.
> 
>> On 08/31/11 11:01 PM, Roman Naumenko wrote:
>>> Well, might be the reason. 8 drivers is certainly limit too much
>>> for a
>>> stock psu. But there should be some traces, no?
>>> How did you figure out the reason for errors on your system?
>>> 
>>> --Roman
>>> 
>>> Daniel Kjar said the following, on 31-08-11 9:43 PM:
 Careful... are you overtaxing your power supply? My 148 system was
 behaving like that when I put too many drives in an ultra 20.
 
 On 8/31/2011 7:48 PM, Roman Naumenko wrote:
> Hi,
> 
> I have SunOS 5.11 oi_148 installed on my storage server with 8
> disks
> in raidz2 pool.
> It hangs about once in a week and I had to restart it.
> Can you help me troubleshoot it?
> 
> It has some zfs volumes shared over nfs and afpd. (afpd is
> unfortunately a development version to satisfy OSX Lion).
> 
> roks@data:~$ afpd -V
> afpd 2.2.0 - Apple Filing Protocol (AFP) daemon of Netatalk
> 
> afpd has been compiled with support for these features:
> 
> AFP3.x support: Yes
> TCP/IP Support: Yes
> DDP(AppleTalk) Support: No
> CNID backends: dbd last tdb
> SLP support: No
> Zeroconf support: Yes
> TCP wrappers support: Yes
> Quota support: Yes
> Admin group support: Yes
> Valid shell checks: Yes
> cracklib support: No
> Dropbox kludge: No
> Force volume uid/gid: No
> ACL support: Yes
> EA support: ad | sys
> LDAP support: Yes
> 
> It also has time-slider enabled, which is pretty buggy peace of
> hmmm
> software, but it shouldn't cause server to crash or hang.
> 
> So the problems start with nfs and/or afpd timeouts on clients,
> but
> I still can ssh to the server. Can't read any files or logs
> though.
> Then network service disappears in a minute or few minutes,
> console
> becomes frozen and I have to do hard restart at that point.
> 
> Where should I look to understand what causing this?
> Since I can't reproduce the problem, I'd like to get prepared
> when
> it happens next time.
> I couldn't find anything unusual in the logs after restart.
> 
> time-slider complains for some reason about space on rpool
> Aug 31 19:41:36 data time-sliderd: [ID 702911 daemon.notice] No
> more
> hourly snapshots left
> Aug 31 19:41:36 data time-sliderd: [ID 702911 daemon.warning]
> rpool
> exceeded 80% capacity. Hourly and daily automatic snapshots were
> destroyed
> 
> Where does it see 80%?
> 
> $ df -h
> 
> Filesystem Size Used Avail Use% Mounted on
> rpool/ROOT/solaris 5.5G 3.0G 2.6G 54% /
> swap 1.4G 396K 1.4G 1% /etc/svc/volatile
> /usr/lib/libc/libc_hwcap1.so.1 5.5G 3.0G 2.6G 54% /lib/libc.so.1
> swap 1.4G 8.0K 1.4G 1% /tmp
> swap 1.4G 52K 1.4G 1% /var/run
> rpool/export 2.6G 32K 2.6G 1% /export
> rpool/export/home 2.6G 33K 2.6G 1% /export/home
> rpool/export/home/usr1 2.6G 38K 2.6G 1% /export/home/usr1
> rpool/export/home/usr2 3.0G 385M 2.6G 13% /export/home/usr2
> rpool 2.6G 48K 2.6G 1% /rpool
> 
> 
> --Roman
> 
> ___
> OpenIndiana-discuss mailing list
> OpenIndiana-discuss@openindiana.org
> http://openindiana.org/mailman/listinfo/openindiana-discuss
 
>>> 
>>> ___
>>> OpenIndiana-discuss mailing list
>>> OpenIndiana-discuss@openindiana.org
>>> http://openindiana.org/mailman/listinfo/openindiana-discuss
> 
>> --
>> Dr. Daniel Kjar
>> Assistant Professor of Biology
>> Division of Mathematics and Natural Sciences
>> Elmira College
>> 1 Park Place
>> Elmira, NY 14901
>> 607-735-1826
>> http://faculty.elmira.edu/dkjar
> 
>> "...humans send their young men to war; ants send their old ladies"
>> -E. O. Wilson
>> ___
>> OpenIndiana-discuss mailing list
>> OpenIndiana-discuss@openindiana.org
>> http://openindiana.org/mailman/listinfo/openindiana-discuss
> ___
> OpenIndiana-discuss mailing list
> OpenIndiana-discuss@openindiana.org
> http://openindiana.org/mailman/listinfo/openindiana-discuss

___

Re: [OpenIndiana-discuss] SATA device errors, possibly due to IRQ conflict

[Jason Matthews]

since your bios doesnt reliably detect drives on reboot i would start by 
looking for firmware upgrades. 

smc likes to say dont upgrade your firmware unless you have a problem, i be you 
meet that requirement. 

j

Sent from Jasons' hand held

On Sep 29, 2011, at 2:31 AM, Daniel  wrote:

> Hi,
> 
> I've got a server running OpenIndiana 148 on a Supermicro *X8ST3-F* that has
> been working perfectly for months right up until I added some more storage.
> 
> The board has 6 * SATA ports and 8 * SAS ports. Previously all the drives in
> my storage pool were attached to the 8 SAS ports and only my rpool drive was
> using one of the SATA ports.
> 
> Now that I have added another 4 drives I've had to connect them to the SATA
> ports - this is when the system started to become unstable.
> 
> I have had periods of very heavy usage that have cause no problems
> whatsoever (for example, I copied 4 TB of data on to the pool, most of which
> would have had to go on the new drives then did several scrubs over the next
> few days). The system seems perfectly happy to sustain a 350mb+ read or
> write (or a bit of both) for hours on end with no errors at all. Then other
> times, typically overnight or early morning when it's just ticking over with
> < 500k read/write, it will fall apart.
> 
> There are three kinds of failure I'm experiencing, seemingly randomly:
> 
> 1. Errors about failed read/write on 2 or 4 SATA drives in /var/adm/messages
> and system io hung - system has to have the power cut to recover - ssh won't
> connect, can't get past the username prompt on the terminal. No ZFS errors
> reported
> 2. Errors about failed read/write, system io NOT hung, ZFS reporting faulted
> drives (2 or 4) and hundreds of thousands of errors. In this scenario, the
> machine can be rebooted cleanly BUT the failed drives don't get detected by
> BIOS. Usually a full power down, wait 30 seconds, power back up will allow
> the drives to be detected again. When it powers back up ZFS will report lots
> of errors but sort itself out after a resilver - I haven't actually had any
> perminent data loss yet, zfs has always recovered.
> 3. No errors at all in either /var/adm/messages or zpool status but hung io.
> 
> 
> I've swaped the drive connections around to prove it isn't the new disks
> that are at fault and this has confirmed that it's whichever devices are
> connected to the SATA controller that are having the problem.
> 
> When I rebooted the machine after the latest failure I checked the
> /var/adm/messages and there are thousands (9995 in total but that may be
> from several reboots) messages identical to the following:
> 
> "[ID 954099 kern.info] NOTICE: IRQ19 is being shared by drivers with
> different interrupt levels."
> 
> In case it's useful:
> 
> cs2dsb@chronos:~$ echo ::interrupts -d | pfexec mdb -k
> IRQ  Vect IPL BusTrg Type   CPU Share APIC/INT# Driver Name(s)
> 90x80 9   PCILvl Fixed  1   1 0x0/0x9   acpi_wrapper_isr
> 11   0xd1 14  PCILvl Fixed  2   1 0x0/0xb   hpet_isr
> 16   0x84 9   PCILvl Fixed  7   1 0x0/0x10  uhci#0
> 18   0x82 9   PCILvl Fixed  5   2 0x0/0x12  uhci#5, ehci#0
> 19   0x86 9   PCILvl Fixed  3   6 0x0/0x13  uhci#4, uhci#2,
> pci-ide#0,
> pci-ide#1, pci-ide#1, pci-ide#0
> 21   0x85 9   PCILvl Fixed  0   1 0x0/0x15  uhci#1
> 23   0x83 9   PCILvl Fixed  6   2 0x0/0x17  uhci#3, ehci#1
> 24   0x81 7   PCIEdg MSI4   1 - pcieb#4
> 25   0x60 6   PCIEdg MSI1   1 - e1000g#0
> 26   0x61 6   PCIEdg MSI2   1 - e1000g#1
> 27   0x40 5   PCIEdg MSI3   1 - mpt#0
> 32   0x20 2  Edg IPIall 1 - cmi_cmci_trap
> 160  0xa0 0  Edg IPIall 0 - poke_cpu
> 208  0xd0 14 Edg IPIall 1 - kcpc_hw_overflow_intr
> 209  0xd3 14 Edg IPIall 1 - cbe_fire
> 210  0xd4 14 Edg IPIall 1 - cbe_fire
> 240  0xe0 15 Edg IPIall 1 - xc_serv
> 241  0xe1 15 Edg IPIall 1 - apic_error_intr
> 
> cs2dsb@chronos:~$ echo ::interrupts | pfexec mdb -k
> IRQ  Vect IPL BusTrg Type   CPU Share APIC/INT# ISR(s)
> 90x80 9   PCILvl Fixed  1   1 0x0/0x9   acpi_wrapper_isr
> 11   0xd1 14  PCILvl Fixed  2   1 0x0/0xb   hpet_isr
> 16   0x84 9   PCILvl Fixed  7   1 0x0/0x10  uhci_intr
> 18   0x82 9   PCILvl Fixed  5   2 0x0/0x12  uhci_intr, ehci_intr
> 19   0x86 9   PCILvl Fixed  3   6 0x0/0x13  uhci_intr, uhci_intr,
> ata_intr, ata_intr, ata_intr, ata_intr
> 21   0x85 9   PCILvl Fixed  0   1 0x0/0x15  uhci_intr
> 23   0x83 9   PCILvl Fixed  6   2 0x0/0x17  uhci_intr, ehci_intr
> 24   0x81 7   PCIEdg MSI4   1 - pcieb_intr_handler
> 25   0x60 6   PCIEdg MSI1   1 - e1000g_intr_pciexpress
> 26   0x61 6   PCIEdg MSI2   1 - e1000g_intr_pciexp

Re: [OpenIndiana-discuss] Best way to map SAS drives to physicalpositions

[Jason Matthews]

I use the LSI sas2ircu util and fmtopo to identify and flip on the
'identify' light on my disk chassis.

I wrote a script, it is very rough and would require some tuning for your
config, but it might solve your problem if your hardware is compatible.

Compatible means, LSI controller and disk shelf that is SES2 compliant,
supports page code 0xa,  additional element status and sets the element
index present bit. If your gear does that, then I have a script that could
help you.

If you meet these restrictions, I will publish it some where along with the
sas2ircu binary that I lifted from SMC.

j.

-Original Message-
From: Rich [mailto:rerc...@acm.jhu.edu] 
Sent: Monday, October 03, 2011 9:57 AM
To: Discussion list for OpenIndiana
Subject: Re: [OpenIndiana-discuss] Best way to map SAS drives to
physicalpositions

May I ask what kind of SAS HBA you're using?

- Rich

On Mon, Oct 3, 2011 at 10:25 AM, James Carlson 
wrote:
> Jeppe Toustrup wrote:
>> On Mon, Oct 3, 2011 at 16:00, James Carlson 
wrote:
>>> Have you tried "cfgadm -lv"?
>>
>> Yes, "cfgadm -lv" does not list the drives connected through SAS, but
>> "cfgadm -lav" does. The output does however not make me much wiser
>> than the output I can get from path_to_inst (only lines related to SAS
>> drives shown):
>
> OK.  It sounds like what you're looking for is something more like the
> enclosure lists that luxadm can provide with FC boxes.  I don't think
> that exists, though, with SAS.
>
> --
> James Carlson         42.703N 71.076W         
>
> ___
> OpenIndiana-discuss mailing list
> OpenIndiana-discuss@openindiana.org
> http://openindiana.org/mailman/listinfo/openindiana-discuss
>

___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss
___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] Best way to map SAS drivestophysicalpositions

[Jason Matthews]

What did you stumble upon? Maybe its better than mine? ;-)

-Original Message-
From: Jeppe Toustrup [mailto:openindi...@tenzer.dk] 
Sent: Monday, October 03, 2011 12:10 PM
To: Discussion list for OpenIndiana
Subject: Re: [OpenIndiana-discuss] Best way to map SAS
drivestophysicalpositions

On Mon, Oct 3, 2011 at 20:55, Russell Hansen 
wrote:
> I stumbled upon a nice script and link to a utility for just such an
occasion...

That looks very promising, thank you for sharing! I'll have to give
that a try tomorrow.

--
Venlig hilsen / Kind regards
Jeppe Toustrup (aka. Tenzer)

___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss
___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

[OpenIndiana-discuss] 151 auto installer

[Jason Matthews]

Are their plans to release the AI for OI151 soon?

I was planning on using OI148 and AI for a production environment. However,
the repo is reportedly broken with no 'will' to fix it. This leaves me in
the lurch if there is not AI for OI151.

Alternately, does someone have a snaphot of the OI148 repo before the 151
packages broke it?


Thanks,
j.
___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] 151 auto installer

[Jason Matthews]

Josh,

That's helpful thanks. It is funny, I was just looking at your blog when
this email came in

j.

___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] Poor relative performance of SAS overSATA drives

[Jason Matthews]

> We've generally used only SAS for our ZFS pools, and will continue to
> do so. However, we're looking at adding some SSDs and those are almost
> exclusively SATA. Do folks see issues mixing SATA SSDs as cache and
> log devices in with SAS disk for hybrid storage pools?


I put my SAS drives in disk shelves (Sanima-SC/Newisys NDS-2240) and my
cache in the system. My config uses an 9205-8e for the shelves, a 9211-8i
for boot disk & L2ARC and an DDRdrive X1 for ZIL acceleration. In general, I
write off non-LSI or non-Marvell on-board disk controllers.

My current line of thinking is anything up to the SAS2008 chipset is
probably too slow to fully a accommodate a system loaded with modern
massively utilized SSDs and drive the storage portion of the pool well if
you have a significant number of disks.

A number of fully utilized SSDs can give disk controllers a solid run for
their money -- forget about putting them behind RAID cards. The next spin of
HP's raid controller is rumored to have a Nehalem processor on it so it can
keep up with SSDs.


On a side note, if you put most SSDs in the an HP p410 RAID, the system
shuts down within a couple of minutes complaining about thermal overloads --
go figure. Likewise, if you put an LSI card into a DL380G7 and plug into the
passive back plane it improperly detects the drive location and on occasion
I believe it fully corrupted disks. While HP may be Solaris friendly, it
doesn't seem to be 3rd party friendly.

BTW, the Seagate Pulsar is SAS. I haven't had the pleasure of giving one a
whirl yet.

j.





___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] Poor relative performance of SAS overSATA drives

[Jason Matthews]

What is the link speed of the SAS drives as reported by the controller?

j.

-Original Message-
From: Hong Wei Liam [mailto:weiliam.h...@gmail.com] 
Sent: Wednesday, October 26, 2011 4:59 PM
To: Discussion list for OpenIndiana
Subject: Re: [OpenIndiana-discuss] Poor relative performance of SAS overSATA
drives

I had initially  tried only having the SAS drives installed with the same
results.

Later, the SATA drives were added in for comparison.

Regards,
WL


On Oct 27, 2011, at 2:27 AM, Jason J. W. Williams wrote:

> Is the card hosting any SATA and SAS drives on the same port, or are
> they segregated SAS on one and SATA on the other?
> 
> -J
> 
> On Wed, Oct 26, 2011 at 9:01 AM, weiliam.hong 
wrote:
>> Greetings,
>> 
>> I have a fresh installation of OI151a:
>> - SM X8DTH, 12GB RAM, LSI 9211-8i (latest IT-mode firmware)
>> - pool_A : SG ES.2 Constellation (SAS)
>> - pool_B : WD RE4 (SATA)
>> - no settings in /etc/system
>> 
>> 
>> *zpool status output*
>> ---
>> admin@openindiana:~# zpool status
>>  pool: pool_A
>>  state: ONLINE
>>  scan: none requested
>> config:
>> 
>>NAME   STATE READ WRITE CKSUM
>>pool_A ONLINE   0 0 0
>>  mirror-0 ONLINE   0 0 0
>>c7t5000C50035062EC1d0  ONLINE   0 0 0
>>c8t5000C50034C03759d0  ONLINE   0 0 0
>> 
>>  pool: pool_B
>>  state: ONLINE
>>  scan: none requested
>> config:
>> 
>>NAME   STATE READ WRITE CKSUM
>>pool_B ONLINE   0 0 0
>>  mirror-0 ONLINE   0 0 0
>>c1t50014EE057FCD628d0  ONLINE   0 0 0
>>c2t50014EE6ABB89957d0  ONLINE   0 0 0
>> 
>> 
>> *Load generation via 2 concurrent dd streams:*
>> --
>> dd if=/dev/zero of=/pool_A/bigfile bs=1024k count=100
>> dd if=/dev/zero of=/pool_B/bigfile bs=1024k count=100
>> 
>> 
>> *Initial Observation*
>> ---
>> 
>>   capacity operationsbandwidth
>> poolalloc   free   read  write   read  write
>> --  -  -  -  -  -  -
>> pool_A  1.68G  2.72T  0652  0  73.4M
>>  mirror1.68G  2.72T  0652  0  73.4M
>>c7t5000C50035062EC1d0  -  -  0619  0  73.4M
>>c8t5000C50034C03759d0  -  -  0619  0  73.4M
>> --  -  -  -  -  -  -
>> pool_B  1.54G  1.81T  0  1.05K  0   123M
>>  mirror1.54G  1.81T  0  1.05K  0   123M
>>c1t50014EE057FCD628d0  -  -  0  1.02K  0   123M
>>c2t50014EE6ABB89957d0  -  -  0  1.01K  0   123M
>> 
>> *
>> 10-15mins later*
>> =--
>> 
>>   capacity operationsbandwidth
>> poolalloc   free   read  write   read  write
>> --  -  -  -  -  -  -
>> pool_A  15.5G  2.70T  0 50  0  6.29M
>>  mirror15.5G  2.70T  0 50  0  6.29M
>>c7t5000C50035062EC1d0  -  -  0 62  0  7.76M
>>c8t5000C50034C03759d0  -  -  0 50  0  6.29M
>> --  -  -  -  -  -  -
>> pool_B  28.0G  1.79T  0  1.07K  0   123M
>>  mirror28.0G  1.79T  0  1.07K  0   123M
>>c1t50014EE057FCD628d0  -  -  0  1.02K  0   123M
>>c2t50014EE6ABB89957d0  -  -  0  1.02K  0   123M
>> 
>> 
>> 
>> Questions:
>> 1. Why does SG SAS drives degrade to <10 MB/s while WD RE4 remain
consistent
>> at >100MB/s after 10-15 min?
>> 2. Why does SG SAS drive show only 70+ MB/s where is the published
figures
>> are > 100MB/s refer here
>>
?
>> 3. All 4 drives are connected to a single HBA, so I assume the mpt_sas
>> driver is used. Are SAS and SATA drives handled differently ?
>> 
>> 
>> This is a test server, so any ideas to try and help me understand greatly
>> appreciated.
>> 
>> 
>> Many thanks,
>> WL
>> 
>> 
>> 
>> 
>> ___
>> OpenIndiana-discuss mailing list
>> OpenIndiana-discuss@openindiana.org
>> http://openindiana.org/mailman/listinfo/openindiana-discuss
>> 
> ___
> OpenIndiana-discuss mailing list
> OpenIndiana-discuss@openindiana.org
> http://openindiana.org/mailman/listinfo/openindiana-discuss


___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss
___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/l