Re: [OpenIndiana-discuss] ML350G6 What to do?

[jason matthews]

On 12/10/15 5:35 AM, the outsider wrote:
  
3. idea is to create a ZFS RAID Z2 pool with 4 or more drives
RAIDZ2 on four drives, if you need any kind of performance, is silly. 
The red drives are still fairly slow by todays standards. Performance 
wise, you would be better off with a set of 2x2 mirrors. WWJD - what 
would jason do. Well, mirrors, that's what I would do.


In my humble opinion, you should buy more smaller drives than a just a 
few big ones. WD doesnt have the best name in the navy for reliability 
either so consider spares as well. I use 6TB red drives in my home 
system, but I keep them in 3-way mirrors -- i wont roll the dice using 
western digital without extreme protection. The WD's cousin, HGST, 
produces highly reliable drives but they cost more.


The specs say you can put up to 8 WD Red drives into an enclosure, where 
the limiting factor is vibration. The WD Pro drives can do 16 
drives/encosure or so the specs say.  I am not sure how many bays you 
have, probably 8, but I would plan to use them all. Resilvering a 4TB 
drive takes a long long time.


Aside from raidz2 versus mirrors, please consider how you are going to 
maintain a backup in case the unexpected happens. It might be that you 
backup to a single 6tb drive. So, that is something to think about. HGST 
has some new cool 10TB drives, but i havent checked on pricing for those.




The questions I have:

1. What should I install: OI Hipster or OI 151a8 or ??



I use oi151a9 at home and a combination of a9 and smartos at work. 
smartos provides the flexibility of LX branded zones for those instances 
where you dont want the potential instability by adding KVM to the mix. 
This has been handy for example, to run the controller for ubiquiti 
access points or to spin up modern versions of ELK without having to 
port a bunch of software way outside of our core line of business.


I primarily view the  driver as ZFS. When people balk when I tell them 
Apsalar is using IllumOS I proceed to describe the system hundreds of 
netapp filers on which we can run applications with fantastic data 
handling capabilities -- then they sort of get it in abstract way. For 
me, it is about ZFS. I am happy to use freebsd or any illumos 
distribution. The politics and packaging systems are a distraction to 
me. It is not like your system is going to run faster or slower based on 
your distribution.




2. What is the current limit for OI and WD RED series SATA drive sizes?

Limit? I dunno about a limit. My 6TB reds worked out of the box on a LSI 
2308 based controller. If there is a problem, it will in your controller 
not in IllumOS.




3. I could create a RAID1 config on the P410i with 2x 500GB for the OI OS,
then I am a bit safer regarding disk failure. But would it work?

Safer? No, not safer. I see no reason to use hardware raid.



4. Because the P410i doesn't have JBOD function I want to set all disks as
RAID0 single disk and create a ZFS pool around them, or ?

This is the definition of jenky

Why dont you just use a proper controller, like an LSI 9207-8i.





___
openindiana-discuss mailing list
openindiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] KVM Failed to allocate memory: Resource temporarily unavailable

[Jonathan Adams]

how much swap do you have?

KVM automatically allocates the same amount of swap as memory when you
start an instance, if you don't have the free swap it won't work.

Jon

On 10 December 2015 at 04:04, Fucai Liang （BLCT） 
wrote:

>
>
> Hello, guys：
>
> I has a server running oi_151.1.8, the server have 32G memory .
>
>
> root@oi01:~# prtconf | grep Memory
> Memory size: 32760 Megabytes
> root@oi01:~# echo "::memstat" | mdb -k
> Page SummaryPagesMB  %Tot
>      
> Kernel 451736  17645%
> ZFS File Data   78429   3061%
> Anon19849770%
> Exec and libs 586 20%
> Page cache   2170 80%
> Free (cachelist)10432400%
> Free (freelist)   7821076 30551   93%
>
> Total 8384278 32751
> Physical  8384277 32751
> root@oi01:~#
>
>
> I lauch a KVM  VM  by
>
>
> #!/bin/sh
>
> qemu-kvm \
> -vnc 0.0.0.0:2 \
> -cpu host \
> -smp 2 \
> -m 8192 \
> -no-hpet \
> -localtime \
> -drive file=/dev/zvol/rdsk/rpool/svrinit,if=virtio,index=0 \
> -net nic,vlan=0,name=e1000g0,model=e1000,macaddr=2:8:20:bd:ae:01 \
> -net
> vnic,vlan=0,name=e1000g0,ifname=vnic01,macaddr=2:8:20:bd:ae:01 \
> -vga std \
> -daemonize
>
>
> root@oi01:~# echo "::memstat" | mdb -k
> Page SummaryPagesMB  %Tot
>      
> Kernel 461061  18015%
> ZFS File Data   78446   3061%
> Anon  2131265  8325   25%
> Exec and libs 918 30%
> Page cache   2255 80%
> Free (cachelist)10015390%
> Free (freelist)   5700318 22266   68%
>
> Total 8384278 32751
> Physical  8384277 32751
> root@oi01:~#
>
>
> then I launch second  KVM VM,
>
> #!/bin/sh
>
> qemu-kvm \
> -enable-kvm \
> -vnc 0.0.0.0:3 \
> -cpu host \
> -smp 2 \
> -m 8192 \
> -no-hpet \
> -localtime \
> -drive file=/dev/zvol/rdsk/rpool/svr03,if=virtio,index=0 \
> -net nic,vlan=0,name=e1000g0,model=e1000,macaddr=2:8:20:bd:ae:03 \
> -net
> vnic,vlan=0,name=e1000g0,ifname=vnic03,macaddr=2:8:20:bd:ae:03 \
> -vga std \
> -daemonize
>
>
>
> I got err message  :
> Failed to allocate memory: Resource temporarily unavailable
>
> -bash: fork: Not enough space
>
> .
>
> the same ietmes happen on OmniOS , the following is a OmniOS-discuss  mail
> .
>
> That caused by availrmem not enough for memory locked .
>
> Is that mean I can only use about 14G memory for KVM VM in a 32G memory
> installed server?
> OS can only locked about 14G memory .
>
> ——FOLLOWING--
>
> Hello, guys：
>
> I has a server running OmniOS v11 r151016.  the server have 32G memory .
> I star tow kvm virtual machines by running the following commands:
>
> qemu-system-x86_64 -enable-kvm -vnc 0.0.0.0:12 -cpu host -smp 4 -m 8192
> -no-hpe
>
>
> qemu-system-x86_64 -enable-kvm -vnc 0.0.0.0:11 -cpu host -smp 2 -m 4096
> -no-hpe
>
> one use 8G memory and the other one use 4G memory.
>
> now the memory usage of the system as following:
>
> root@BLCC01:/root# prtconf | grep Memory
> Memory size: 32760 Megabytes
> root@BLCC01:/root#  echo "::memstat" | mdb -k
> Page SummaryPagesMB  %Tot
>      
> Kernel 549618  21467%
> ZFS File Data  668992  26138%
> Anon  3198732 12495   38%
> Exec and libs1411 50%
> Page cache   4402170%
> Free (cachelist)10578410%
> Free (freelist)   3950545 15431   47%
>
> Total 8384278 32751
> Physical  8384277 32751
> root@BLCC01:/root# swap -sh
> total: 12G allocated + 35M reserved = 12G used, 6.8G available
> root@BLCC01:/root# swap -l
> swapfile devswaplo   blocks free
> /dev/zvol/dsk/rpool/swap 263,2 8  8388600  8388600
> root@BLCC01:/root#
>
>
> root@BLCC01:/root# prctl $$
>
> project.max-locked-memory
>   usage   12.0GB
>   system  16.0EBmax   deny
>  -
> project.max-port-ids
>   privileged  8.19K   -   deny
>  -
>   system  65.5K max   deny
>  -
> project.max-shm-memory
>   privileged 

Re: [OpenIndiana-discuss] KVM Failed to allocate memory: Resource temporarily unavailable

[Fucai Liang （BLCT）]

Thanks  for your help！

root@oi01:~# swap -sh
total: 8.1G allocated + 21M reserved = 8.1G used, 16G available
root@oi01:~#  swap -l
swapfile devswaplo   blocks free
/dev/zvol/dsk/rpool/swap 100,2 8 12574712 12574712
root@oi01:~#


16G swap ， so KVM can locked 16G memory .

thanks again .


--
Fucai Liang 




On Dec 10, 2015, at 17:24, Jonathan Adams  wrote:

> how much swap do you have?
> 
> KVM automatically allocates the same amount of swap as memory when you
> start an instance, if you don't have the free swap it won't work.
> 
> Jon
> 
> On 10 December 2015 at 04:04, Fucai Liang （BLCT） 
> wrote:
> 
>> 
>> 
>> Hello, guys：
>> 
>> I has a server running oi_151.1.8, the server have 32G memory .
>> 
>> 
>> root@oi01:~# prtconf | grep Memory
>> Memory size: 32760 Megabytes
>> root@oi01:~# echo "::memstat" | mdb -k
>> Page SummaryPagesMB  %Tot
>>      
>> Kernel 451736  17645%
>> ZFS File Data   78429   3061%
>> Anon19849770%
>> Exec and libs 586 20%
>> Page cache   2170 80%
>> Free (cachelist)10432400%
>> Free (freelist)   7821076 30551   93%
>> 
>> Total 8384278 32751
>> Physical  8384277 32751
>> root@oi01:~#
>> 
>> 
>> I lauch a KVM  VM  by
>> 
>> 
>> #!/bin/sh
>> 
>> qemu-kvm \
>>-vnc 0.0.0.0:2 \
>>-cpu host \
>>-smp 2 \
>>-m 8192 \
>>-no-hpet \
>>-localtime \
>> -drive file=/dev/zvol/rdsk/rpool/svrinit,if=virtio,index=0 \
>>-net nic,vlan=0,name=e1000g0,model=e1000,macaddr=2:8:20:bd:ae:01 \
>>-net
>> vnic,vlan=0,name=e1000g0,ifname=vnic01,macaddr=2:8:20:bd:ae:01 \
>>-vga std \
>>-daemonize
>> 
>> 
>> root@oi01:~# echo "::memstat" | mdb -k
>> Page SummaryPagesMB  %Tot
>>      
>> Kernel 461061  18015%
>> ZFS File Data   78446   3061%
>> Anon  2131265  8325   25%
>> Exec and libs 918 30%
>> Page cache   2255 80%
>> Free (cachelist)10015390%
>> Free (freelist)   5700318 22266   68%
>> 
>> Total 8384278 32751
>> Physical  8384277 32751
>> root@oi01:~#
>> 
>> 
>> then I launch second  KVM VM,
>> 
>> #!/bin/sh
>> 
>> qemu-kvm \
>>-enable-kvm \
>>-vnc 0.0.0.0:3 \
>>-cpu host \
>>-smp 2 \
>>-m 8192 \
>>-no-hpet \
>>-localtime \
>> -drive file=/dev/zvol/rdsk/rpool/svr03,if=virtio,index=0 \
>>-net nic,vlan=0,name=e1000g0,model=e1000,macaddr=2:8:20:bd:ae:03 \
>>-net
>> vnic,vlan=0,name=e1000g0,ifname=vnic03,macaddr=2:8:20:bd:ae:03 \
>>-vga std \
>>-daemonize
>> 
>> 
>> 
>> I got err message  :
>> Failed to allocate memory: Resource temporarily unavailable
>> 
>> -bash: fork: Not enough space
>> 
>> .
>> 
>> the same ietmes happen on OmniOS , the following is a OmniOS-discuss  mail
>> .
>> 
>> That caused by availrmem not enough for memory locked .
>> 
>> Is that mean I can only use about 14G memory for KVM VM in a 32G memory
>> installed server?
>> OS can only locked about 14G memory .
>> 
>> ——FOLLOWING--
>> 
>> Hello, guys：
>> 
>> I has a server running OmniOS v11 r151016.  the server have 32G memory .
>> I star tow kvm virtual machines by running the following commands:
>> 
>> qemu-system-x86_64 -enable-kvm -vnc 0.0.0.0:12 -cpu host -smp 4 -m 8192
>> -no-hpe
>> 
>> 
>> qemu-system-x86_64 -enable-kvm -vnc 0.0.0.0:11 -cpu host -smp 2 -m 4096
>> -no-hpe
>> 
>> one use 8G memory and the other one use 4G memory.
>> 
>> now the memory usage of the system as following:
>> 
>> root@BLCC01:/root# prtconf | grep Memory
>> Memory size: 32760 Megabytes
>> root@BLCC01:/root#  echo "::memstat" | mdb -k
>> Page SummaryPagesMB  %Tot
>>      
>> Kernel 549618  21467%
>> ZFS File Data  668992  26138%
>> Anon  3198732 12495   38%
>> Exec and libs1411 50%
>> Page cache   4402170%
>> Free (cachelist)10578410%
>> Free (freelist)   3950545 15431   47%
>> 
>> Total 8384278 32751
>> Physical  

Re: [OpenIndiana-discuss] KVM Failed to allocate memory: Resource temporarily unavailable

[Jonathan Adams]

If you have the disk space, you might be as well creating a swap size of
memory x 2. Old Solaris used to suggest you set it to memory x 4, but that
was when memory was limited and expensive.

On 10 December 2015 at 10:01, Fucai Liang （BLCT） 
wrote:

> Thanks  for your help！
>
> root@oi01:~# swap -sh
> total: 8.1G allocated + 21M reserved = 8.1G used, 16G available
> root@oi01:~#  swap -l
> swapfile devswaplo   blocks free
> /dev/zvol/dsk/rpool/swap 100,2 8 12574712 12574712
> root@oi01:~#
>
>
> 16G swap ， so KVM can locked 16G memory .
>
> thanks again .
>
>
> --
> Fucai Liang
>
>
>
>
> On Dec 10, 2015, at 17:24, Jonathan Adams  wrote:
>
> > how much swap do you have?
> >
> > KVM automatically allocates the same amount of swap as memory when you
> > start an instance, if you don't have the free swap it won't work.
> >
> > Jon
> >
> > On 10 December 2015 at 04:04, Fucai Liang （BLCT） 
> > wrote:
> >
> >>
> >>
> >> Hello, guys：
> >>
> >> I has a server running oi_151.1.8, the server have 32G memory .
> >>
> >>
> >> root@oi01:~# prtconf | grep Memory
> >> Memory size: 32760 Megabytes
> >> root@oi01:~# echo "::memstat" | mdb -k
> >> Page SummaryPagesMB  %Tot
> >>      
> >> Kernel 451736  17645%
> >> ZFS File Data   78429   3061%
> >> Anon19849770%
> >> Exec and libs 586 20%
> >> Page cache   2170 80%
> >> Free (cachelist)10432400%
> >> Free (freelist)   7821076 30551   93%
> >>
> >> Total 8384278 32751
> >> Physical  8384277 32751
> >> root@oi01:~#
> >>
> >>
> >> I lauch a KVM  VM  by
> >>
> >>
> >> #!/bin/sh
> >>
> >> qemu-kvm \
> >>-vnc 0.0.0.0:2 \
> >>-cpu host \
> >>-smp 2 \
> >>-m 8192 \
> >>-no-hpet \
> >>-localtime \
> >> -drive file=/dev/zvol/rdsk/rpool/svrinit,if=virtio,index=0 \
> >>-net nic,vlan=0,name=e1000g0,model=e1000,macaddr=2:8:20:bd:ae:01
> \
> >>-net
> >> vnic,vlan=0,name=e1000g0,ifname=vnic01,macaddr=2:8:20:bd:ae:01 \
> >>-vga std \
> >>-daemonize
> >>
> >>
> >> root@oi01:~# echo "::memstat" | mdb -k
> >> Page SummaryPagesMB  %Tot
> >>      
> >> Kernel 461061  18015%
> >> ZFS File Data   78446   3061%
> >> Anon  2131265  8325   25%
> >> Exec and libs 918 30%
> >> Page cache   2255 80%
> >> Free (cachelist)10015390%
> >> Free (freelist)   5700318 22266   68%
> >>
> >> Total 8384278 32751
> >> Physical  8384277 32751
> >> root@oi01:~#
> >>
> >>
> >> then I launch second  KVM VM,
> >>
> >> #!/bin/sh
> >>
> >> qemu-kvm \
> >>-enable-kvm \
> >>-vnc 0.0.0.0:3 \
> >>-cpu host \
> >>-smp 2 \
> >>-m 8192 \
> >>-no-hpet \
> >>-localtime \
> >> -drive file=/dev/zvol/rdsk/rpool/svr03,if=virtio,index=0 \
> >>-net nic,vlan=0,name=e1000g0,model=e1000,macaddr=2:8:20:bd:ae:03
> \
> >>-net
> >> vnic,vlan=0,name=e1000g0,ifname=vnic03,macaddr=2:8:20:bd:ae:03 \
> >>-vga std \
> >>-daemonize
> >>
> >>
> >>
> >> I got err message  :
> >> Failed to allocate memory: Resource temporarily unavailable
> >>
> >> -bash: fork: Not enough space
> >>
> >> .
> >>
> >> the same ietmes happen on OmniOS , the following is a OmniOS-discuss
> mail
> >> .
> >>
> >> That caused by availrmem not enough for memory locked .
> >>
> >> Is that mean I can only use about 14G memory for KVM VM in a 32G memory
> >> installed server?
> >> OS can only locked about 14G memory .
> >>
> >> ——FOLLOWING--
> >>
> >> Hello, guys：
> >>
> >> I has a server running OmniOS v11 r151016.  the server have 32G memory .
> >> I star tow kvm virtual machines by running the following commands:
> >>
> >> qemu-system-x86_64 -enable-kvm -vnc 0.0.0.0:12 -cpu host -smp 4 -m 8192
> >> -no-hpe
> >>
> >>
> >> qemu-system-x86_64 -enable-kvm -vnc 0.0.0.0:11 -cpu host -smp 2 -m 4096
> >> -no-hpe
> >>
> >> one use 8G memory and the other one use 4G memory.
> >>
> >> now the memory usage of the system as following:
> >>
> >> root@BLCC01:/root# prtconf | grep Memory
> >> Memory size: 32760 Megabytes
> >> root@BLCC01:/root#  echo "::memstat" | mdb -k
> >> Page SummaryPagesMB  %Tot
> >>  

[OpenIndiana-discuss] ML350G6 What to do?

[the outsider]

I already use OI 151a9 on a HP N54L server since 2013 but I build it poorly,
with 2x2TB in basic ZFS mirror and 1 very old 250GB for OI itself. 

 

To prevent myself from disaster I searched for a bargain and found something
nicely second hand, so I bought an HP ML350G6 with the (infamous) P410i RAID
card with 512MB RAM and battery backup. 

Server has 1 Xeon L5630, 6x 3,5" SAS/SATA bays and 32 GB RAM. It should
consume low power due to the 40W processor. 

 

it is just a little bit bigger then the HP microserver ;-) But now I have a
battery backupped HD controller, 32GB RAM and 6 3,5" bays and it was cheaper
than a new Microserver. 

 

The server is for home use. I would appreciate suggestions based on the
following needs:

1. the server needs to run several Solaris zones for testing and separating
different jobs

2. the server needs to have SATA drives (WD RED series) 2,3 or 4 TB each

3. idea is to create a ZFS RAID Z2 pool with 4 or more drives

 

The questions I have:

1. What should I install: OI Hipster or OI 151a8 or ??

2. What is the current limit for OI and WD RED series SATA drive sizes? 

3. I could create a RAID1 config on the P410i with 2x 500GB for the OI OS,
then I am a bit safer regarding disk failure. But would it work? 

4. Because the P410i doesn't have JBOD function I want to set all disks as
RAID0 single disk and create a ZFS pool around them, or ? 

 

Br,

Roel

 

 

___
openindiana-discuss mailing list
openindiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] ML350G6 What to do?

[Jussi Hamalainen]

On Thu, 10 Dec 2015, the outsider wrote:


To prevent myself from disaster I searched for a bargain and found something
nicely second hand, so I bought an HP ML350G6 with the (infamous) P410i RAID
card with 512MB RAM and battery backup.

2. the server needs to have SATA drives (WD RED series) 2,3 or 4 TB each


I don't mean to spoil your party but I need to bring up something 
which I have shared on this list before:


The HP SmartArray controllers require drives with HP firmware. There 
are some non-standard commands the array controller uses when talking 
to the drives. Using non-HP drives will lead to strange issues with 
drives randomly going offline or "failing".


--
-=[ Count Zero / TBH - Jussi Hämäläinen - email co...@theblah.fi ]=-
___
openindiana-discuss mailing list
openindiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] ML350G6 What to do?

[Reginald Beardsley via openindiana-discuss]

FWIW 

My N40L is configured with 4 x 2 TB.  Each drive has a small (~100 GB) slice 
with the rest of the disk in a large slice.  I have a 4 way mirror for rpool 
using the small slices and the rest is in a RAIDZ2 pool.  Performance has been 
quite satisfactory.

I configured it this way because you could not boot from EFI disks and had to 
use EFI for disks larger than 2 TB.  I seem to recall that the EFI boot problem 
may have been solved.  In which case you could do a 6 way mirrored rpool and 
RAIDZ2 for the rest of the disks using 4 TB disks.

Since you have 6 bays you could also do a 2 way mirror rpool (e.g. 128 GB SSD) 
and a 4 disk RAIDZ2 using EFI labeled disks.

Have Fun!
Reg

___
openindiana-discuss mailing list
openindiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss