Re: recommendations for file server based zfs appliance

[Stefan Hagen]

* Outback Dingo wrote:

Sorry but  need to as do yo want some under powered atom system? or a
core i3i7 cpu and 16 Gbs memoery, with 8 GB ports and 6 SATA ports in
a small custom build.
because thats exactly what we sell and we sell it for less the an XL
 by a few hundred bucs, less the drives. it will easly take 6 3.5
drives and 2 2.5. IF you really
want the specs on a sweet low end system with higher specs then the
miniXL and cheaper also then say a super micro or HP box let me know.


Who is "we"?

Best Regards,
Stefan
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: recommendations for file server based zfs appliance

[Outback Dingo]

On Fri, Aug 18, 2017 at 8:03 AM, Stefan Hagen
 wrote:
> * Patrick M. Hausen wrote:
>>>
>>> Am 18.08.2017 um 11:19 schrieb Pete French :
>>> The HP micro servers work very well, and you can pick them up remakably
>>> cheaply [...]
>>> Not sure about ECC memory support there though.
>>
>>
>> They do support ECC, no problem.
>>
>> They are available with different CPU configurations from
>> as Pete said remarkably cheap Celeron D based systems
>> up to Xeon CPUs.
>
>
> I've just sold my Microserver Gen8 (Xeon) just recently.
> It's a beautiful little machine, but I didn't make me happy in the long run.
>
> Reasons:
> - Limited to 8GB Ram in total
> - Only 4 HDDs
> - JBOD support is not great
> - Harddrives are never going to sleep (not supported)
>
> Installing FreeBSD was harder than expected. The machine refused to boot
> FreeBSD
> from the internal non-raid SATA ports. I didn't try FreeNAS though.
>
> Also, in case you go with a Microserver - I think all non-xeon models do not
> support AES-NI, which will cut the throughput in case you plan to encrypt
> your
> drives.
>
> iX-Systems would have been my choice to replace this machine until I found
> out
> that I can build something myself that suits my needs even better.
>
> Note, I was looking at the 8 bay model and I would have to add around $280
> for
> shipping and tax to Germany.
>
> I'm now going for this custom build:
> - Fractal Design Define R5
>   Reason: Silent, enough space for 12 drives
> - SuperMicro MBD-A1SAI-2550F-O
>   Reason: IPMI, ECC, AES-NI, 64GB Ram, Intel NIC, low power consumption
> - 4x 8GB Kingson ECC SODIMM
>   Reason: 16GB modules are not yet available
> - Seasonic X-Series Fanless X-400FL 400W passiv
>   Reason: Fanless -> silent
>
> The complete order was around 1000€
>
> That being said, I'm planning to put an older 3ware controller in JBOD mode
> in
> it and I also have SSDs and drives already. The board has only 6 SATA ports.
> But that would be enough for you.
>
> This might be a little big for you. But maybe it gives you ideas.
>
> Best Regards,
> Stefan
>

Sorry but  need to as do yo want some under powered atom system? or a
core i3i7 cpu and 16 Gbs memoery, with 8 GB ports and 6 SATA ports in
a small custom build.
because thats exactly what we sell and we sell it for less the an XL
 by a few hundred bucs, less the drives. it will easly take 6 3.5
drives and 2 2.5. IF you really
want the specs on a sweet low end system with higher specs then the
miniXL and cheaper also then say a super micro or HP box let me know.

> ___
> freebsd-stable@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: recommendations for file server based zfs appliance

[Pete French]

> temperature sensor is somehow controlled by the raid controller. So enabling 
> JBOD technically works, but the machine is a lot louder than with enabled
> raid. As a workaround I've built 4 raid arrays with one disk each.

Upgrade the firmware. That fixed it for me - I run JBOD and the fanes are nice
and quiet. Noisy on boot, like all HP servers, but then settle down when
the OS starts. I have this positioned under my TV and low noise was
the main reason for choosing it.

Of course, getting the firmware upgraded means finding an HP firmware
maintenance DVD image from somewhere, as they restricted downloading them.

-pete.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: recommendations for file server based zfs appliance

[Patrick M. Hausen]

Hi Stefan,

> Am 18.08.2017 um 14:35 schrieb Stefan Hagen :
>> What do you mean by "JBOD support"? Disable RAID in the systems BIOS
>> setup, put ZFS on AHCI drives ...
> 
> Activating JBOD deactivates the B120i raid controller. Unfortnately the 
> temperature sensor
> is somehow controlled by the raid controller. So enabling JBOD technically 
> works, but the
> machine is a lot louder than with enabled raid. As a workaround I've built 4 
> raid arrays with one disk each.
> 
> It works, but it doesn't feel right to me.

Weird - the machine seems rather quiet to me, but it's in a separate
room with the laser printers and no one actually working there.

I don't know how to measure noise, unfortunately. I have another
Gen8 at my wife's office for their not-for-profit organisation.
Quiet as a mouse with FreeNAS 11. This time two also very quiet
(most of the time) computer based office desks in that room.

My machine here at my office:

root@filer:~ # ipmitool sensor
UID Light| 0x0| discrete   | 0x0180| na| na| na 
   | na| na| na
Health LED   | 0x0| discrete   | 0x0180| na| na| na 
   | na| na| na
01-Inlet Ambient | 28.000 | degrees C  | ok| na| na| na 
   | na| 42.000| 46.000
02-CPU   | 40.000 | degrees C  | ok| na| na| na 
   | na| 70.000| na
03-P1 DIMM 1-2   | 42.000 | degrees C  | ok| na| na| na 
   | na| 87.000| na
04-HD Max| na || na| na| na| na 
   | na| 60.000| na
05-Chipset   | 63.000 | degrees C  | ok| na| na| na 
   | na| 105.000   | na
06-Chipset Zone  | 48.000 | degrees C  | ok| na| na| na 
   | na| 68.000| 73.000
07-VR P1 Zone| 52.000 | degrees C  | ok| na| na| na 
   | na| 88.000| 93.000
08-Supercap Max  | na || na| na| na| na 
   | na| 65.000| na
09-iLO Zone  | 47.000 | degrees C  | ok| na| na| na 
   | na| 72.000| 77.000
10-PCI 1 | na || na| na| na| na 
   | na| 100.000   | na
11-PCI 1 Zone| 39.000 | degrees C  | ok| na| na| na 
   | na| 64.000| 69.000
12-Sys Exhaust   | 48.000 | degrees C  | ok| na| na| na 
   | na| 68.000| 73.000
13-LOM   | na || na| na| na| na 
   | na| 100.000   | na
Fan 1| 25.088 | percent| ok| na| na| na 
   | na| na| na
Power Supply 1   | 0x0| discrete   | 0x0180| na| na| na 
   | na| na| na
Memory   | 0x0| discrete   | 0x4080| na| na| na 
   | na| na| na

So, temperature sensor working, too. Or are you referring to the
*HDD* temperature? I never put disks to sleep in servers ...

Patrick


signature.asc
Description: Message signed with OpenPGP

Re: recommendations for file server based zfs appliance

[Stefan Hagen]

Hi Patrick,

* Patrick M. Hausen wrote:

smbios.system.product="ProLiant MicroServer Gen8"
CPU: Intel(R) Xeon(R) CPU E3-1220L V2 @ 2.30GHz (2294.80-MHz K8-class CPU)

16 GB RAM, FreeBSD installation was no problem at all.


You got me. I just double checked. 16GB is the limit. You're right.


What do you mean by "JBOD support"? Disable RAID in the systems BIOS
setup, put ZFS on AHCI drives ...


Activating JBOD deactivates the B120i raid controller. Unfortnately the 
temperature sensor is somehow controlled by the raid controller. So enabling 
JBOD technically works, but the machine is a lot louder than with enabled

raid. As a workaround I've built 4 raid arrays with one disk each.

It works, but it doesn't feel right to me.

It's my home file server sitting basically next to my monitor. In an office 
environment, you probably don't care about the noise so much.


See: 
https://community.hpe.com/t5/ProLiant-Servers-Netservers/MicroServer-Gen8-is-noisy/td-p/6171563


Best Regards,
Stefan
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: recommendations for file server based zfs appliance

[Rafał Lukawiecki]

The SuperMicro X10SDV-6C+-TLN4F or X10SDV-4C+-TLN4F have twin 10Gb Intel NIC 
and a twin 1Gb, 4 ports in total, plus a management one. Support 128GB ECC 
RDIMM in total or 64GB UDIMM. 
--
Rafal Lukawiecki


> On 18 Aug 2017, at 12:58, Pete French  wrote:
> 
> The only 10 gig cards we have are
> some 2nd hand Mellanox ConnectX-2 cards because they are dirt cheap on
> eBay.

___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: recommendations for file server based zfs appliance

[Patrick M. Hausen]

Hi!

> Am 18.08.2017 um 14:03 schrieb Stefan Hagen :
> 
> * Patrick M. Hausen wrote:
>>> Am 18.08.2017 um 11:19 schrieb Pete French :
>>> The HP micro servers work very well, and you can pick them up remakably 
>>> cheaply [...]
>>> Not sure about ECC memory support there though.
>> 
>> They do support ECC, no problem.
>> 
>> They are available with different CPU configurations from
>> as Pete said remarkably cheap Celeron D based systems
>> up to Xeon CPUs.
> 
> I've just sold my Microserver Gen8 (Xeon) just recently.
> It's a beautiful little machine, but I didn't make me happy in the long run.
> 
> Reasons:
> - Limited to 8GB Ram in total
> - Only 4 HDDs
> - JBOD support is not great
> - Harddrives are never going to sleep (not supported)
> 
> Installing FreeBSD was harder than expected. The machine refused to boot 
> FreeBSD
> from the internal non-raid SATA ports. I didn't try FreeNAS though.

smbios.system.product="ProLiant MicroServer Gen8"
CPU: Intel(R) Xeon(R) CPU E3-1220L V2 @ 2.30GHz (2294.80-MHz K8-class CPU)

16 GB RAM, FreeBSD installation was no problem at all.

What do you mean by "JBOD support"? Disable RAID in the systems BIOS
setup, put ZFS on AHCI drives ...

ahci0:  port 
0x10c0-0x10c7,0x10c8-0x10cb,0x10d0-0x10d7,0x10d8-0x10db,0x10e0-0x10ff mem 
0xfacd-0xfacd07ff irq 17 at device 31.2 on pci0
ahci0: [ITHREAD]
ahci0: AHCI v1.30 with 6 6Gbps ports, Port Multiplier supported
ahcich0:  at channel 0 on ahci0
ahcich0: [ITHREAD]
ada0 at ahcich0 bus 0 scbus0 target 0 lun 0
ada0:  ATA-8 SATA 3.x device
ada0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes)
ada0: Command Queueing enabled
ada0: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)

This is our office file, print and mail server. No FreeNAS this time,
just plain FreeBSD.

Patrick


signature.asc
Description: Message signed with OpenPGP

Re: recommendations for file server based zfs appliance

[Stefan Hagen]

* Patrick M. Hausen wrote:

Am 18.08.2017 um 11:19 schrieb Pete French :
The HP micro servers work very well, and you can pick them up remakably cheaply 
[...]
Not sure about ECC memory support there though.


They do support ECC, no problem.

They are available with different CPU configurations from
as Pete said remarkably cheap Celeron D based systems
up to Xeon CPUs.


I've just sold my Microserver Gen8 (Xeon) just recently.
It's a beautiful little machine, but I didn't make me happy in the long run.

Reasons:
- Limited to 8GB Ram in total
- Only 4 HDDs
- JBOD support is not great
- Harddrives are never going to sleep (not supported)

Installing FreeBSD was harder than expected. The machine refused to boot FreeBSD
from the internal non-raid SATA ports. I didn't try FreeNAS though.

Also, in case you go with a Microserver - I think all non-xeon models do not
support AES-NI, which will cut the throughput in case you plan to encrypt your
drives.

iX-Systems would have been my choice to replace this machine until I found out
that I can build something myself that suits my needs even better.

Note, I was looking at the 8 bay model and I would have to add around $280 for
shipping and tax to Germany.

I'm now going for this custom build:
- Fractal Design Define R5
  Reason: Silent, enough space for 12 drives
- SuperMicro MBD-A1SAI-2550F-O
  Reason: IPMI, ECC, AES-NI, 64GB Ram, Intel NIC, low power consumption
- 4x 8GB Kingson ECC SODIMM
  Reason: 16GB modules are not yet available
- Seasonic X-Series Fanless X-400FL 400W passiv
  Reason: Fanless -> silent

The complete order was around 1000€

That being said, I'm planning to put an older 3ware controller in JBOD mode in
it and I also have SSDs and drives already. The board has only 6 SATA ports.
But that would be enough for you.

This might be a little big for you. But maybe it gives you ideas.

Best Regards,
Stefan
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: recommendations for file server based zfs appliance

[Rafał Lukawiecki]

I use FreeNAS on an old Gen7 HP MicroServer. It works well but jails are too 
slow and needing an upgrade. Unfortunately, FreeBSD is *not* (yet?) supported 
on HPE MicroServer Gen10, so you couldn't run FreeNAS on it, see my bug report 
here

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=221350

and forum discussion here: 

https://forums.freebsd.org/threads/61936/

HPE MicroServer Gen8 are limited to only 16GB RAM which isn't enough for my 
needs. I will be using a SuperMicro X10SDV-6C+-TLN4F or X10SDV-4C+-TLN4F in the 
next server, but your needs (CPU) may need a higher or a lower spec, so do some 
research. 
--
Rafal Lukawiecki


> On 18 Aug 2017, at 10:40, Patrick M. Hausen  wrote:
> 
> Hi, all,
> 
>> Am 18.08.2017 um 11:19 schrieb Pete French :
>> The HP micro servers work very well, and you can pick them up remakably 
>> cheaply [...]
>> Not sure about ECC memory support there though.
> 
> They do support ECC, no problem.
> 
> They are available with different CPU configurations from
> as Pete said remarkably cheap Celeron D based systems
> up to Xeon CPUs.
> 
> If you want something that conserves power but still features
> 8 cores, Supermicro has got a small 8-core Atom based
> system:
> https://www.supermicro.nl/products/system/midtower/5028/SYS-5028A-TN4.cfm
> 
> I run this at home and I am really satisfied with it. ECC too, of course.
> Capable of running VMs in bhyve ...
> 
> I'd suggest just using FreeNAS if you intend to build a file server.
> And of course you can always order a preconfigured FreeNAS
> mini from iX Systems.
> 
> HTH,
> Patrick
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: recommendations for file server based zfs appliance

[Patrick M. Hausen]

Hi all,

> Am 18.08.2017 um 13:58 schrieb Pete French :
> 
>> Maybe the folk that made hardware suggestions can post which net
>> interface(s) they are using and whether they are seeing driver issues?
> 
> The HP boxes have Broadcom ethernet controllers driven with the 'bge'
> driver, and thatw orks fine. I stick to Intel or Broadcom controllers for
> gigabit ether where I have a choice.

The Supermicro system I mentioned features 4 Intel igb(4) interfaces.
I bridge them to save an extra switch in my home network - works great.

BTW: this Supermicro box looks suspiciously similar to at least one
generation of the FreeNAS mini ...

Patrick


signature.asc
Description: Message signed with OpenPGP

Re: recommendations for file server based zfs appliance

[Pete French]

> Maybe the folk that made hardware suggestions can post which net
> interface(s) they are using and whether they are seeing driver issues?

The HP boxes have Broadcom ethernet controllers driven with the 'bge'
driver, and thatw orks fine. I stick to Intel or Broadcom controllers for
gigabit ether where I have a choice. The only 10 gig cards we have are
some 2nd hand Mellanox ConnectX-2 cards because they are dirt cheap on
eBay. They work fine in FreeBSD using the mlxen driver (add MK_OFED=yes 
to tour /etc/src.conf) but we have had a lot of problems getting them to talk
to anthing other than other identical cards - a set of Qlogic (I think)
HP cards would work very occasionally, but most of the time would not
find a carrier. We are using twinax copper SFF+ for the 10 gig, maybe
I would have got better luck with optical, but it wasnt importnat enough
to do any serious testing on.

-pete.



___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: recommendations for file server based zfs appliance

[Rick Macklem]

Patrick M. Hausen wrote:
>Hi, all,
>
>> Am 18.08.2017 um 11:19 schrieb Pete French :
>> The HP micro servers work very well, and you can pick them up remakably 
>> cheaply [...]
>> Not sure about ECC memory support there though.
>
>They do support ECC, no problem.
>
>They are available with different CPU configurations from
>as Pete said remarkably cheap Celeron D based systems
>up to Xeon CPUs.
>
>If you want something that conserves power but still features
>8 cores, Supermicro has got a small 8-core Atom based
>system:
>https://www.supermicro.nl/products/system/midtower/5028/SYS-5028A-TN4.cfm
>
>I run this at home and I am really satisfied with it. ECC too, of course.
>Capable of running VMs in bhyve ...
>
>I'd suggest just using FreeNAS if you intend to build a file server.
>And of course you can always order a preconfigured FreeNAS
>mini from iX Systems.
I can't help with specific hardware recommendations (fortunately others already 
have),
but I will mention that "If you are going to get bit, it usually is related to 
the network
interface(s) on the machine.", due to net driver issues.

This shouldn't be a problem with an iX Systems box, since they test it with 
FreeNAS
(based on FreeBSD).

Maybe the folk that made hardware suggestions can post which net interface(s)
they are using and whether they are seeing driver issues?

Good luck with it, rick
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: recommendations for file server based zfs appliance

[Patrick M. Hausen]

Hi, all,

> Am 18.08.2017 um 11:19 schrieb Pete French :
> The HP micro servers work very well, and you can pick them up remakably 
> cheaply [...]
> Not sure about ECC memory support there though.

They do support ECC, no problem.

They are available with different CPU configurations from
as Pete said remarkably cheap Celeron D based systems
up to Xeon CPUs.

If you want something that conserves power but still features
8 cores, Supermicro has got a small 8-core Atom based
system:
https://www.supermicro.nl/products/system/midtower/5028/SYS-5028A-TN4.cfm

I run this at home and I am really satisfied with it. ECC too, of course.
Capable of running VMs in bhyve ...

I'd suggest just using FreeNAS if you intend to build a file server.
And of course you can always order a preconfigured FreeNAS
mini from iX Systems.

HTH,
Patrick


signature.asc
Description: Message signed with OpenPGP

Re: recommendations for file server based zfs appliance

[Marcelo Araujo]

2017-08-18 17:11 GMT+08:00 Sami Halabi :

> Hi all,
>
> I am planning to use zfs appliance for backup purposes, do you have
> reccomenations for small box with 4-16 gb ram ecc, and 4 hard drives
> sas/sata.
>
> Thanks in advance,
> Sami
>



Take a look at iXsystems, they have a nice FreeNAS-Mini.


Best,

-- 
Marcelo Araujo(__)ara...@freebsd.org
\\\'',)http://www.FreeBSD.org    \/  \ ^
Power To Server. .\. /_)
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: recommendations for file server based zfs appliance

[Pete French]

The HP micro servers work very well, and you can pick them up remakably 
cheaply if you look. I have six of these in various laces, all running 
ZFS and FreeBSD to perform various funcytions. Not sure about ECC memory 
support there though. Also theres only one expansion slot, which we ut 
10 gig cards into (low profile). They all used to come with ull iLo 
support too, though I understand the most recent model drops this.


-pete.

On 18/08/2017 10:11, Sami Halabi wrote:

Hi all,

I am planning to use zfs appliance for backup purposes, do you have
reccomenations for small box with 4-16 gb ram ecc, and 4 hard drives
sas/sata.

Thanks in advance,
Sami
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"



___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

recommendations for file server based zfs appliance

[Sami Halabi]

Hi all,

I am planning to use zfs appliance for backup purposes, do you have
reccomenations for small box with 4-16 gb ram ecc, and 4 hard drives
sas/sata.

Thanks in advance,
Sami
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"