Re: ZFS with 32-bit, non-x86 kernel
On Fri, Oct 04, 2019 at 09:53:07PM +0200, Marek Zarychta wrote: > On 04.10.2019 21:37, Ian Lepore wrote: > > On Fri, 2019-10-04 at 13:27 -0600, Warner Losh wrote: > >> On Fri, Oct 4, 2019, 1:07 PM Dennis Clarke wrote: > >> > >>> On 10/4/19 10:05 AM, Andriy Gapon wrote: > > Does anyone use ZFS with a 32-bit kernel, that is also not i386 ? > If you do, could you please let me know? Along with uname -rmp output. > Thank you! > > >>> > >>> I don't know if that has even been attempted by anyone. The ZIL and ZFS > >>> log comonents require substantial amounts of memory and I am not aware > >>> of anyone with arm devices that have 8GB+ of memory. I have had FreeBSD > >>> current on RISC-V running fairly well with ZFS however that was a purely > >>> rv64imafdc architecture. > >>> > >> > >> In the FreeBSD 10 time frame I know people were running ZFS on arm7 boards. > >> Iirc, there was a long list of tweaks needed to size of the ZIL. A quick > >> google didn't find it. > >> > >> Otoh, I looked at ZFS for NanoBSD when it first came out. I gave up because > >> the 256MB boards at the time made any kind of storage traffic ran things > >> out of memory. > >> > >> Warner > >> > >> > >> I will watch this thread with curiosity. > > > > There have been several threads about using zfs on armv7 over the > > years. Some of them are from 2013 and indicate little sucess. Others, > > from 2015, indicate it works... > > > > https://lists.freebsd.org/pipermail/freebsd-arm/2015-March/010607.html > > https://lists.freebsd.org/pipermail/freebsd-arm/2015-March/010649.html > > > > There have also been some bug reports as recently as 2017 indicating > > that people are still doing this on small armv7 systems. > > > > -- Ian > > Following this thread, where Bernd Walter wrote small howto: > > https://lists.freebsd.org/pipermail/freebsd-arm/2019-February/019455.html > > I had converted root filesystem to ZFS on SD card used with > RaspberryPi2, then used it with no issues running 13-CURRENT for 6 > months until that old SD card got worn. Yes, a system with 1G RAM works fine. I use it mostly on 64 bit systems, like Pi3, Pine64, Pinebook, ... All of them are 1G-2G RAM. But I also have a lot of 2GB Wandboards, which are 32bit, have two uSD slots and work great. I also have some 1GB Allwinner A20 boards with 1GB RAM and two uSD slots on which I might do it as well to give those boards a purpose. SD cards are notorious for problems after power failure. ZFS works great with flash based media and can handle such media errors just fine. I'm running two wandboards in such a zroot mirror setup to programm microcontrollers with avrdude, openocd, run TTL-UART, ... A lot of missuse and since they are running headless I often just powercycle them if something with USB hangs again. I also found out that reversing an A-Plug can produce a short circuit on the host 5V rail and zroot survided those spontanous reboots just fine. Would be a shame if I couldn't use the wandboards anymore. -- B.Walter http://www.bwct.de Modbus/TCP Ethernet I/O Baugruppen, ARM basierte FreeBSD Rechner uvm. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: ZFS with 32-bit, non-x86 kernel
On Fri, Oct 04, 2019 at 05:05:25PM +0300, Andriy Gapon wrote:
>
> Does anyone use ZFS with a 32-bit kernel, that is also not i386 ?
> If you do, could you please let me know? Along with uname -rmp output.
> Thank you!
[51]wb1# uname -rmp
12.0-RELEASE arm armv7
It is a wandboard qaud with iMX6 and 2G RAM.
I'm using two uSD cards with zroot.
I'm also using the same setup on a raspberry Pi1:
[13]time1# uname -rmp
12.0-RELEASE arm armv6
But 512MB RAM are to low for zroot.
It technically does work, but hoks up the CPU in arc_reclaim_thread when
a scrub runs and since it takes forever it always runs.
last pid: 80786; load averages: 6.56, 5.86, 5.75 up 185+05:51:55 11:10:04
372 threads: 5 running, 349 sleeping, 18 waiting
CPU: 0.1% user, 0.0% nice, 86.9% system, 3.8% interrupt, 9.2% idle
Mem: 4960K Active, 38M Inact, 128M Wired, 259M Free
ARC: 24M Total, 8591K MFU, 8896K MRU, 34K Anon, 251K Header, 6888K Other
2676K Compressed, 74M Uncompressed, 28.31:1 Ratio
Swap:
PID USERNAMEPRI NICE SIZERES STATETIMEWCPU COMMAND
8 root -8- 096K arc_re 1597.2 53.94%
zfskern{arc_reclaim_thread}
10 root155 ki31 0 8192 RUN1765.3 9.26% idle
0 root -8- 0 2064K - 75.7H 2.63%
kernel{dp_sync_taskq}
11 root-80- 0 144K WAIT67.8H 1.88% intr{intc0,28:
bcm_dma0}
20 root -8- 0 8192 mmcreq 39.0H 1.31% mmcsd0: mmc/sd
card
12 root -8- 024K - 27.4H 0.87% geom{g_down}
11 root-60- 0 144K WAIT28.6H 0.74% intr{swi4: clock
(0)}
11 root-88- 0 144K WAIT21.1H 0.66% intr{intc0,70: +}
12 root -8- 024K - 19.3H 0.64% geom{g_up}
[16]time1# zpool status
pool: zroot
state: ONLINE
scan: scrub in progress since Mon Jun 10 03:58:19 2019
34.0G scanned at 3.52K/s, 867M issued at 89/s, 1.77G total
0 repaired, 47.78% done, no estimated completion time
config:
NAME STATE READ WRITE CKSUM
zrootONLINE 0 0 0
mirror-0 ONLINE 0 0 0
diskid/DISK-081Cs2a ONLINE 0 0 0
da0s2a ONLINE 0 0 0
errors: No known data errors
--
B.Walter http://www.bwct.de
Modbus/TCP Ethernet I/O Baugruppen, ARM basierte FreeBSD Rechner uvm.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: ZFS with 32-bit, non-x86 kernel
On 04/10/2019 22:24, Justin Hibbits wrote: > On Fri, 4 Oct 2019 15:06:52 -0400 > Dennis Clarke wrote: > >> On 10/4/19 10:05 AM, Andriy Gapon wrote: >>> >>> Does anyone use ZFS with a 32-bit kernel, that is also not i386 ? >>> If you do, could you please let me know? Along with uname -rmp >>> output. Thank you! >>> >> >> I don't know if that has even been attempted by anyone. The ZIL and >> ZFS log comonents require substantial amounts of memory and I am not >> aware of anyone with arm devices that have 8GB+ of memory. I have had >> FreeBSD current on RISC-V running fairly well with ZFS however that >> was a purely rv64imafdc architecture. >> >> I will watch this thread with curiosity. >> >> > > I did try using ZFS on 32-bit powerpc (8GB RAM), and even got a bugfix > pushed into the ZFS/Illumos repo for it, but it was too unstable to be > usable. I'd love to try again later though. The instability was not surprising, IMO. ZFS depends on 64-bit atomic operations. For platforms that do not provide them there is an attempt to emulate them, but it is not quite consistent. In particular, reads are just plain reads on all platforms, so torn values can be seen on all 32-bit platforms. But it's easier to fix those of them that have 64-bit operations. I am thinking about proposing to remove ZFS support from platforms that do not provide 64-bit atomics for the kernel. I think that those are 32-bit powerpc-s and some flavors of 32-bit mips, but I am quite confused about the latter. There are so many of them: mips.mips, mipsel, mipshf, mipselhf, mipsn32. -- Andriy Gapon ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: ZFS with 32-bit, non-x86 kernel
In message , Ian Le pore writes: >There have also been some bug reports as recently as 2017 indicating >that people are still doing this on small armv7 systems. I actually have a potential off-site backup server in my lab right now, consisting of a BeagleBoneBlack and two USB disks, seems to work. The basic scheme is a cronjob which: zfs import inl run various rsyncs zfs snapshot -r inl@$YYMMDDHHMM zfs export inl The import/export is so the USB disks spin down. Not sure if ZFS will croak the 512M RAM on other workloads, but for this one it seems to work fine so far. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 p...@freebsd.org | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: ZFS with 32-bit, non-x86 kernel
On 04.10.2019 21:37, Ian Lepore wrote: > On Fri, 2019-10-04 at 13:27 -0600, Warner Losh wrote: >> On Fri, Oct 4, 2019, 1:07 PM Dennis Clarke wrote: >> >>> On 10/4/19 10:05 AM, Andriy Gapon wrote: Does anyone use ZFS with a 32-bit kernel, that is also not i386 ? If you do, could you please let me know? Along with uname -rmp output. Thank you! >>> >>> I don't know if that has even been attempted by anyone. The ZIL and ZFS >>> log comonents require substantial amounts of memory and I am not aware >>> of anyone with arm devices that have 8GB+ of memory. I have had FreeBSD >>> current on RISC-V running fairly well with ZFS however that was a purely >>> rv64imafdc architecture. >>> >> >> In the FreeBSD 10 time frame I know people were running ZFS on arm7 boards. >> Iirc, there was a long list of tweaks needed to size of the ZIL. A quick >> google didn't find it. >> >> Otoh, I looked at ZFS for NanoBSD when it first came out. I gave up because >> the 256MB boards at the time made any kind of storage traffic ran things >> out of memory. >> >> Warner >> >> >> I will watch this thread with curiosity. > > There have been several threads about using zfs on armv7 over the > years. Some of them are from 2013 and indicate little sucess. Others, > from 2015, indicate it works... > > https://lists.freebsd.org/pipermail/freebsd-arm/2015-March/010607.html > https://lists.freebsd.org/pipermail/freebsd-arm/2015-March/010649.html > > There have also been some bug reports as recently as 2017 indicating > that people are still doing this on small armv7 systems. > > -- Ian Following this thread, where Bernd Walter wrote small howto: https://lists.freebsd.org/pipermail/freebsd-arm/2019-February/019455.html I had converted root filesystem to ZFS on SD card used with RaspberryPi2, then used it with no issues running 13-CURRENT for 6 months until that old SD card got worn. -- Marek Zarychta signature.asc Description: OpenPGP digital signature
Re: ZFS with 32-bit, non-x86 kernel
On Fri, 2019-10-04 at 13:27 -0600, Warner Losh wrote: > On Fri, Oct 4, 2019, 1:07 PM Dennis Clarke wrote: > > > On 10/4/19 10:05 AM, Andriy Gapon wrote: > > > > > > Does anyone use ZFS with a 32-bit kernel, that is also not i386 ? > > > If you do, could you please let me know? Along with uname -rmp output. > > > Thank you! > > > > > > > I don't know if that has even been attempted by anyone. The ZIL and ZFS > > log comonents require substantial amounts of memory and I am not aware > > of anyone with arm devices that have 8GB+ of memory. I have had FreeBSD > > current on RISC-V running fairly well with ZFS however that was a purely > > rv64imafdc architecture. > > > > In the FreeBSD 10 time frame I know people were running ZFS on arm7 boards. > Iirc, there was a long list of tweaks needed to size of the ZIL. A quick > google didn't find it. > > Otoh, I looked at ZFS for NanoBSD when it first came out. I gave up because > the 256MB boards at the time made any kind of storage traffic ran things > out of memory. > > Warner > > > I will watch this thread with curiosity. There have been several threads about using zfs on armv7 over the years. Some of them are from 2013 and indicate little sucess. Others, from 2015, indicate it works... https://lists.freebsd.org/pipermail/freebsd-arm/2015-March/010607.html https://lists.freebsd.org/pipermail/freebsd-arm/2015-March/010649.html There have also been some bug reports as recently as 2017 indicating that people are still doing this on small armv7 systems. -- Ian ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: ZFS with 32-bit, non-x86 kernel
On Fri, Oct 4, 2019, 1:07 PM Dennis Clarke wrote: > On 10/4/19 10:05 AM, Andriy Gapon wrote: > > > > Does anyone use ZFS with a 32-bit kernel, that is also not i386 ? > > If you do, could you please let me know? Along with uname -rmp output. > > Thank you! > > > > I don't know if that has even been attempted by anyone. The ZIL and ZFS > log comonents require substantial amounts of memory and I am not aware > of anyone with arm devices that have 8GB+ of memory. I have had FreeBSD > current on RISC-V running fairly well with ZFS however that was a purely > rv64imafdc architecture. > In the FreeBSD 10 time frame I know people were running ZFS on arm7 boards. Iirc, there was a long list of tweaks needed to size of the ZIL. A quick google didn't find it. Otoh, I looked at ZFS for NanoBSD when it first came out. I gave up because the 256MB boards at the time made any kind of storage traffic ran things out of memory. Warner I will watch this thread with curiosity. > > > -- > Dennis Clarke > RISC-V/SPARC/PPC/ARM/CISC > UNIX and Linux spoken > GreyBeard and suspenders optional > ___ > freebsd-current@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org" > ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: ZFS with 32-bit, non-x86 kernel
On Fri, 4 Oct 2019 15:06:52 -0400 Dennis Clarke wrote: > On 10/4/19 10:05 AM, Andriy Gapon wrote: > > > > Does anyone use ZFS with a 32-bit kernel, that is also not i386 ? > > If you do, could you please let me know? Along with uname -rmp > > output. Thank you! > > > > I don't know if that has even been attempted by anyone. The ZIL and > ZFS log comonents require substantial amounts of memory and I am not > aware of anyone with arm devices that have 8GB+ of memory. I have had > FreeBSD current on RISC-V running fairly well with ZFS however that > was a purely rv64imafdc architecture. > > I will watch this thread with curiosity. > > I did try using ZFS on 32-bit powerpc (8GB RAM), and even got a bugfix pushed into the ZFS/Illumos repo for it, but it was too unstable to be usable. I'd love to try again later though. - Justin ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: ZFS with 32-bit, non-x86 kernel
On 10/4/19 10:05 AM, Andriy Gapon wrote: Does anyone use ZFS with a 32-bit kernel, that is also not i386 ? If you do, could you please let me know? Along with uname -rmp output. Thank you! I don't know if that has even been attempted by anyone. The ZIL and ZFS log comonents require substantial amounts of memory and I am not aware of anyone with arm devices that have 8GB+ of memory. I have had FreeBSD current on RISC-V running fairly well with ZFS however that was a purely rv64imafdc architecture. I will watch this thread with curiosity. -- Dennis Clarke RISC-V/SPARC/PPC/ARM/CISC UNIX and Linux spoken GreyBeard and suspenders optional ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
ZFS with 32-bit, non-x86 kernel
Does anyone use ZFS with a 32-bit kernel, that is also not i386 ? If you do, could you please let me know? Along with uname -rmp output. Thank you! -- Andriy Gapon ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
