I am still new in the freebsd zfs world so I am asking the question below kindly bear with me:Is it necessary to create a boot partition on each disk, to make it part of raidz configurationWhen you say boot pool what does it mean exactly ? you mean to say should I create separate boot pool and data pool something like zpool create bootpool raidz disk1-p1 disk2-p1 zpool create datapool raidz disk1-p3 disk2-p3 Or you mean something else.I am still not able to understand virtualbox limit of 5 disk how it is blocking me.what is your recommendation to arrange 13 disk in raidz configuration (you can avoid this question if it is going beyond )
On Thu, Feb 18, 2021 at 4:48 PM Toomas Soome <tso...@me.com> wrote: > > > On 18. Feb 2021, at 12:52, Thebest videos <sri.chityala...@gmail.com> > wrote: > > Ok, We also generated .img file using our custom OS from freebsd source. > we are uploaded img file to digital ocean images. then we are creating > droplet. everything working fine for basic operating system. but we are > facing same issue at droplet side. atleast on virtual box we are able to > create single vdev upto 5 disks and 2 vdevs with 3 disk each vdev(i mean > upto 6 disks). but digital ocean side we are unable to create atleast > single vdev with 3 disks. its working fine with 2 disks as mirror pool. we > are raised the issue on digital ocean like any restrictions on number of > disks towards the RAIDZ. but they says there is no constraints on number of > disks. we can create as RAIDZ as many number of disks. we still don't > understand where is the mistake. we also raised same query on freebsd forum > but no response. as i already shares the manual steps which we are > following to create partitions and RAIDZ configuration. are we making any > mistake from commands which we are following towards RAIDZ configuration or > as you said its king of restrictions on number of disks on virtual box and > might digital ocean side. i mean restricitons on vender side?!. Any > guesses if it works(if no mistakes from commands we are using)if we attach > CD/image to any bare metal server...?! or any suggestions? > > > > I have no personal experience with digital ocean, but the basic test is > the same; if you get loader OK prompt, use lsdev -v command to check how > many disks you can actually see. There actually is another option too — > with BIOS boot, when you see the very first spinner, press space key and > you will get boot: prompt. This is very limited but still useful prompt > from gptzfsboot proagram (the one which will try to find and start > /boot/loader). On boot: prompt, you can enter: status — this will produce > the same report as you get from lsdev. > > So, if you know your VM should have, say, 10 disks, but boot: status or ok > lsdev will show less, then you know, there must be BIOS limit (we do use > BIOS INT13h to access the disks). > > Please note, if the provider does offer option to use UEFI, it *may* > support greater number of boot disks, the same check does apply with UEFI > as well (lsdev -v). > > rgds, > toomas > > > These are commands we are using to create partitions and RAIDZ > configuration > NOTE: we are creating below gpart partitions(boot,swap,root) on all hard > disks then adding those hard disks in zpool command > Doubt: should we create partitions(boot,swap,root) on all hard disks to > make part of RAIDZ configuration or is it enough to add in zpool as raw > disks or making 2-3 disks as bootable then remaining as raw disks? anyway > please check below commands we are using to create partitions and zpool > configurations > gpart create -s gpt /dev/da0 > gpart add -a 4k -s 512K -t freebsd-boot da0 > gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 da0 > gpart add -a 1m -s 2G -t freebsd-swap -l swap1 da0 > gpart add -a 1m -t freebsd-zfs -l disk1 da0 > zpool create -o altroot=/mnt datapool raidz2 ada0p3 ada1p3 ada2p3 ada3p3 > raidz2 ada4p3 ada5p3 ada6p3 ada7p3 > zfs create -o mountpoint=/ -o canmount=noauto datapool/boot > mount -t zfs datapool/boot /mnt > cp -r /temp/* /mnt/. > zpool set bootfs=datapool/boot datapool > zfs create -o mountpoint=/storage -o canmount=noauto datapool/storage > zfs create -o mountpoint=/conf -o canmount=noauto datapool/conf > shutdown and remove iso/img and start it again > zpool import datapool > mkdir /conf /storage > mount -t zfs datapool/conf /conf > mount -t zfs datapool/storage /storage > > > > On Thu, Feb 18, 2021 at 3:33 PM Toomas Soome <tso...@me.com> wrote: > >> >> >> On 18. Feb 2021, at 11:52, Thebest videos <sri.chityala...@gmail.com> >> wrote: >> >> as per your reply, im not clear >> although i've tried to create 2 pools with 4 disks(for testing purpose) >> each pool in a single vdev as expected it works. but that is not our >> requirement since we intended to chose single pool as many number of disks >> which should part of multiple vdev's based of condition(max 5 disks each >> vdev) and any disks left after part of vdev should act as spare disks. >> finally max 5 disks are coming ONLINE in vdev remaining disks going as >> says OFFLINE state disk state is UNKNOWN. is there anyway to fix this issue. >> >> >> >> If you want to use virtualbox, then there is limit that virtualbox does >> only see first 5 disk devices. This is vbox limit and there are only two >> options about it - either accept it or to file feature request to >> virtualbox developers. >> >> Different systems can set different limits there, for example, VMware >> Fusion does support booting from first 12 disks. It also can have more >> disks than 12, but only first 12 are visible for boot loader. >> >> Real hardware is vendor specific. >> >> rgds, >> toomas >> >> >> On Thu, Feb 18, 2021 at 1:24 AM Toomas Soome via openindiana-discuss < >> openindiana-discuss@openindiana.org> wrote: >> >>> >>> >>> > On 17. Feb 2021, at 20:49, Thebest videos <sri.chityala...@gmail.com> >>> wrote: >>> > >>> > NOTE: we are getting issues after shutdown , then remove ISO file from >>> > virtualBox then power on the server. if we attach an ISO file we are >>> safe >>> > with our Zpool stuff. and we are creating boot,swap,root partitions on >>> each >>> > disks. >>> >>> vbox seems to have limit on boot disks - it appears to “see” 5, My vbox >>> has IDE for boot disk, and I did add 6 sas disks, I only can see 5 — ide + >>> 4 sas. >>> >>> So all you need to do is to add disk for boot pool, and make sure it is >>> first one - once kernel is up, it can see all the disks. >>> >>> rgds, >>> toomas >>> >>> >>> > I'm not able to understand First 5 disks are ONLINE and remaining >>> disks are >>> > UNKNOWN state after power off and then power on >>> > actually our requirement is to create RAIDZ1/RAIDZ2 with single >>> vdev(upto 5 >>> > disks per vdev) if more than 5 or less than 10 disks then those >>> disks(after >>> > 5disks) are spare part shouldn't be included any vdev. if we have >>> > multiple's of 5 disks then we need to create multiple vdev in a pool >>> > example: RAIDZ2 : if total 7 disks then 5 disks as single vdev, >>> remaining 2 >>> > disks as spare parts nothing to do. and if we have 12 disks intotal >>> then 2 >>> > vdevs (5 disks per vdev) so total 10 disks in 2 vdevs remaining 2disks >>> as >>> > spare. >>> > RAIDZ1: if we have only 3 disks then we should create RAIDZ1 >>> > >>> > Here, we wrote a zfs script for our requirements(but currently testing >>> with >>> > manual commands). We are able to createRAIDZ2 with a single vdev in a >>> pool >>> > for 5 disks. it works upto 9 disks but if we have 10 disks then 2 >>> vdevs are >>> > created after power on the same error coming like zfs: i/o error all >>> copies >>> > blocked. >>> > I was testing the RAIDZ like I'm creating 2 vdevs which have 3 disks >>> per >>> > each vdev.its working fine even after shutdown and power on(as says >>> that we >>> > are removing the ISO file after shutdown). >>> > but the issue is when we create 2 vdevs with 4 disks per each vdev.this >>> > time we are not getting error its giving options like we press esc >>> button >>> > what kind of options we see those options are coming. if i type lsdev >>> -v(as >>> > you said before). first 5 disks are online and the remaining 3 disks >>> are >>> > UNKNOWN. >>> > >>> > FInally, I need to setup RAIDZ configuration with 5 multiples of disks >>> per >>> > each vdev. please look once again below commands im using to create >>> > partitions and RAIDZ configuration >>> > >>> > NOTE: below gpart commands are running for each disk >>> > >>> > gpart create -s gpt ada0 >>> > >>> > >>> > gpart add -a 4k -s 512K -t freebsd-boot ada0 >>> > >>> > >>> > gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ada0 >>> > >>> > >>> > gpart add -a 1m -s 2G -t freebsd-swap -l swap0 ada0 >>> > >>> > >>> > gpart add -a 1m -t freebsd-zfs -l disk0 ada0 >>> > >>> > >>> > zpool create -o altroot=/mnt datapool raidz2 ada0p3 ada1p3 ada2p3 >>> > ada3p3 raidz2 ada4p3 ada5p3 ada6p3 ada7p3 >>> > >>> > >>> > zfs create -o mountpoint=/ -o canmount=noauto datapool/boot >>> > >>> > >>> > mount -t zfs datapool/boot /mnt >>> > >>> > >>> > mount_cd9660 /dev/cd0 /media >>> > >>> > >>> > cp -r /media/* /mnt/. >>> > >>> > >>> > zpool set bootfs=datapool/boot datapool >>> > >>> > >>> > shutdown and remove ISO and power on the server >>> > >>> > >>> > kindly suggest me steps if im wrong >>> > >>> > On Wed, Feb 17, 2021 at 11:51 PM Thebest videos < >>> sri.chityala...@gmail.com> >>> > wrote: >>> > >>> >> prtconf -v | grep biosdev not working on freebsd >>> >> i think its legacy boot system(im not sure actually i didnt find >>> anything >>> >> about EFI related stuff) is there anyway to check EFI >>> >> >>> >> Create the pool with EFI boot: >>> >> # zpool create -B rpool raidz c0t0d0 c0t1d0 c0t3d0 >>> >> >>> >> how can i create pool with EFI >>> >> and -B what it refers? >>> >> >>> >> On Wed, Feb 17, 2021 at 11:00 PM John D Groenveld <groenv...@acm.org> >>> >> wrote: >>> >> >>> >>> In message <272389262.2537371.1613575739...@mail.yahoo.com>, >>> Reginald >>> >>> Beardsley >>> >>> via openindiana-discuss writes: >>> >>>> I was not aware that it was possible to boot from RAIDZ. It wasn't >>> >>> possible wh >>> >>> >>> >>> With the current text installer, escape to a shell. >>> >>> Confirm the disks are all BIOS accessible: >>> >>> # prtconf -v | grep biosdev >>> >>> Create the pool with EFI boot: >>> >>> # zpool create -B rpool raidz c0t0d0 c0t1d0 c0t3d0 >>> >>> Exit and return to the installer and then F5 Install to an Existing >>> Pool >>> >>> >>> >>> John >>> >>> groenv...@acm.org >>> >>> >>> >>> _______________________________________________ >>> >>> openindiana-discuss mailing list >>> >>> openindiana-discuss@openindiana.org >>> >>> https://openindiana.org/mailman/listinfo/openindiana-discuss >>> >>> >>> >> >>> > _______________________________________________ >>> > openindiana-discuss mailing list >>> > openindiana-discuss@openindiana.org >>> > https://openindiana.org/mailman/listinfo/openindiana-discuss >>> >>> >>> _______________________________________________ >>> openindiana-discuss mailing list >>> openindiana-discuss@openindiana.org >>> https://openindiana.org/mailman/listinfo/openindiana-discuss >>> >> <Screenshot 2021-02-18 at 12.38.35 PM.png> >> >> >> > _______________________________________________ openindiana-discuss mailing list openindiana-discuss@openindiana.org https://openindiana.org/mailman/listinfo/openindiana-discuss