Re: [zfs-discuss] Encryption?
On Sun, Jul 11, 2010 at 11:51 AM, Michael Johnson mjjohnson@yahoo.com wrote: I'm planning on running FreeBSD in VirtualBox (with a Linux host) and giving it raw disk access to four drives, which I plan to configure as a raidz2 volume. On top of that, I'm considering using encryption. I understand that ZFS doesn't yet natively support encryption, so my idea was to set each drive up with full-disk encryption in the Linux host (e.g., using TrueCrypt or dmcrypt), mount the encrypted drives, and then give the virtual machine access to the virtual unencrypted drives. So the encryption would be transparent to FreeBSD. However, I don't know enough about ZFS to know if this is a good idea. I know that I need to specifically configure VirtualBox to respect cache flushes, so that data really is on disk when ZFS expects it to be. Would putting ZFS on top of full-disk encryption like this cause any problems? E.g., if the (encrypted) physical disk has a problem and as a result a larger chunk of the unencrypted data is corrupted, would ZFS handle that well? Are there any other possible consequences of this idea that I should know about? (I'm not too worried about any hits in performance; I won't be reading or writing heavily, nor in time-sensitive applications.) I should add that since this is a desktop I'm not nearly as worried about encryption as if it were a laptop (theft or loss are less likely), but encryption would still be nice. However, data integrity is the most important thing (I'm storing backups of my personal files on this), so if there's a chance that ZFS wouldn't handle errors well when on top of encryption, I'll just go without it. Thanks, Michael you can also create zfs on top of GELI[1][2] devices. Create the encrypted disks first and then use that to create zpool. Exact steps (assuming single disk, da1): - create the key # dd if=/dev/random of=/root/da1.key bs=64 count=1 - initialize GELI disk, if you want to only use the key as authentication method or automatically attach on boot, check the reference links for initialization and configuration (-K and -b) # geli init -s 4096 -K da1.key /dev/da1 - attach GELI disk # geli attach -k da1.key /dev/da1 - create zpool, either directly on geli disk or by creating it on top of GPT direct: # zpool create securepool da1.eli on top of GPT: # gpart create -s gpt da1.eli # gpart add -t freebsd-zfs da1.eli # zpool create securepool da1.elip1 - adjust rc.conf and loader.conf accordingly Another tutorial: http://forums.freebsd.org/showthread.php?t=2775 [1] http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/disks-encrypting.html [2] http://www.freebsd.org/cgi/man.cgi?query=geliapropos=0sektion=0manpath=FreeBSD+8.0-RELEASEformat=html -- O ascii ribbon campaign - stop html mail - www.asciiribbon.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Should i enable Write-Cache ?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Le 09/07/2010 01:37, Edward Ned Harvey a écrit : From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Philippe Schwarz . But if you're impressed with performance by enabling writeback, you can still do better ... The most effective thing you could possibly do is to disable the writeback, and add SSD for log device. ZFS is able to perform in this configuration, better than the WriteBack. And in this situation, surprisingly, enabling the WriteBack actually hurts performance slightly. . Hi, i bought a little SSD (OCZ Agility 30GB) and added half to L2ARC and second half to ZIL: zpool add zfsda1 log da3s2 zpool add zfsda1 cache da3s1 zpool status pool: zfsda1 state: ONLINE scrub: none requested config: NAMESTATE READ WRITE CKSUM zfsda1 ONLINE 0 0 0 da1 ONLINE 0 0 0 logsONLINE 0 0 0 da3s2 ONLINE 0 0 0 cache da3s1 ONLINE 0 0 0 errors: No known data errors OK, let's try to burst the write (WC disabled on the Raid controller) ... Result is ...awful! A `zpool iostat -v 1` shows: - - Although the L2ARC (da3s1) is showed separately from the pool, the ZIL (da3s2) is shown within the pool. Is it the normal behaviour ? - - ZIL seems to be quiet almost all the time and burst sometimes. OK, i may the normal behaviour of a cache. capacity operationsbandwidth pool used avail read write read write - -- - - - - - - zfsda1 2.36G 694G 0176 0 21.8M da1 2.36G 694G 0 56 0 6.88M da3s2 128K 15.0G 0119 0 15.0M cache - - - - - - da3s1 3.06G 11.7G 0 0 0 0 - -- - - - - - - But, at the end of the copy process (copy a 1GB file from to the same pool), the used capacity of the ZIL remains unchanged... Puzzling.. - - And ,last but not least... the copy isn't faster at all! - -- Without ZILL2ARC time cp /zfsda1/rnd /zfsda1/rn2 real3m23.297s - -- With ZILL2ARC time cp /zfsda1/rnd /zfsda1/rn2 real3m34.847s Should i call my (dummy) test into question ? Thanks. Best regards. - -- Lycée Maximilien Perret, Alfortville -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkw5m0UACgkQlhqCFkbqHRZf1ACgkMIz6gf+H4bXfK5GH1HkWwag WPgAn2H6/j344LdFEOiig3MAxEy68yG2 =Mjn9 -END PGP SIGNATURE- ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Should i enable Write-Cache ?
On 7/11/2010 3:21 AM, Philippe Schwarz wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, i bought a little SSD (OCZ Agility 30GB) and added half to L2ARC and second half to ZIL: zpool add zfsda1 log da3s2 zpool add zfsda1 cache da3s1 zpool status pool: zfsda1 state: ONLINE scrub: none requested config: NAMESTATE READ WRITE CKSUM zfsda1 ONLINE 0 0 0 da1 ONLINE 0 0 0 logsONLINE 0 0 0 da3s2 ONLINE 0 0 0 cache da3s1 ONLINE 0 0 0 errors: No known data errors OK, let's try to burst the write (WC disabled on the Raid controller) ... Result is ...awful! A `zpool iostat -v 1` shows: - - Although the L2ARC (da3s1) is showed separately from the pool, the ZIL (da3s2) is shown within the pool. Is it the normal behaviour ? Yes, it's just a quirk of the output format. - - ZIL seems to be quiet almost all the time and burst sometimes. OK, i may the normal behaviour of a cache. capacity operationsbandwidth pool used avail read write read write - -- - - - - - - zfsda1 2.36G 694G 0176 0 21.8M da1 2.36G 694G 0 56 0 6.88M da3s2 128K 15.0G 0119 0 15.0M cache - - - - - - da3s1 3.06G 11.7G 0 0 0 0 - -- - - - - - - But, at the end of the copy process (copy a 1GB file from to the same pool), the used capacity of the ZIL remains unchanged... Puzzling.. - - And ,last but not least... the copy isn't faster at all! - -- Without ZILL2ARC time cp /zfsda1/rnd /zfsda1/rn2 real3m23.297s - -- With ZILL2ARC time cp /zfsda1/rnd /zfsda1/rn2 real3m34.847s Should i call my (dummy) test into question ? Thanks. Best regards. ZIL speeds up synchronous writes only. Operations like 'cp' use async writes, so ZIL will be of no benefit, since it's not being used. -- Erik Trimble Java System Support Mailstop: usca22-123 Phone: x17195 Santa Clara, CA ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Encryption?
I'm planning on running FreeBSD in VirtualBox (with a Linux host) and giving it raw disk access to four drives, which I plan to configure as a raidz2 volume. Wouldn't it be better or just as good to use fuse-zfs for such a configuration? I/O from VirtualBox isn't really very good, but then, I haven't tested the linux/fbsd configuration... Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Legality and the future of zfs...
On Jul 10, 2010, at 14:20, Edward Ned Harvey wrote: A few companies have already backed out of zfs as they cannot afford to go through a lawsuit. Or, in the case of Apple, who could definitely afford a lawsuit, but choose to avoid it anyway. This was covered already: http://mail.opensolaris.org/pipermail/zfs-discuss/2009-October/033125.html ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Should i enable Write-Cache ?
On Sun, 11 Jul 2010, Philippe Schwarz wrote: But, at the end of the copy process (copy a 1GB file from to the same pool), the used capacity of the ZIL remains unchanged... Puzzling.. - - And ,last but not least... the copy isn't faster at all! Note that the slog device is only used for synchronous writes, and a local file copy is not normally going to use synchronous writes. Also, even if the slog was used, it gets emptied pretty quickly. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Encryption?
On Sun, Jul 11, 2010 at 4:21 AM, Roy Sigurd Karlsbakk r...@karlsbakk.net wrote: I'm planning on running FreeBSD in VirtualBox (with a Linux host) and giving it raw disk access to four drives, which I plan to configure as a raidz2 volume. Wouldn't it be better or just as good to use fuse-zfs for such a configuration? I/O from VirtualBox isn't really very good, but then, I haven't tested the linux/fbsd configuration... ZFS-FUSE is horribly unstable, although that's more an indication of the stability of the storage stack on Linux. We've been testing it at work to see how dedupe support will affect our FreeBSD+ZFS storage servers. We can't keep it (Linux+ZFS) running for more than a few days. Drives drop off at random, the pool locks up, resilvers never complete. When it does work, it works nicely. It's just hard to keep it running. You definitely want to do the ZFS bits from within FreeBSD. -- Freddie Cash fjwc...@gmail.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Legality and the future of zfs...
On Sat, Jul 10, 2010 at 1:20 PM, Edward Ned Harvey solar...@nedharvey.comwrote: From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Peter Taps A few companies have already backed out of zfs as they cannot afford to go through a lawsuit. Or, in the case of Apple, who could definitely afford a lawsuit, but choose to avoid it anyway. I am in a stealth startup company and we rely on zfs for our application. The future of our company, and many other businesses, depends on what happens to zfs. For a lot of purposes, ZFS is the clear best solution. But maybe you're not necessarily in one of those situations? Perhaps you could use Microsoft VSS, or Linux BTRFS? 'Course, by all rights, those are copy-on-write too. So why doesn't netapp have a lawsuit against kernel.org, or microsoft? Maybe cuz they just know they'll damage their own business too much by suing Linus, and they can't afford to go up against MS. I guess. Because VSS isn't doing anything remotely close to what WAFL is doing when it takes snapshots. I haven't spent much time looking at the exact BTRFS implementation, but I'd imagine the fact its on-disk format isn't finalized (last I heard) would make it a bit pre-mature to file a lawsuit. I'm sure they're actively watching it as well. Furthermore, I'm sure the fact one of the core zfs developers, Matt Ahrens, previously interned for the filesystem group at NetApp had just a *BIT* to do with the lawsuit. From their perspective, it's just a bit too convenient someone gets access to the crown jewels, then runs off to a new company and creates a filesystem that looks and feels so similar. Of course, taking stabs in the dark on this mailing list without having access to all of the court documents isn't really constructive in the first place. Then again, neither are people trying to claim they have a solid understanding of the validity of the lawsuit(s), on this mailing list, who aren't IP lawyers. --Tim ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Encryption?
On Jul 11, 2010, at 5:11 PM, Freddie Cash fjwc...@gmail.com wrote: ZFS-FUSE is horribly unstable, although that's more an indication of the stability of the storage stack on Linux. Not really, more an indication of the pseudo-VFS layer implemented in fuse. Remember fuse provides it's own VFS API separate from the Linux VFS API so file systems can be implemented in user space. Fuse needs a little more work to handle ZFS as a file system. -Ross ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Encryption?
on 11/07/2010 15:54 Andriy Gapon said the following: on 11/07/2010 14:21 Roy Sigurd Karlsbakk said the following: I'm planning on running FreeBSD in VirtualBox (with a Linux host) and giving it raw disk access to four drives, which I plan to configure as a raidz2 volume. Wouldn't it be better or just as good to use fuse-zfs for such a configuration? I/O from VirtualBox isn't really very good, but then, I haven't tested the linux/fbsd configuration... Like Freddie already mentioned, I'd heard that fuse-zfs wasn't really all that good of an option, and I wanted something that was more stable/reliable. Hmm, an unexpected question IMHO - wouldn't it better to just install FreeBSD on the hardware? :-) If an original poster is using Linux as a host OS, then probably he has some very good reason to do that. But performance and etc -wise, directly using FreeBSD, of course, should win over fuse-zfs. Right? [Installing and maintaining one OS instead of two is the first thing that comes to mind] I'm going with a virtual machine because the box I ended up building for this was way more powerful than I needed for just my file server; thus, I figured I'd use it as a personal machine too. (I wanted ECC RAM, and there just aren't that many motherboards that support ECC RAM that are also really cheap and low-powered.) And since I'm much more comfortable with Linux, I wanted to use it for the personal side of things. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
