Re: [zfs-discuss] Practical Application of ZFS
long live the king - Original Message - From: "Jason King" To: Sent: Wednesday, January 07, 2009 5:33 PM Subject: Re: [zfs-discuss] Practical Application of ZFS > On Wed, Jan 7, 2009 at 3:51 PM, Kees Nuyt wrote: >> On Tue, 6 Jan 2009 21:41:32 -0500, David Magda >> wrote: >> >>>On Jan 6, 2009, at 14:21, Rob wrote: >>> >>>> Obviously ZFS is ideal for large databases served out via >>>> application level or web servers. But what other practical ways are >>>> there to integrate the use of ZFS into existing setups to experience >>>> it's benefits. >>> >>>Remember that ZFS is made up of the ZPL and the DMU (amongst other >>>things). The ZPL is the POSIX compatibility layer that most of us use. >>>The DMU is the actual transactional object model that stores the >>>actual data objects (e.g. files). >>> >>>It would technically be possible for (say) MySQL to create a database >>>engine on top of that transactional store. >> >> I wouldn't be surprised to see that happen, >> given that: >> >> - InnoDB used to be the only transactional >> storage engine in MySQL >> >> - Innobase, the creator of InnoDB, has been >> acquired by Oracle >> >> - MySQL desparately needs a replacement >> for the InnoDB storage engine >> >> - MySQL has been acquired by SUN >> >> - ZFS (ZPL,DMU) is by SUN. >> >> - performance of the MySQL/InnoDB/ZFS stack is sub-optimal. >> >> No, I don't have any inside information. > > Well if you look at some of the diagrams from > http://www.opensolaris.org/os/community/zfs/docs/zfs_last.pdf it's > obvious that it's been thought of already. > > I actually thought a neat project would be to create a transactional > API that was more or less a thin layer on top of ZFS, and then create > a database using the hotspot jvm (so probably in java, but not > necessairly so) to handle the query parsing, optimization, etc. The > thought was the query could be compiled to java bytecode (and possibly > to native machine language all without having to write a native > machine language compiler). Of course it looks like derby does the > 'compile to bytecode' stuff already. But the backend userland > transactional api using ZFS might still be an interesting project if > anyone was interested. > ___ > zfs-discuss mailing list > zfs-discuss@opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Practical Application of ZFS
On Wed, Jan 7, 2009 at 3:51 PM, Kees Nuyt wrote: > On Tue, 6 Jan 2009 21:41:32 -0500, David Magda > wrote: > >>On Jan 6, 2009, at 14:21, Rob wrote: >> >>> Obviously ZFS is ideal for large databases served out via >>> application level or web servers. But what other practical ways are >>> there to integrate the use of ZFS into existing setups to experience >>> it's benefits. >> >>Remember that ZFS is made up of the ZPL and the DMU (amongst other >>things). The ZPL is the POSIX compatibility layer that most of us use. >>The DMU is the actual transactional object model that stores the >>actual data objects (e.g. files). >> >>It would technically be possible for (say) MySQL to create a database >>engine on top of that transactional store. > > I wouldn't be surprised to see that happen, > given that: > > - InnoDB used to be the only transactional > storage engine in MySQL > > - Innobase, the creator of InnoDB, has been > acquired by Oracle > > - MySQL desparately needs a replacement > for the InnoDB storage engine > > - MySQL has been acquired by SUN > > - ZFS (ZPL,DMU) is by SUN. > > - performance of the MySQL/InnoDB/ZFS stack is sub-optimal. > > No, I don't have any inside information. Well if you look at some of the diagrams from http://www.opensolaris.org/os/community/zfs/docs/zfs_last.pdf it's obvious that it's been thought of already. I actually thought a neat project would be to create a transactional API that was more or less a thin layer on top of ZFS, and then create a database using the hotspot jvm (so probably in java, but not necessairly so) to handle the query parsing, optimization, etc. The thought was the query could be compiled to java bytecode (and possibly to native machine language all without having to write a native machine language compiler). Of course it looks like derby does the 'compile to bytecode' stuff already. But the backend userland transactional api using ZFS might still be an interesting project if anyone was interested. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Practical Application of ZFS
For SuperUsers, and the little envionments, the JAVA embedded thing does all... http://java-source.net/open-source/database-engines ;-) z - Original Message - From: "Kees Nuyt" To: Sent: Wednesday, January 07, 2009 4:51 PM Subject: Re: [zfs-discuss] Practical Application of ZFS > On Tue, 6 Jan 2009 21:41:32 -0500, David Magda > wrote: > >>On Jan 6, 2009, at 14:21, Rob wrote: >> >>> Obviously ZFS is ideal for large databases served out via >>> application level or web servers. But what other practical ways are >>> there to integrate the use of ZFS into existing setups to experience >>> it's benefits. >> >>Remember that ZFS is made up of the ZPL and the DMU (amongst other >>things). The ZPL is the POSIX compatibility layer that most of us use. >>The DMU is the actual transactional object model that stores the >>actual data objects (e.g. files). >> >>It would technically be possible for (say) MySQL to create a database >>engine on top of that transactional store. > > I wouldn't be surprised to see that happen, > given that: > > - InnoDB used to be the only transactional > storage engine in MySQL > > - Innobase, the creator of InnoDB, has been > acquired by Oracle > > - MySQL desparately needs a replacement > for the InnoDB storage engine > > - MySQL has been acquired by SUN > > - ZFS (ZPL,DMU) is by SUN. > > - performance of the MySQL/InnoDB/ZFS stack is sub-optimal. > > No, I don't have any inside information. > -- > ( Kees Nuyt > ) > c[_] > ___ > zfs-discuss mailing list > zfs-discuss@opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Practical Application of ZFS
On Tue, 6 Jan 2009 21:41:32 -0500, David Magda wrote: >On Jan 6, 2009, at 14:21, Rob wrote: > >> Obviously ZFS is ideal for large databases served out via >> application level or web servers. But what other practical ways are >> there to integrate the use of ZFS into existing setups to experience >> it's benefits. > >Remember that ZFS is made up of the ZPL and the DMU (amongst other >things). The ZPL is the POSIX compatibility layer that most of us use. >The DMU is the actual transactional object model that stores the >actual data objects (e.g. files). > >It would technically be possible for (say) MySQL to create a database >engine on top of that transactional store. I wouldn't be surprised to see that happen, given that: - InnoDB used to be the only transactional storage engine in MySQL - Innobase, the creator of InnoDB, has been acquired by Oracle - MySQL desparately needs a replacement for the InnoDB storage engine - MySQL has been acquired by SUN - ZFS (ZPL,DMU) is by SUN. - performance of the MySQL/InnoDB/ZFS stack is sub-optimal. No, I don't have any inside information. -- ( Kees Nuyt ) c[_] ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Practical Application of ZFS
On Jan 6, 2009, at 14:21, Rob wrote: > Obviously ZFS is ideal for large databases served out via > application level or web servers. But what other practical ways are > there to integrate the use of ZFS into existing setups to experience > it's benefits. Remember that ZFS is made up of the ZPL and the DMU (amongst other things). The ZPL is the POSIX compatibility layer that most of us use. The DMU is the actual transactional object model that stores the actual data objects (e.g. files). It would technically be possible for (say) MySQL to create a database engine on top of that transactional store. I believe that the Lustre people are using the DMU for their future data store back end. The DMU runs in userland so anyone can use it for any object store system. People keep talking about ZFS in the context of replacing UFS/FFS, ext3, WAFL, etc., but few are utilizing (or realize the availability of) the transactional store. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Practical Application of ZFS
On Tue, Jan 6, 2009 at 2:58 PM, Rob wrote: > Wow. I will read further into this. That seems like it could have great > applications. I assume the same is true of FCoE? > -- > Yes, iSCSI, FC, FCOE all present out a LUN to Windows. For the layman, from the windows system the disk will look identical to a SCSI disk plugged directly into the motherboard. That's not entirely accurate, but close enough for you to get an idea. --Tim ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Practical Application of ZFS
Wow. I will read further into this. That seems like it could have great applications. I assume the same is true of FCoE? -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Practical Application of ZFS
On Tue, 6 Jan 2009, Rob wrote: > Are you saying that a Windows Server can access a ZFS drive via > iSCSI and store NTFS files? A volume is created under ZFS, similar to a large sequential file. The iSCSI protocol is used to export that volume as a LUN. Windows can then format it and put NTFS on it. Bob == Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Practical Application of ZFS
I am not experienced with iSCSI. I understand it's block level disk access via TCP/IP. However I don't see how using it eliminates the need for virtualization. Are you saying that a Windows Server can access a ZFS drive via iSCSI and store NTFS files? -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Practical Application of ZFS
On Tue, 6 Jan 2009, Rob wrote: > The only way I can visualize doing so would be to virtualize the > windows server and store it's image in a ZFS pool. That would add > additional overhead but protect the data at the disk level. It would > also allow snapshots of the Windows Machine's virtual file. However > none of these benefits would protect Windows from hurting it's own > data, if you catch my meaning. With OpenSolaris you can use its built in SMB/CIFS service so that files are stored natively in ZFS by the Windows client. Since the files are stored natively, individual lost/damaged files can be retrieved from a ZFS snapshot if snapshots are configured to be taken periodically. If you use iSCSI or the forthcoming COMSTAR project (iSCSI, FC target, FCOE) then you can create native Windows volumes and the whole volume could be "protected" via snapshots but without the ability to retrieve individual files. As you say, Windows could still destroy its own volume. Snapshots of iSCSI volumes will be similar to if the Windows system suddenly lost power at the time the snapshot was taken. As far as ZFS portability goes, ZFS is also supported on FreeBSD, on Linux in an inferior mode, and soon on OS-X. The main interoperability issues seem to be with the disk partitioning strategies used by the different operating systems. Bob == Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Practical Application of ZFS
Hello, - One way is virtualization, if you use a virtualization technology that uses NFS for example, you could add your virtual images on a ZFS filesystem. NFS can be used without virtualization too, but as you said the machines are windows, i don't think the NFS client for windows is production ready. Maybe somebody else on the list can say... - Virtualization inside solaris branded zones... IIRC, the idea is have branded zones to support another OS (like GNU/Linux/ MS Windows, etc). - Another option is iSCSI, and you would not need virtualization. Leal [http://www.eall.com.br/blog] > ZFS is the bomb. It's a great file system. What are > it's real world applications besides solaris > userspace? What I'd really like is to utilize the > benefits of ZFS across all the platforms we use. For > instance, we use Microsoft Windows Servers as our > primary platform here. How might I utilize ZFS to > protect that data? > > The only way I can visualize doing so would be to > virtualize the windows server and store it's image in > a ZFS pool. That would add additional overhead but > protect the data at the disk level. It would also > allow snapshots of the Windows Machine's virtual > file. However none of these benefits would protect > Windows from hurting it's own data, if you catch my > meaning. > > Obviously ZFS is ideal for large databases served out > via application level or web servers. But what other > practical ways are there to integrate the use of ZFS > into existing setups to experience it's benefits. -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
