Re: [Zope3-dev] future of fssync (was: RE: [SpringCleaning07])
Jim Fulton <[EMAIL PROTECTED]> on Mittwoch, 24. Januar 2007 at 16:48 Uhr +0100 wrote: > >Maybe, but I like your idea of using utilities below. My original >thinking was >along these lines: fssync should strive to serialize "all" object >data. (Note that it isn't always obvious what data is intrinsic to >an object.) I felt therefore, that inheriting synchronization >adapters would be bad. If someone created a subclass and forgot to >create an adapter, then their data would be serialized incompletely. >I like the idea of using named utilities, using dotted class names as >utility ids. See below. Ok, I will give named utilities a try. > >> - Can zope.app.fssync.fspickle be replaced by zope.xmlpickle? (It >> seems >> that fspickle preserves location ids but it does not seem to >> preserve the >> order of dict items) > >I'm a bit rusty on this. I would think that these can be combined. >It should, I think, be possible to generate location-aware pickles >and then use zmlpickle to represent these as XML. The trick, I >think, would be use the pickle persistent-reference mechanism to >refer to other objects by location, when necessary. I say this >without looking at the code. :) If you need me to dig deeper, I'll >try to find time to do so. I found such a combination in zope.fssync.server.entryadapter. There a location-aware pickle was created by fspickle.dumps and converted via xmlpickle.toxml. This version generated xmlpickles with changing dict representations which in turn led to false alarms that objects had been modified. I will try to combine the xmlpickle._PicklerThatSortsDictItems with the fspickle.ParentPersistentIdGenerator and see whether this solves the problem. If this doesn't work, I will ask you for further assistance. > Uwe > Dr. Uwe Oestermeier Institut für Wissensmedien Knowledge Media Research Center Konrad-Adenauer-Str. 40 D-72072 Tuebingen Germany [EMAIL PROTECTED] Tel. +49 7071 979-208 Fax +49 7071 979-100 ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] future of fssync (was: RE: [SpringCleaning07])
On Jan 24, 2007, at 7:18 AM, Uwe Oestermeier wrote: Sorry for replying so late. I have just checked in some bug fixes for fssync (r72206). No apologies needed. Thanks for picking this up! This was indeed not much work. Jim Fulton <[EMAIL PROTECTED]> wrote: I don't think a whole lot is needed to make fssync a reality: 1. Cure any bitrot that has set in. It would also ne nice to replace existing tests with modern doctests. 2. Provide a Python API. fssync originally had a Python API, but this was replaced with a web-based API. I think there should be both a Python API that wasn't encumbered in any way by security, and a protected web-based API. The Python API should really be Zope and even ZODB independent. I don't think this would be a lot of work. The original one wasn't and would be useful in many cases. 3. I think there should be a secure web-based interface. This will require: - Adding security checks that the user is allowed to access serialization and de-serialization adapters, - Adding security declarations for these adapters, I don't think any of these would require a great deal of work. I can try to deal with these tasks, but I fear that there is a little bit more to do. While playing with real data, I noticed - problems to serialize large sites (I run into a stack overflow and have still to figure out why this happened), Hm, interesting. I'll be curious to hear how this turns out. - problems with unicode filenames. Ah, interesting. As a first step I would like to clean up the existing code. Concerning this I have questions/suggestions: - Can the fssync:adapter zcml directive be replaced by ordinary trusted adapters? Maybe, but I like your idea of using utilities below. My original thinking was along these lines: fssync should strive to serialize "all" object data. (Note that it isn't always obvious what data is intrinsic to an object.) I felt therefore, that inheriting synchronization adapters would be bad. If someone created a subclass and forgot to create an adapter, then their data would be serialized incompletely. I like the idea of using named utilities, using dotted class names as utility ids. See below. An alternative might be to see if some ideas from Ape could be used to catch when an object serialization is incomplete. I think that this would be a lot of work and not something to pursue in the short term. I have, on multiple occasions wanted class-based adapters without inheritance. The security-checker registry is another use case for this. At some point, I'd like to extend the component architecture in this direction. - Can zope.app.fssync.fspickle be replaced by zope.xmlpickle? (It seems that fspickle preserves location ids but it does not seem to preserve the order of dict items) I'm a bit rusty on this. I would think that these can be combined. It should, I think, be possible to generate location-aware pickles and then use zmlpickle to represent these as XML. The trick, I think, would be use the pickle persistent-reference mechanism to refer to other objects by location, when necessary. I say this without looking at the code. :) If you need me to dig deeper, I'll try to find time to do so. - What's the purpose of zope.app.fssync.fsbundle? Are there still usecases for this code or can it be removed? Yes - What's the purpose of zope.app.fssync.fsregistry? Can this be replaced by utility registrations? I believe that this is the class-based adapter registry discussed above. Using utilities is an intriguing idea. Would you use the dotted class name as the utility id? I assume that the utilities would actually be adapter factories. All in all the fssync code seem to be in an old-fashioned but usable shape and it's a pity that it has not been maintained. Yes. It's a pity that there isn't more time to work on useful projects like this. This is also very early code. It goes back to 2002. We've learned a lot since then. Perhaps the maintenance can be made easier if we can change the code without deprecation warnings. Nobody seems to have used fssync in the last two years. +1 Thanks again for looking at this. Jim -- Jim Fulton mailto:[EMAIL PROTECTED]Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporationhttp://www.zope.com http://www.zope.org ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] future of fssync (was: RE: [SpringCleaning07])
2007/1/24, Uwe Oestermeier <[EMAIL PROTECTED]>: Perhaps the maintenance can be made easier if we can change the code without deprecation warnings. Nobody seems to have used fssync in the last two years. I think you can take that granted. fssync has been pretty useless so far. -- Best regards, Miklós Prisznyák Principal Consultant, owner http://www.jegenye.com ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] future of fssync (was: RE: [SpringCleaning07])
Sorry for replying so late. I have just checked in some bug fixes for fssync (r72206). This was indeed not much work. Jim Fulton <[EMAIL PROTECTED]> wrote: >I don't think a whole lot is needed to make fssync a reality: > >1. Cure any bitrot that has set in. It would also ne nice to replace >existing >tests with modern doctests. > >2. Provide a Python API. fssync originally had a Python API, but this >was > replaced with a web-based API. I think there should be both a >Python > API that wasn't encumbered in any way by security, and a >protected web-based > API. The Python API should really be Zope and even ZODB >independent. > I don't think this would be a lot of work. The original one >wasn't and would > be useful in many cases. > >3. I think there should be a secure web-based interface. This will >require: > > - Adding security checks that the user is allowed to access >serialization > and de-serialization adapters, > > - Adding security declarations for these adapters, > >I don't think any of these would require a great deal of work. I can try to deal with these tasks, but I fear that there is a little bit more to do. While playing with real data, I noticed - problems to serialize large sites (I run into a stack overflow and have still to figure out why this happened), - problems with unicode filenames. As a first step I would like to clean up the existing code. Concerning this I have questions/suggestions: - Can the fssync:adapter zcml directive be replaced by ordinary trusted adapters? - Can zope.app.fssync.fspickle be replaced by zope.xmlpickle? (It seems that fspickle preserves location ids but it does not seem to preserve the order of dict items) - What's the purpose of zope.app.fssync.fsbundle? Are there still usecases for this code or can it be removed? - What's the purpose of zope.app.fssync.fsregistry? Can this be replaced by utility registrations? All in all the fssync code seem to be in an old-fashioned but usable shape and it's a pity that it has not been maintained. Perhaps the maintenance can be made easier if we can change the code without deprecation warnings. Nobody seems to have used fssync in the last two years. Regards, Uwe Dr. Uwe Oestermeier Institut für Wissensmedien Knowledge Media Research Center Konrad-Adenauer-Str. 40 D-72072 Tuebingen Germany [EMAIL PROTECTED] Tel. +49 7071 979-208 Fax +49 7071 979-100 ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] future of fssync (was: RE: [SpringCleaning07])
On Jan 2, 2007, at 10:23 AM, Stephan Richter wrote: On Wednesday 20 December 2006 16:03, Jeff Shell wrote: zope.fssync +1 0 It's too bad that this seems to have gone unfinished. The biggest pain (well, one of the big pains) we experience with Zope 3 is the lack of anything like Zope 2's export/import. Or, going further back, 'manage_exportHack'. :). This is a side topic and I'm not going to elaborate further except to beg for some low to medium level export/import capability. It seems that fssync was one of the alternatives, or could be, if I recall correctly. The question is why you need it. zope.fssync was originally developed for TTW development via the filesystem. I think this use case has gone away. Certainly TTW development has a much smaller future than once envisioned, but it is still done to some degree. What use cases do *you* have for using fssync? I think fssync would be very useful for content management. I'd love to be able to use fssync to manage a wiki (or other collection of documents) offline. Of course, there is the use case that Jeff mentioned, which is export/ import. I don't think a whole lot is needed to make fssync a reality: 1. Cure any bitrot that has set in. It would also ne nice to replace existing tests with modern doctests. 2. Provide a Python API. fssync originally had a Python API, but this was replaced with a web-based API. I think there should be both a Python API that wasn't encumbered in any way by security, and a protected web-based API. The Python API should really be Zope and even ZODB independent. I don't think this would be a lot of work. The original one wasn't and would be useful in many cases. 3. I think there should be a secure web-based interface. This will require: - Adding security checks that the user is allowed to access serialization and de-serialization adapters, - Adding security declarations for these adapters, I don't think any of these would require a great deal of work. Sadly, I won't be able to work on fssync any time soon, but I'd be willing to provide some high-level advice and oversight if someone else was able and interested in working on it. Jim -- Jim Fulton mailto:[EMAIL PROTECTED]Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporationhttp://www.zope.com http://www.zope.org ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
