Re: [Zope-dev] Serving big files thru zope?

[Dieter Maurer]

Dario Lopez-Kästen wrote at 2004-3-1 08:24 +0100:
> ...
>Is the current state of ZServer such that it is suboptimal for serving 
>large files? if so,
>
>a) what is the problem with the current implementation?

In order to reduce memory consumption and decouple Zope from
ZServer (and the response delivery), a large file
is spooled via a temporary.

This means, that the file content is read from ZODB, stored
in a temporary file and then delivered from this temporary file.

At least, when your client has a high bandwidth to your server,
this serving through a temporary file is a waste.

Serving static content via Python/Zope is about one order
of magnitude slower than serving the same static content
via Apache.

>b) is there a fix available?

Probably, but it will be a lot of work:

  When your client (or its connection) is slow, you
  want to spool large files indirectly (to free the
  scares Zope worker thread soon).
  Only when you know your client is fast, would you want
  to serve directly.
  An adaptive scheme is possible but much more demanding
  than the simple schema (to use a temporary file for
  all sufficiently large content).

  You could code ZServer in "C" and speed it up thereby.
  Maybe "psyco" helps a bit?

-- 
Dieter

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] server for new protocol?

[Nikolay Kim]

it'll be open source.

source in attach.

it not very usefull without additional products that must actually
handle emails. 

now i'm working on mail storage products.


> Ooh!  This is really cool.  Will this be open source?  I'll be a lot of people 
> will be interested in this.
> 
> 
>   -Fred
> 
> -- 
> Fred L. Drake, Jr.  
> PythonLabs at Zope Corporation




SMTPService.tar.gz
Description: application/compressed-tar
___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

[Zope-dev] running test.py with instance_home products.

[Chris McDonough]

Has anyone found a reasonable strategy for using Zope's test.py to test
products that are installed outside the Zope software home?  I try
running it from the software home and using the --dir option to point it
to where my product is, but it tells me it can't find any tests.

- C



___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Custom class: cut,copy,rename?

[Max M]

Ian Beatty wrote:

> Am I not inheriting from the proper set of base classes? Is there an 
easier
> way? According to the ZDG (2.4 edition, vs. my Zope 2.7),
> OFS.SimpleItem.Item (subclassed by OFS.SimpleItem.SimpleItem) is 
supposed to
> provide cut-and-paste functionality. So, how might I be breaking it?

The order of the parent classes matters. Ie. Some methods are defined in 
SimpleItem that returns None. Clipped from the source::

# This keeps simple items from acquiring their parents
# objectValues, etc., when used in simple tree tags.
def objectValues(self, spec=None):
return ()
objectIds=objectItems=objectValues
So if SimpleItem is before ObjectManager the traversal order of the 
parent objects will make shure that you get the wrong methods :-( So 
this will *not* work:

class aSimpleItem(
 CatalogAware, PropertyManager,
 SimpleItem.SimpleItem, ObjectManager):
This order works:

class aSimpleItem(
 CatalogAware, PropertyManager, SimpleItem):
class anObjectManager(
 ObjectManager, aSimpleItem):
regards Max M

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please

[Chris McDonough]

Er, yes... I wrote it.  I think you might have mixed up message headers,
as I was the one who sent the message out to the list announcing it.


On Mon, 2004-03-01 at 18:30, Steve Jibson wrote:
> Chris,
> 
> I'm not sure if you'd even planned to, but have you looked over this 
> replacement "Transience.py"?  Do you have an opinion on the 
> re-implementation?  I plan on looking over it myself, but even if I like 
> it, I'd sleep better while it runs on my customer's servers if I knew it 
> had your "blessing".
> 
> Thanks,
> Steve
> 
> 
> 
> Chris McDonough wrote:
> >>I installed new Transience.py. During my little test it works fine.
> >>But real test will be on Monday when students start logging in as complete
> >>classes, sometimes there are hundreds of them logging on simultaneously, 
> >>so we will see. 
> > 
> > 
> > Any news? ;-)
> 
> ___
> Zope-Dev maillist  -  [EMAIL PROTECTED]
> http://mail.zope.org/mailman/listinfo/zope-dev
> **  No cross posts or HTML encoding!  **
> (Related lists - 
>  http://mail.zope.org/mailman/listinfo/zope-announce
>  http://mail.zope.org/mailman/listinfo/zope )


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please

[Steve Jibson]

Doh!

Scrap my last message.  I was confused.  The file he's testing is the 
one Chris wrote (so I guess he's seen it, duh).



Chris McDonough wrote:
I installed new Transience.py. During my little test it works fine.
But real test will be on Monday when students start logging in as complete
classes, sometimes there are hundreds of them logging on simultaneously, 
so we will see. 


Any news? ;-)

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please

[Steve Jibson]

Chris,

I'm not sure if you'd even planned to, but have you looked over this 
replacement "Transience.py"?  Do you have an opinion on the 
re-implementation?  I plan on looking over it myself, but even if I like 
it, I'd sleep better while it runs on my customer's servers if I knew it 
had your "blessing".

Thanks,
Steve


Chris McDonough wrote:
I installed new Transience.py. During my little test it works fine.
But real test will be on Monday when students start logging in as complete
classes, sometimes there are hundreds of them logging on simultaneously, 
so we will see. 


Any news? ;-)
___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )

[Zope-dev] Slow FTP Downloads - Zope 2.6.4, 2.7

[Edward Pollard]

Some of you might recognize this thread from the [EMAIL PROTECTED] mailing 
list. With great reluctance smothered by desperation I thought it might 
be fruitful to disseminate it on this list as well.

We have been using Zope 2.6.0 for some time. When we decided an upgrade 
was in order (to get away from ZODB killing maintenance tools), we 
moved to Zope 2.6.2, as that was what was available at the time. At the 
tail end of that upgrade, we discovered some crippling FTP bugs which 
seem to have been acknowledged and fixed in later revisions.

Attempts to move to Zope 2.6.4 and 2.7 have been stymied, however, 
since any servers we have set up with these versions of Zope refuse to 
allow users to download via FTP faster than 10k/second. During these 
downloads, the CPU shoots straight to 100%. While we can have 10 
different connections active, they all go at 10k/sec and the CPU is 
always at 100% regardless of how many connections we have. This is 
across a variety of FTP clients, so we don't think that is related to 
the issue.

Experiments with the Profiler have been fruitless, and lack of 
experience with strace has blocked any progression at locating this 
problem with that, aside from noting an tremendous amount of failed 
open and stat64 calls - but since these are also in 2.6.0 I am not 
inclined to believe them significant to this problem.

The server is a Dell Poweredge 1750 single processor 2.4GHz P4 Xeon 
with hyper-threading. RedHat 9 is running on the hardware mirrored 
internal HD, Zope on a NFS mounted (via TCP) volume. Moving Zope to the 
local disk and disabling hyper-threading has had no impact on this 
problem.

So far, all I've been able to determine is that nobody else is having 
this problem. My mom always said I was special.

Anyone with thoughts not previously shared on the other list? My only 
option at this point seems to be to go back to 2.6.0 while keeping 
later revisions on the ZODB tools. An inability to eventually rectify 
this problem would be pretty hard blow against using Zope at our 
institution, however, so I'm trying to avoid seriously contemplating 
that decision since I've grown so fond of Zope.

---
Edward J. Pollard, B.Sc
Webmaster, University of Lethbridge
___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )

[Zope-dev] Re: Custom class: cut,copy,rename?

[Ian Beatty]

On 2004-03-01 3:44 PM, Chris Withers is reputed to have said:

> Ian Beatty wrote:
> 
>> OFS.ObjectManager.ObjectManager
>> Products.ZCatalog.CatalogPathAwareness.CatalogAware
>> OFS.PropertyManager.PropertyManager
>> OFS.SimpleItem.SimpleItem
> 
> I'm pretty sure all you should actually need is:
> 
>> OFS.ObjectManager.ObjectManager
>> Products.ZCatalog.CatalogPathAwareness.CatalogAware

I presume you mean "for cut, paste, rename, etc." If I remove
PropertyManager, my 'Properties' ZMI tab disappears. I'm not quite sure what
SimpleItem provides for me, but it's in the examples I'm trying to work
from.

>> OFS.SimpleItem.Item (subclassed by OFS.SimpleItem.SimpleItem) is supposed to
>> provide cut-and-paste functionality. So, how might I be breaking it?
> 
> Show us some error messages, exceptions, types ,values and tracebacks adn
> amaybe 
> we can help ;-)

Okay. This is bizarre. I've got a containment hierarchy of several classes;
I'll talk about the top 3. I've got things set up so that I can create
instances of A within my Zope root (or any other "normal" Zope folder), I
can create instances of B and only B within A, and I can create instances of
C and only C within B.

When I try to rename an instance of A, the ZMI gives me the standard
rename-object form; but when I type in a new ID and click 'OK', I get a ZMI
page with a red exclamation mark, an 'OK' button, and nothing else except
the word '_notifyOfCopyTo'. The error log shows a "Copy Error" with the
following traceback:

Traceback (innermost last):
  Module ZPublisher.Publish, line 100, in publish
  Module ZPublisher.mapply, line 88, in mapply
  Module ZPublisher.Publish, line 40, in call_object
  Module OFS.CopySupport, line 230, in manage_renameObjects
  Module OFS.CopySupport, line 252, in manage_renameObject
Copy Error: 
[html snipped]

Now, If I view the ZMI for A, select a contained item of type B, and try to
rename it, it fails but with a different signature: the notification page
says 'The object term_0 does not support this operation' instead of
'_notifyOfCopyTo'. ("term_0 is the ID of the B instance.) The error log
shows a Copy Error with this traceback:

Traceback (innermost last):
  Module ZPublisher.Publish, line 100, in publish
  Module ZPublisher.mapply, line 88, in mapply
  Module ZPublisher.Publish, line 40, in call_object
  Module OFS.CopySupport, line 230, in manage_renameObjects
  Module OFS.CopySupport, line 247, in manage_renameObject
  Module OFS.CopySupport, line 393, in _verifyObjectPaste
Copy Error: 
[html snipped]

Attempting to rename an instance of C within B fails in the same way as
renaming A (i.e., with a '_notifyOfCopyTo').

I've currently got A subclassing ObjectManager, CatalogAware, and
PropertyManager; B subclassing the same plus SimpleItem (just as an
experiment); and C subclassing the same as A. If I remove SimpleItem from
B's list of superclasses, I still get the same set of errors, *except* when
attempting to rename C. Now, I instead get the failure message 'You do not
possess the permission required to call
manage_addProduct/KBProduct/manage_KBCourseAddForm in the context of the
container into which you are pasting, thus you are not able to perform this
operation.' The Error Log traceback is:

Traceback (innermost last):
  Module ZPublisher.Publish, line 100, in publish
  Module ZPublisher.mapply, line 88, in mapply
  Module ZPublisher.Publish, line 40, in call_object
  Module OFS.CopySupport, line 230, in manage_renameObjects
  Module OFS.CopySupport, line 247, in manage_renameObject
  Module OFS.CopySupport, line 378, in _verifyObjectPaste
Copy Error: 
[html snipped]

I'm completely bewildered here. Any suggestions would be tremendously
appreciated.

By the way, if this isn't the appropriate venue for this kind of question,
please point me in the right direction. This list seems to focus on Zope's
internals, but the Zope Users list seems a bit low-level for Python-based
product development.

Thanks,  

..Ian

-- --- -- --- -- --- -- --- -- --- -- --- -- --- -- --- -- --- --
Dr. Ian Beatty   [EMAIL PROTECTED]
Physics Education Research Group  voice: 413.545.9483
Department of Physics   fax: 413.545.4884
Univ. of Massachusetts  AIM: (available upon request)
Amherst, MA 01003-4525 USA   http://umperg.physics.umass.edu/
-- --- -- --- -- --- -- --- -- --- -- --- -- --- -- --- -- --- --



___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please

[Chris McDonough]

> I installed new Transience.py. During my little test it works fine.
> But real test will be on Monday when students start logging in as complete
> classes, sometimes there are hundreds of them logging on simultaneously, 
> so we will see. 

Any news? ;-)



___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Custom class: cut,copy,rename?

[Chris Withers]

Ian Beatty wrote:

OFS.ObjectManager.ObjectManager
Products.ZCatalog.CatalogPathAwareness.CatalogAware
OFS.PropertyManager.PropertyManager
OFS.SimpleItem.SimpleItem
I'm pretty sure all you should actually need is:

> OFS.ObjectManager.ObjectManager
> Products.ZCatalog.CatalogPathAwareness.CatalogAware
OFS.SimpleItem.Item (subclassed by OFS.SimpleItem.SimpleItem) is supposed to
provide cut-and-paste functionality. So, how might I be breaking it?
Show us some error messages, exceptions, types ,values and tracebacks adn amaybe 
we can help ;-)

cheers,

Chris

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )

[Zope-dev] Re: Usability tweaks

[Casey Duncan]

On Mon, 01 Mar 2004 20:25:54 +
Christian Theune <[EMAIL PROTECTED]> wrote:

> Hi,
> 
> as I'm tweaking stuff here and there right now, I stumbled over the
> findSupport dtml that has a multiple select menu with a height of 4.
> It's almost impossible to find what you look for easily. Increasing
> the height to 8 would be good. Anything against that?

Sounds great to me.
 
> Christian
> 
> PS: Anything against simply doing those small changes?

I say go ahead. Just make sure you write liberal check-in comments and
test in various browsers (even stupid stuff breaks on *some* browsers).
Also update CHANGES.txt so people can tell what they're getting into
when they install the new release. I think enough of us monitor the
checkins to catch anything wacky pretty early.

At least that's my opinion...

-Casey


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

[Zope-dev] Usability tweaks

[Christian Theune]

Hi,

as I'm tweaking stuff here and there right now, I stumbled over the
findSupport dtml that has a multiple select menu with a height of 4.
It's almost impossible to find what you look for easily. Increasing the
height to 8 would be good. Anything against that?

Christian

PS: Anything against simply doing those small changes?

-- 
Christian Theune, gocept gmbh & co. kg
http://www.gocept.com - [EMAIL PROTECTED]
fon: 03496 3099112 fax: 03496 3099118 mobile: 0179 7808366


signature.asc
Description: This is a digitally signed message part
___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Serving big files thru zope?

[Chris McDonough]

I think we should work on making Zope perform better when large files
are downloaded at the PyCon sprint in March, as you suggested Paul,
FWIW.

On Mon, 2004-03-01 at 13:41, Paul Winkler wrote:
> On Mon, Mar 01, 2004 at 01:03:24PM +0100, Dario Lopez-K?sten wrote:
> > I am also wondering about the security aspects of bypassing Zope to 
> > serve files - acess to some of the files we need to serve is restricted 
> > to particular users with particular permissions, and I cannot clearly 
> > see how this would be possible to accomplish without Zope's security 
> > machinery.
> 
> Tricky.  The only solution I've heard of (but not tried)
> is to use Squid configured for disk caching and forcing
> revalidation against the proxied server (zope) before allowing
> a download from the cache.
>  
> > So, I am looking for a solution that either utilises Zope+ZServer with 
> > additional fixes or that replaces ZServer altoghether with something 
> > better (Apache, Pound, Twisted?). All this assuming of course that 
> > ZServer is not good at serving large files to many users.
> 
> I'm not sure about that anymore.
> My latest experiments with large files 
> suggest that Zope / ZServer might not be as
> bad as I thought when compared to Apache *over a network*.
> I had previously only tested on localhost and observed
> that Apache is about 10x faster than ZServer for a 40 MB file.
> I had (probably erroneously) thought that this explained
> the poor download times from Zope that I had seen in the field.
> But on a 100 Mbps local network, the download times were pretty
> close - the network becomes the bottleneck and Apache was only
> a little bit faster than Zope.
> 
> However, ZEO is a whole other story.
> The time to load a 40 MB file from ZEO and serve it,
> is about another 10x slower than plain Zope  without ZEO.
> This is painfully bad and readily apparent to users.
> But if your ZEO cache is large enough to hold all the data, 
> subsequent download times are as fast as plain Zope. 
> More investigation needed, I'm not sure about the accuracy
> or relevance of these results.  But almost certainly,
> if using large files with Zope with ZEO, you will want to configure 
> a much larger ZEO cache size than the default (20 MB). 
> 
> Also, this experiment was only for a single download at
> a time.  If you expect a lot of concurrent downloads of
> large files, better do some testing to see if Zope's large
> file download time deteriorates under load.
> I am not especially optimistic ;-)


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

[Zope-dev] Custom class: cut,copy,rename?

[Ian Beatty]

Greetings.

I've created a package in Python with some basic classes. They work in the
ZMI, except for the fact that the ZMI paste and rename functionality fails
with the message "The object xxx does not support this operation."

I've dug into the source code for standard types ('OFS.Folder',
'OFS.SimpleItem', etc.) and discovered some tantalizing bits (such as
'manage_renameObject' in 'OFS.CopySupport'), but I can't figure out how to
put it together to make my classes renameable and pasteable. I'm rather
surprised that behavior doesn't come automatically when inheriting from the
standard base classes.

One of my folderish classes subclasses the following bases:

OFS.ObjectManager.ObjectManager
Products.ZCatalog.CatalogPathAwareness.CatalogAware
OFS.PropertyManager.PropertyManager
OFS.SimpleItem.SimpleItem

Am I not inheriting from the proper set of base classes? Is there an easier
way? According to the ZDG (2.4 edition, vs. my Zope 2.7),
OFS.SimpleItem.Item (subclassed by OFS.SimpleItem.SimpleItem) is supposed to
provide cut-and-paste functionality. So, how might I be breaking it?

Bewildered,

..Ian

-- --- -- --- -- --- -- --- -- --- -- --- -- --- -- --- -- --- --
Dr. Ian Beatty   [EMAIL PROTECTED]
Physics Education Research Group  voice: 413.545.9483
Department of Physics   fax: 413.545.4884
Univ. of Massachusetts  AIM: (available upon request)
Amherst, MA 01003-4525 USA   http://umperg.physics.umass.edu/
-- --- -- --- -- --- -- --- -- --- -- --- -- --- -- --- -- --- --



___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Serving big files thru zope?

[Jeremy Hylton]

On Mon, 2004-03-01 at 13:41, Paul Winkler wrote:
> However, ZEO is a whole other story.
> The time to load a 40 MB file from ZEO and serve it,
> is about another 10x slower than plain Zope  without ZEO.
> This is painfully bad and readily apparent to users.
> But if your ZEO cache is large enough to hold all the data, 
> subsequent download times are as fast as plain Zope. 
> More investigation needed, I'm not sure about the accuracy
> or relevance of these results.  But almost certainly,
> if using large files with Zope with ZEO, you will want to configure 
> a much larger ZEO cache size than the default (20 MB). 

It really would be good to get to the bottom of the bad ZEO performance
for loading an object the first time.  I won't have time to do it until
after the ZODB 3.3 release, but I think it's a worthwhile effort to
understand what ZEO is doing.

You're right, too, that a large ZEO cache makes a big difference.  If
you've got the disk space, setting it several hundred MB shouldn't be a
problem.

Jeremy



___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Serving big files thru zope?

[Paul Winkler]

On Mon, Mar 01, 2004 at 01:03:24PM +0100, Dario Lopez-K?sten wrote:
> I am also wondering about the security aspects of bypassing Zope to 
> serve files - acess to some of the files we need to serve is restricted 
> to particular users with particular permissions, and I cannot clearly 
> see how this would be possible to accomplish without Zope's security 
> machinery.

Tricky.  The only solution I've heard of (but not tried)
is to use Squid configured for disk caching and forcing
revalidation against the proxied server (zope) before allowing
a download from the cache.
 
> So, I am looking for a solution that either utilises Zope+ZServer with 
> additional fixes or that replaces ZServer altoghether with something 
> better (Apache, Pound, Twisted?). All this assuming of course that 
> ZServer is not good at serving large files to many users.

I'm not sure about that anymore.
My latest experiments with large files 
suggest that Zope / ZServer might not be as
bad as I thought when compared to Apache *over a network*.
I had previously only tested on localhost and observed
that Apache is about 10x faster than ZServer for a 40 MB file.
I had (probably erroneously) thought that this explained
the poor download times from Zope that I had seen in the field.
But on a 100 Mbps local network, the download times were pretty
close - the network becomes the bottleneck and Apache was only
a little bit faster than Zope.

However, ZEO is a whole other story.
The time to load a 40 MB file from ZEO and serve it,
is about another 10x slower than plain Zope  without ZEO.
This is painfully bad and readily apparent to users.
But if your ZEO cache is large enough to hold all the data, 
subsequent download times are as fast as plain Zope. 
More investigation needed, I'm not sure about the accuracy
or relevance of these results.  But almost certainly,
if using large files with Zope with ZEO, you will want to configure 
a much larger ZEO cache size than the default (20 MB). 

Also, this experiment was only for a single download at
a time.  If you expect a lot of concurrent downloads of
large files, better do some testing to see if Zope's large
file download time deteriorates under load.
I am not especially optimistic ;-)

-- 

Paul Winkler
http://www.slinkp.com
Look! Up in the sky! It's THE PREPUBESCENT PROFESSOR!
(random hero from isometric.spaceninja.com)

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Zope 2.7 tutorial

[Chris McDonough]

Thank you Christian!

On Mon, 2004-03-01 at 12:01, Christian Theune wrote:
> Hi,
> 
> gocept hired a new developer and thus I send him dig through the Zope
> tutorial at first. He gave me a list of bugs he found in the tutorial
> which we corrected here. Therefore I'll update the tutorial in the 2.7
> branch in the next days. I'd updated it in 2.8 (HEAD) as well but it is
> severely broken in there (can't even create a tutorial).
> 
> Cheers,
> Christian


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

RE: [Zope-dev] Serving big files thru zope?

[Sandor Palfy]

> I am also wondering about the security aspects of bypassing Zope to 
> serve files - acess to some of the files we need to serve is 
> restricted 
> to particular users with particular permissions, and I cannot clearly 
> see how this would be possible to accomplish without Zope's security 
> machinery.

Same situation here. Is there any way to provide authentication
information for Apache from Zope (which works at least with cookie based
auth)? 

Regards,
Sandor


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

[Zope-dev] Zope 2.7 tutorial

[Christian Theune]

Hi,

gocept hired a new developer and thus I send him dig through the Zope
tutorial at first. He gave me a list of bugs he found in the tutorial
which we corrected here. Therefore I'll update the tutorial in the 2.7
branch in the next days. I'd updated it in 2.8 (HEAD) as well but it is
severely broken in there (can't even create a tutorial).

Cheers,
Christian

-- 
Christian Theune, gocept gmbh & co. kg
http://www.gocept.com - [EMAIL PROTECTED]
fon: 03496 3099112 fax: 03496 3099118 mobile: 0179 7808366


signature.asc
Description: This is a digitally signed message part
___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

[Zope-dev] Re: server for new protocol?

[Casey Duncan]

On Mon, 1 Mar 2004 10:09:36 -0500
Fred Drake <[EMAIL PROTECTED]> wrote:

> On Sunday 29 February 2004 11:21 pm, Nikolay Kim wrote:
>  > P.S. i'm developing smtp server for zope and already have working
>  > code.
> 
> Ooh!  This is really cool.  Will this be open source?  I'll be a lot
> of people will be interested in this.

Me too, me too! ;^)

-Casey


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

[Zope-dev] Re: [Zope3-dev] Interfaces in Zope 2.5, 2.7, and 3.x

[Jim Fulton]

Terry Hancock wrote:
I have a product that uses a component/interface model
internally.  I wrote this against the "scarecrow" Interface 
module in Zope 2.5.1.
Cool.

However that module has limitations.  One particular one is
what I call a "SubInterface" -- which is a special case of 
Interface.Attribute (a subclass of it) that asserts that an
object contained within a parent interface implements a
particular interface.
(Note that I responsed to this in the "Sub Interfaces" thread
 on zope3-dev, if anyone's intereested. :)

Anyway, I wrote my own interface module to do this,
extending the Interface module in Zope 2.5.1 (which lacks
this feature).  All of my code actually imports interface support
from my module instead of directly from Interface.
For unrelated reasons, though, I had to upgrade to Zope 2.7
(and someday, I imagine I'll want to consider moving to Zope 3.x).
My first attempt with Zope 2.7, though yields total breakage!

The Interface module has been extensively overhauled, method
names changed, etc. After giving up on "just fixing a few name-
changes", which seems to be woefully inadequate, I decided to
try to lift the old Interface module -- but now Zope reads the 
"__implements__" attribute and isn't happy if it isn't one of its own
interface objects!


Sorry about that.

These changes were made for Zope 2.6.  I'm pretty sure the changes
were made before the Zope 2.6 beta cycle.  Nobody complained
then, so I assumed I hadn't broken anything. :)
Now, I can implement a new module based on the Zope 2.7 Interface -- but
a close look at that seems to show that it, too, is rather experimental (lots of
stubs, not-yet-implemented attributes, etc).  So I'm also sort of interested in
the "interface" module in Zope X3.
We discussed this on zope3-dev and came to the conclusion that you
want to use Zope 3 interfaces and schema.  Zope 3 interfaces will become
standard in Zope 2 in Zope 2.8.
It appears that some new bells and whistles have been added to the
Interface module (e.g. what's a "superinterface"?).   Any chance, I'm
wondering if "SubInterface" is now redundant?
Also, is there a general guide to "best practice" of using / learning the
Interface module?  I note there are now interfaces defined for the interface
module (cool!), and it looks like some self-documentation has been
implemented.
The biggest problem (and this seems to be the usual problem with
"self-documenting" tools) is that there doesn't seem to be a "start here"
defined anywhere.  So you wind up starting somewhere in the middle and
wasting a lot of time trying to decipher the big picture from lots of details,
which is backwards. It'd be nicer if there was a "README" to get you started
on this, maybe define some jargon, etc. 
Yes, that would definately help.  We've been doing that a lot elsewhere.

> It's really hard to infer what the
overall design concept is by only looking at these details -- in particular,
I want to know what was conceptually changed from the older Interface
module in 2.5.1.
Well, we probably won't help you there. That was too long ago, sorry.

Of course, the version in 2.5 was pretty close to the scarecrow proposal.
Presumabley, if you had a README.txt, you could do the conceptual diff yourself.

This seems to also be a problem with combining Zope 3 and Zope 2
interfaces, as was suggested by a post on the Zope3 mailing list:
http://mail.zope.org/pipermail/zope3-dev/2003-December/009023.html
(and following -- note that "__interfaces__" should read "__implements__"
in the original post).
Martijn's suggestion was to use an alternate name for the "__implements__"
attribute.  I suppost I could do that -- call mine "__narya_interfaces__", but
that sounds ugly.  In particular, I don't really want my interfaces module
to actually *conflict* with Zope's.
As I mentioned on the zope3-dev list, you want to use the facade Interface
from (Packages3/Interface) in Zope 2.7. This is a bit of a pain to set up.
The facade will be a standard part of Zope 2.8.

I also note that the post from the Zope3 list suggests that I shouldn't
really be directly assigning "__implements__", but using something called
"implements()".  Hmm. That's different too -- is there any good starting
place for understanding Zope3's "interface" module?
See zope.interface.interfaces in a zope 3 checkout. See particularly the
declaration api.
And how different are the Zope 2.7 / Zope X3 interface modules?  If I
re-write my module to conform to Zope 2.7, will I just have to do it
again?
You should write it against the Zope 3 interfaces.

Could I instead leap ahead and use the Zope X3 interface module in
Zope 2.7?
Yes.

Also, since I'm posting to Zope3-dev as well -- what do you think of the
SubInterface concept? 
As we discussed on zope3-dev, it's a great concept.  That's why we implemented
it with the Object scema field type. :)
> Should I be contributing code directly to Zope X3's
interface instead of subclassing it?
Absolutely!

Jim

--
Jim Fulton 

Re: [Zope-dev] server for new protocol?

[Fred Drake]

On Sunday 29 February 2004 11:21 pm, Nikolay Kim wrote:
 > P.S. i'm developing smtp server for zope and already have working code.

Ooh!  This is really cool.  Will this be open source?  I'll be a lot of people 
will be interested in this.


  -Fred

-- 
Fred L. Drake, Jr.  
PythonLabs at Zope Corporation


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Serving big files thru zope?

[Dario Lopez-Kästen]

Stefan H. Holek wrote:

Dario,

The best solution for me always has been to serve such files statically 
from Apache, bypassing ZServer altogether. In fact I have added support 
for static URLs to my version of ExtFile 
.

ZEO clusters will need a comon disk share (SAMBA, NFS) for this to work.
Hi, thanks for the reply.

I am a bit hesitant to revert to filesystem based storage at this point 
- we used to use LocalFS but switched from it in favor of DirectoryStorage.

I am also wondering about the security aspects of bypassing Zope to 
serve files - acess to some of the files we need to serve is restricted 
to particular users with particular permissions, and I cannot clearly 
see how this would be possible to accomplish without Zope's security 
machinery.

So, I am looking for a solution that either utilises Zope+ZServer with 
additional fixes or that replaces ZServer altoghether with something 
better (Apache, Pound, Twisted?). All this assuming of course that 
ZServer is not good at serving large files to many users.

Thanks,

/dario

--
-- ---
Dario Lopez-Kästen, IT Systems & Services Chalmers University of Tech.
___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please

[Anthony Baxter]

Chris McDonough wrote:
That is very critital and 'blocker' bug for Zope. I do not understand how
people could use Zope on a sites with high load..


They don't use sessions under high load.
Or they don't use the standard session machinery. We use my SQLSession 
stuff, and it's fine under load.

--
Anthony Baxter <[EMAIL PROTECTED]>
It's never too late to have a happy childhood.
___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Serving big files thru zope?

[Stefan H. Holek]

Dario,

The best solution for me always has been to serve such files statically 
from Apache, bypassing ZServer altogether. In fact I have added support for 
static URLs to my version of ExtFile .

ZEO clusters will need a comon disk share (SAMBA, NFS) for this to work.

HTH,
Stefan
--On Montag, 01. März 2004 08:24 +0100 Dario Lopez-Kästen 
<[EMAIL PROTECTED]> wrote:

Hello,

I need to serve big files from zope - PDF, movies, large PS-files,
Students lab-results, etc. We recently moved from LocalFS to  Dirstorage
to manage ZEO-ability in a clustered enviromnet).
Quite a while ago, I remember having read somewhere on the lists that
there is a pacth/fix for Zope/ZServer that optimises the way zope serves
large files.
Is the current state of ZServer such that it is suboptimal for serving
large files? if so,
a) what is the problem with the current implementation?
b) is there a fix available?
if not b) then I am willing to start looking into it. I need this for my
unis student web-site.
Thanx,

/dario

--
-- ---
Dario Lopez-Kästen, IT Systems & Services Chalmers University of Tech.
___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -  http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


--
The time has come to start talking about whether the emperor is as well
dressed as we are supposed to think he is.   /Pete McBreen/
___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )