Re: Is async the answer
Graham Leggett wrote: > Davi Arnaut wrote: > >>> The proxy that the LiveJournal folks wrote, I think, copies all the data >>> from the origin server into a file and then uses sendfile to send to the >>> client... >> Doesn't this limit the network bandwidth to the bandwidth of the disk >> and/or file system? > > Yes, and the effective bandwidth of the disk can be significantly higher > than both the cache backend (which is often expensive) and the network > frontend (which has slow potential slow clients typing up your resources). > > Don't forget that your cache disk is most often RAM backed, meaning > effectively your cache disk is a ramdisk, with all the speed advantages > that go with it. > This is true for expensive hardware and very well designed operating systems and file systems.. and the space is not infinite. But... OK. Back to the topic I thought that one of the key points of async/event based servers were that we use software to scale and not hardware (so that hardware is not the bottleneck)... like serving thousands of slow clients from commodity hardware. -- Davi Arnaut
Re: Is async the answer
lör 2008-01-19 klockan 09:57 -0500 skrev Davi Arnaut: > Doesn't this limit the network bandwidth to the bandwidth of the disk > and/or file system? Depends on the working set and your amount of memory. If it's just temporary storage then no, as most data won't even hit the disk. If it's more of a cache then partially. Updates will use write bandwidth to the disks, and not so frequently accessed objects will use read bandwidth as well. the filesystem "bandwidth" is pretty negiable these days. Very close to raw I/O + memory cache. Regards Henrik signature.asc Description: Detta är en digitalt signerad meddelandedel
Re: Is async the answer
fre 2008-01-18 klockan 16:17 -0500 skrev Akins, Brian: > Paul Q and I have been kicking around the idea that even if we go to a > completely async core, etc. that modules could mark some hooks as "blocking" > and they would run basically how they do today. (One day, Paul, I'll > actually think about this more...) In the end you need a bit of mixture between the models to work out. threads or even processes for complex processing or libraries outside your control, and async for the basic core to keep it lightweight in resources/request/connection. There is quite a bit of research in programming models supporting mixed async/threaded/tasklet scheduling without forcing the programmer to know all details. Quite interesting reading if you haven't read those papers yet. For example the tame approach (C++ preprocessor using libasync) used by OKWS and it's related cousin tamer (more lightweight library) is quite fun to work with, at least in theory. Regarding CPU performance then you need a more complex workload than pure sendfile() shuffling of data to see much of a difference between threaded or async models. Especially if you look at smaller requests where the two almost diverge to the same model.. (N threads doing fast successive batch processing one request at a time with no wait time, or a event loop doing pretty much the same batching..). Regards Henrik signature.asc Description: Detta är en digitalt signerad meddelandedel
Re: [ANNOUNCEMENT] Apache HTTP Server 2.2.8 (2.0.63, 1.3.41) Released
Ruediger Pluem wrote: On 01/19/2008 06:04 PM, Jim Jagielski wrote: Apache HTTP Server 2.2.8 Released Can someone with the needed karma please add the version numbers 1.3.41, 2.0.63 and 2.2.8 to bugzilla? Done
Re: [ANNOUNCEMENT] Apache HTTP Server 2.2.8 (2.0.63, 1.3.41) Released
On 01/19/2008 06:04 PM, Jim Jagielski wrote: >Apache HTTP Server 2.2.8 Released Can someone with the needed karma please add the version numbers 1.3.41, 2.0.63 and 2.2.8 to bugzilla? Thanks Regards Rüdiger
[ANNOUNCEMENT] Apache HTTP Server 2.2.8 (2.0.63, 1.3.41) Released
Apache HTTP Server 2.2.8 Released
The Apache Software Foundation and the Apache HTTP Server Project
are
pleased to announce the release of version 2.2.8 of the Apache
HTTP Server
("Apache"). This version of Apache is principally a bug and
security fix
release. The following potential security flaws are addressed:
* CVE-2007-6421 (cve.mitre.org)
mod_proxy_balancer: Correctly escape the worker route and the
worker
redirect string in the HTML output of the balancer manager.
Reported by SecurityReason.
A flaw was found in the mod_proxy_balancer module. On sites
where
mod_proxy_balancer is enabled, a cross-site scripting attack
against
an authorized user is possible.
* CVE-2007-6422 (cve.mitre.org)
Prevent crash in balancer manager if invalid balancer name is
passed
as parameter. Reported by SecurityReason.
A flaw was found in the mod_proxy_balancer module. On sites
where
mod_proxy_balancer is enabled, an authorized user could send a
carefully crafted request that would cause the Apache child
process
handling that request to crash. This could lead to a denial of
service if using a threaded Multi-Processing Module.
* CVE-2007-6388 (cve.mitre.org)
mod_status: Ensure refresh parameter is numeric to prevent
a possible XSS attack caused by redirecting to other URLs.
Reported by SecurityReason.
A flaw was found in the mod_status module. On sites where
mod_status
is enabled and the status pages were publicly accessible, a
cross-site scripting attack is possible. Note that the server-
status
page is not enabled by default and it is best practice to not
make
this publicly available.
* CVE-2007-5000 (cve.mitre.org)
mod_imagemap: Fix a cross-site scripting issue. Reported by
JPCERT.
A flaw was found in the mod_imap module. On sites where
mod_imap is enabled and an imagemap file is publicly
available, a
cross-site scripting attack is possible.
We consider this release to be the best version of Apache
available, and
encourage users of all prior versions to upgrade.
Apache HTTP Server 2.2.8 is available for download from:
http://httpd.apache.org/download.cgi
Apache 2.2 offers numerous enhancements, improvements, and
performance
boosts over the 2.0 codebase. For an overview of new features
introduced
since 2.0 please see:
http://httpd.apache.org/docs/2.2/new_features_2_2.html
Please see the CHANGES_2.2 file, linked from the download page,
for a
full list of changes. A condensed list, CHANGES_2.2.8 provides the
complete list of changes since 2.2.6 (2.2.7 was not released). A
summary
of security vulnerabilities which were addressed in the previous
2.2.6
and earlier releases is available:
http://httpd.apache.org/security/vulnerabilities_22.html
Apache HTTP Server 1.3.41 and 2.0.63 legacy releases are also
currently
available. See the appropriate CHANGES from the url above. See the
corresponding CHANGES files linked from the download page. The
Apache
HTTP Project developers strongly encourage all users to migrate to
Apache 2.2, as only limited maintenance is performed on these legacy
versions.
This release includes the Apache Portable Runtime (APR) version
1.2.12
bundled with the tar and zip distributions. The APR libraries
libapr
and libaprutil (and on Win32, libapriconv) must all be updated to
ensure
binary compatibility and address many known platform bugs.
This release builds on and extends the Apache 2.0 API. Modules
written
for Apache 2.0 will need to be recompiled in order to run with
Apache 2.2,
and require minimal or no source code changes.
http://svn.apache.org/repos/asf/httpd/httpd/branches/2.2.x/
VERSIONING
When upgrading or installing this version of Apache, please bear
in mind
that if you intend to use Apache with one of the threaded MPMs
(other
than the Prefork MPM), you must ensure that any modules you will
be using
(and the libraries they depend on) are thread-safe.
[ANNOUNCEMENT] Apache HTTP Server 2.0.63 (2.2.8, 1.3.41) Released
Apache HTTP Server 2.0.63 Released
The Apache Software Foundation and the Apache HTTP Server Project
are
pleased to announce the legacy release of version 2.0.63 of the
Apache
HTTP Server ("Apache"). This Announcement notes the significant
changes in
2.0.63 as compared to 2.0.61 (2.0.62 was not released). This
Announcement2.0 document may also be available in multiple
languages at:
http://www.apache.org/dist/httpd/
This version of Apache is principally a bug and security fix
release. The
following potential security flaws are addressed:
* CVE-2007-6388 (cve.mitre.org)
mod_status: Ensure refresh parameter is numeric to prevent
a possible XSS attack caused by redirecting to other URLs.
Reported by SecurityReason.
A flaw was found in the mod_status module. On sites where
mod_status
is enabled and the status pages were publicly accessible, a
cross-site scripting attack is possible. Note that the server-
status
page is not enabled by default and it is best practice to not
make
this publicly available.
* CVE-2007-5000 (cve.mitre.org)
mod_imagemap: Fix a cross-site scripting issue. Reported by
JPCERT.
A flaw was found in the mod_imap module. On sites where
mod_imap is enabled and an imagemap file is publicly
available, a
cross-site scripting attack is possible.
Please see the CHANGES_2.0.63 file in this directory for a full list
of changes for this version.
This release is compatible with modules compiled for 2.0.42 and
later
versions. We consider this release to be the best version of
Apache 2.0
available and encourage users of all prior versions to upgrade.
This release includes the Apache Portable Runtime library suite
release
version 0.9.17, bundled with the tar and zip distributions. These
libraries; libapr, libaprutil, and on Win32, libapriconv must all be
updated to ensure binary compatibility and address many known
platform
bugs.
Apache HTTP Server 2.0.63 is available for download from
http://httpd.apache.org/download.cgi
Please see the CHANGES_2.0 file, linked from the above page, for
a full
list of changes. A condensed list, CHANGES_2.0.63 provides the
complete
list of changes since 2.0.61.
Apache 2.0 offers numerous enhancements, improvements, and
performance
boosts over the 1.3 codebase. For an overview of new features
introduced
after 1.3 please see
http://httpd.apache.org/docs/2.0/new_features_2_0.html
When upgrading or installing this version of Apache, please keep
in mind
the following: If you intend to use Apache with one of the
threaded MPMs,
you must ensure that the modules (and the libraries they depend
on) that
you will be using are thread-safe. Please refer to the
documentation of
these modules and libraries to obtain this information.
Apache 2.2 offers numerous enhancements, improvements, and
performance
boosts over the 2.0 codebase. For an overview of new features
introduced
after 2.0 please see
http://httpd.apache.org/docs/2.2/new_features_2_2.html
We consider Apache 2.2 to be the best available version at the
time of
this release. We offer Apache 2.0.63 as the best legacy version
of Apache
2.0 available. Users should first consider upgrading to the current
release of Apache 2.2 instead.
[ANNOUNCEMENT] Apache HTTP Server 1.3.41 (2.2.8, 2.0.63) Released
Apache HTTP Server 1.3.41 Released
The Apache Software Foundation and the Apache HTTP Server Project
are
pleased to announce the release of version 1.3.41 of the Apache HTTP
Server ("Apache"). This Announcement notes the significant
changes in
1.3.41 as compared to 1.3.39 (1.3.40 was not released).
This version of Apache is is principally a bug and security fix
release.
The following potential security flaws are addressed:
* CVE-2007-6388 (cve.mitre.org)
mod_status: Ensure refresh parameter is numeric to prevent
a possible XSS attack caused by redirecting to other URLs.
Reported by SecurityReason.
A flaw was found in the mod_status module. On sites where
mod_status
is enabled and the status pages were publicly accessible, a
cross-site scripting attack is possible. Note that the server-
status
page is not enabled by default and it is best practice to not
make
this publicly available.
* CVE-2007-5000 (cve.mitre.org)
mod_imap: Fix cross-site scripting issue. Reported by JPCERT.
A flaw was found in the mod_imap module. On sites where
mod_imap is enabled and an imagemap file is publicly
available, a
cross-site scripting attack is possible.
* CVE-2007-3847 (cve.mitre.org)
mod_proxy: Prevent reading past the end of a buffer when parsing
date-related headers. PR 41144.
With Apache 1.3, the denial of service vulnerability applies
only
to the Windows and NetWare platforms.
Please see the CHANGES_1.3.41 file in this directory for a full list
of changes for this version.
Apache 1.3.41 is the current stable release of the Apache 1.3
family. We
strongly recommend that users of all earlier versions, including 1.3
family release, upgrade to to the current 2.2 version as soon as
possible.
We recommend Apache 1.3.41 version for users who require a third
party
module that is not yet available as an Apache 2.x module. Modules
compiled
for Apache 2.x are not compatible with Apache 1.3, and modules
compiled
for Apache 1.3 are not compatible with Apache 2.x.
Apache 1.3.41 is available for download from
http://httpd.apache.org/download.cgi
This service utilizes the network of mirrors listed at:
http://www.apache.org/mirrors/
Binary distributions may be available for your specific platform
from
http://www.apache.org/dist/httpd/binaries/
Binaries distributed by the Apache HTTP Server Project are
provided as a
courtesy by individual project contributors. The project makes no
commitment to release the Apache HTTP Server in binary form for any
particular platform, nor on any particular schedule.
IMPORTANT NOTE FOR APACHE USERS: Apache 1.3 was designed for Unix OS
variants. While the ports to non-Unix platforms (such as Win32,
Netware or
OS2) will function for some applications, Apache 1.3 is not
designed for
these platforms. Apache 2 was designed from the ground up for
security,
stability, or performance issues across all modern operating
systems.
Users of any non-Unix ports are strongly cautioned to move to
Apache 2.
The Apache project no longer distributes non-Unix platform
binaries from
the main download pages for Apache 1.3. If absolutely necessary,
a binary
may be available at http://archive.apache.org/dist/httpd/.
Apache is the most popular web server in the known universe;
about 2/3 of
the servers on the Internet run Apache HTTP Server, or one of its
variants.
Apache 1.3.41 Major changes
Security vulnerabilities
The main security vulnerabilities addressed in 1.3.41 are:
CVE-2007-6388 (cve.mitre.org)
mod_status: Ensure refresh parameter is numeric to prevent
a possible XSS attack caused by redirecting to other URLs.
Reported by SecurityReason.
CVE-2007-5000 (cve.mitre.org)
mod_imap: Fix cross-site scripting issue. Reported by JPCERT.
CVE-2007-3847 (cve.mitre.org)
mod_proxy: Prevent reading past the end of a buffer when parsing
date-related headers. PR 41144.
With Apache 1.3, the denial of service vulnerability applies only
to the Windows and NetWare platforms.
Bugfixes addressed in 1.3.41 are:
More efficient implementation of the CVE-2007-3304 PID table
patch. This fixes issues with excessive memory usage by the
parent process if long-running and with a high number of child
process forks during that timeframe. Also fixes bogus "Bad pid"
errors.
Re: Is async the answer
Davi Arnaut wrote: The proxy that the LiveJournal folks wrote, I think, copies all the data from the origin server into a file and then uses sendfile to send to the client... Doesn't this limit the network bandwidth to the bandwidth of the disk and/or file system? Yes, and the effective bandwidth of the disk can be significantly higher than both the cache backend (which is often expensive) and the network frontend (which has slow potential slow clients typing up your resources). Don't forget that your cache disk is most often RAM backed, meaning effectively your cache disk is a ramdisk, with all the speed advantages that go with it. Regards, Graham -- smime.p7s Description: S/MIME Cryptographic Signature
Re: Is async the answer
On Jan 18, 2008, at 2:16 PM, Justin Erenkrantz wrote: On Jan 18, 2008 10:52 AM, Akins, Brian <[EMAIL PROTECTED]> wrote: Which is why I hate to see a ton of work go into async core if it actually does very little to help performance (or if it hurts it) and makes writing modules harder. It braindead simple nowadays to write well behaved high performance modules (well, mostly) bcs you rarely worry about threads, reads/writes, etc. Full async programming is just as challenging as handling a ton of threads yourself. Speaking for myself, I think writing and using buckets with serf is more straightforward than our complicated bucket brigade system with mixed push/pull paradigms. +1... Although the whole concept of buckets and their brigades has some cool advantages, they are also a semi-constant source of issues...
Re: Is async the answer?
On Jan 18, 2008, at 12:03 PM, Akins, Brian wrote: This is just some ramblings based on some observations, theories, and tests. Partially "devil's advocate" as well. Most of us seem to have convinced our self that high performance network applications (including web servers) must be asynchronous in order to scale. Is this still valid? For that matter, was it ever? http://www.jimjag.com/imo/index.php?/archives/150-Long-time.html
Re: Is async the answer
Akins, Brian wrote: > On 1/18/08 3:07 PM, "Colm MacCarthaigh" <[EMAIL PROTECTED]> wrote: >> That's not even a consideration, >> async is really for dynamic content, proxies, and other non-sendfile >> content. > > For dynamic stuff, "X-sendfile" works well. (Just really starting to play > with that, liking it so far). > > The proxy that the LiveJournal folks wrote, I think, copies all the data > from the origin server into a file and then uses sendfile to send to the > client... Doesn't this limit the network bandwidth to the bandwidth of the disk and/or file system? -- Davi Arnaut
Re: [RESULT] Was: Re: [VOTE] Apache HTTP Server 1.3.41, 2.0.63 and 2.2.8
On 01/19/2008 03:36 PM, Jim Jagielski wrote: > I will be announcing the release today! > > Great! Thanks for all the RM work you have done again. Regards Rüdiger
Re: [RESULT] Was: Re: [VOTE] Apache HTTP Server 1.3.41, 2.0.63 and 2.2.8
I will be announcing the release today!
Re: Is async the answer
Akins, Brian wrote: The proxy that the LiveJournal folks wrote, I think, copies all the data from the origin server into a file and then uses sendfile to send to the client... The proxy enhancements that Niklas contributed do exactly this as well. It has a number of other advantages, such as slow clients not tying up a fast backend. It works very well. Regards, Graham -- smime.p7s Description: S/MIME Cryptographic Signature
Re: Is async the answer
On Fri, 18 Jan 2008, Ruediger Pluem wrote:
The proxy that the LiveJournal folks wrote, I think, copies all the data
from the origin server into a file and then uses sendfile to send to the
client...
Erm, so does the one we wrote, mod_disk_cache ;p
IMHO it doesn't for the first request of the entity (the request that causes
the entity to be cached)
Which is why it doesn't scale with large files, and I hacked it to do
that to be usable with DVD images on ftp.acc.umu.se
(http://issues.apache.org/bugzilla/show_bug.cgi?id=39380 - you might
remember the first try to merge some of it). Yes, it has its flaws,
but it solves the problem for us. I think that some people has tried
it in a proxy setting too with pretty OK result. But this was really
off-topic ;)
Getting to the point, I share Brians concerns with going async just
for the async sake, for similar reasons:
- People are having problems with making modules even thread safe
(see mod_example), forcibly adding async to the mix will raise
the bar even higher for people who needs to whip up a simple module.
- Callback semantics are messy when they go wrong, debugging can be a
pain.
- Threads are rather cheap, even on linux since the advent of NPTL.
- Performance benefits are unclear.
Given that, there are obvious optimisations that can be, and have
been, made. The ones in trunk aimed at not hogging a worker thread for
simply writing the remaining data to the client for example. From what
I've understood this class of changes doesn't really affect modules.
Also, if there is a way of adding async having it optional in modules
then I see no problem with adding it as long as there are cases where
it actually helps, other than adding it to the supported buzzwords
list ;)
/Nikke - who probably ended up off topic after all ;)
--
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Niklas Edmundsson, Admin @ {acc,hpc2n}.umu.se | [EMAIL PROTECTED]
---
I'd love to, but I'm worried about my vertical hold.
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
