Re: [AOLSERVER] More with the Chinese translations...
On Fri, Jun 26, 2009 at 12:04 AM, Janine Sisk wrote:
>
> set translated_page_body [encoding convertto gb2312 $page_body]
This isn't going to work. It's not the encoding of the characters you
need to change but the characters themselves.
You want to do the equivalent of this:
% string map {h H w W} "hello world"
Hello World
Where 'h' and 'H' are similar, but distinct, letters and the encoding
hasn't changed (happens to be utf8 in my terminal, could be ascii
etc.).
In fact, looking at the source code for the software at
mandarintools.com that's all they're doing. It's a poor quality
conversion, according to what I've read, but if it's sufficient well
hey!
The file hcutf8.txt in the .zip source bundle contains a mapping of
simplififed to traditional characters. The first character on each
line (that is not a comment) is the simplified character, followed by
one or more traditional candidate characters.
You could create a Tcl list in the format string-map is expecting,
with each of the traditional characters followed by the simplified
character. Without using a proxy setup, simply map the
utf8-traditional response body into utf8-simplified and send directly
to the browser.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject:
field of your email blank.
Re: [AOLSERVER] More on caching content...
On Thu, Apr 23, 2009 at 7:40 PM, Janine Sisk wrote: > Trying again... anyone??? Bueller? :) Is this tc2sc conversion software actually any good? I remember you were having the same problem 18 months ago, and you said that the company that wrote the software has gone out of business and you don't have the source code. The world expert on c2c conversion says that many such programs do only a simple 'level 1' conversion: http://www.cjk.org/cjk/c2c/c2cbasis.htm If that's the case for the program you're using, throw it away! You could perform an equally poor level 1 tc2sc conversion using Tcl's built-in charset api easily. Level 2 wouldn't be much harder. The advantage however would be that you would not have to wrestle with defunct java software and performance-sucking proxies. Maybe you could figure out how awesome the software is by feeding it some of the examples from the above web page and see how it measures up? -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] problem getting binary data from C to a tcl object to an aolserver connection
On Fri, Aug 29, 2008 at 3:18 AM, Mark Aufflick <[EMAIL PROTECTED]> wrote: > Hi Stephen, > > What does ns_conn write_encoded false do (although it is somewhat self > explanatory)? With the write_encoded flag set (which ns_startcontent sets) ns_write will assume you're sending character data and will want to encode it into iso-8859-1, or whatever is configured. With the write_encoded flag off, ns_write will use Tcl_GetByteArrayFromObj() rather than Tcl_GetStringFromObj(), and because you used Tcl_NewByteArrayObj() in your C code, your bytes will pass through unmolested. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] problem getting binary data from C to a tcl object to an aolserver connection
On Thu, Aug 28, 2008 at 9:50 AM, Mark Aufflick <[EMAIL PROTECTED]> wrote: > Hi all, > > If I Ns_Log() the data in a (char *) I can clearly see that it > contains newlines, and I can also verify that it contains nulls with > memchr. > > I have tried any number of ways to turn it into a tcl object, eg: > >objPtr = Tcl_NewByteArrayObj(str, length); This is the right way to do it. > Whether the tcl code then does an ns_log, ns_return (which I > know isn't supposed to be binary safe) or ns_write, i get all the > newlines converted into \n (ie. two characters \ then n) and chunks of > binary get converted to unicode characters. In AOLserver 4.5, ns_write is the only command that accepts a binary data. You also need to be careful you don't accidentally change it's type once you've created it. set blob [myblobcmd] set length [string length $blob] # blob now garbled utf8 :-( > I can see from Tcl_AppendToObj that that is supposed to happen there, > but how can I output a byte array object without it being converted to > utf8? Something like... ns_startcontent -type application/octet-stream ns_conn write_encoded false ns_write $blob -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Getting a handle on memory usage
On Fri, May 2, 2008 at 6:46 PM, Rick Cobb <[EMAIL PROTECTED]> wrote: > > Just to add something to the discussion: One point I think is confusing > about the C API, and needs more amplification in the documentation, is > the asymmetry between Ns_TclAllocateInterp and Nn_TclDeAllocateInterp. > In our C/C++ modules, we often need to "get the current interp and run > this Tcl command", which may cause recursion back into our module(s). > And our module can be used from either conn threads or non-conn (timer, > multicast reception, etc) threads. We don't want to leak Interps, but we > also don't want to leak ns_sets and other interp-specific Tcl contexts > that should be cleaned up between requests. Allocate single interp per-server per-thread: http://sourceforge.net/tracker/index.php?func=detail&aid=1241351&group_id=130646&atid=719006 Here's the history of nsd/tclinit.c for naviserver and aolserver, if you're looking to compare/port etc. Both have seen lots of changes but have been kept reasonably well in synch. http://freehg.org/u/groks/naviserver/log/fad41c08d28b/nsd/tclinit.c http://freehg.org/u/groks/aolserver/log/9a2383ff0ca5/nsd/tclinit.c (The above are cleaned up conversions of the cvs repos. It's easier to browse history) -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
[AOLSERVER] nsdbi database driver interface 0.1
nsdbi is a database driver interface for naviserver. It provides native bind variable support, transparent prepared queries and handle management, runtime configuration, statistics, and a few other things. There are drivers for postgres, mysql and sqlite. There's also a stub driver included for testing, fault injection etc. It relies on a few utility routines in naviserver, but it wouldn't be too hard to port those to aolserver, should anyone want to. Here's a snapshot of the man page: http://www.groks.org/nsdbi.n.html Tarballs: http://www.groks.org/nsdbi-0.1.tgz http://www.groks.org/nsdbipg-0.1.tgz http://www.groks.org/nsdbimy-0.1.tgz http://www.groks.org/nsdbilite-0.1.tgz Mercurial repositories: http://freehg.org/u/groks/nsdbi/ http://freehg.org/u/groks/nsdbipg/ http://freehg.org/u/groks/nsdbimy/ http://freehg.org/u/groks/nsdbilite/ -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] nssqlite3 troubles
On Feb 7, 2008 9:32 AM, Jeff Rogers <[EMAIL PROTECTED]> wrote:>
>
> The problem is straightforward - the driver checks how many rows are
> returned after executing a statement; 0 returns NS_DML and >0 returns
> NS_ROWS. 'ns_db select' and '0or1row' expect NS_ROWS results, even if
> there are no rows to be read (which is a perfectly valid result).
>
> So how to fix it? There's no direct way to tell what kind of statement
> was parsed in the sqlite api (unlike say, OCI) and checking how many
> rows were changed will give bad results for an update affecting no rows.
sqlite3_stmt *st;
sqlite3_prepare_v2(handle, sql, length, &st, &tail)
if (sqlite3_column_count(st) == 0) {
/* DML */
}
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]>
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject:
field of your email blank.
Re: [AOLSERVER] aolserver vs lighthttpd, benchmarks
On 10/3/07, Tom Jackson <[EMAIL PROTECTED]> wrote: > Thanks, that worked. The -command was executed logging 'okay' + number of > bytes written. > > But the ns_log statements following vwait are not executed. I assume that is > expected? I posted the updated script at: > > http://rmadilo.com/files/nsbgwrite/nsbgwrite.tcl fcopy $inFd $channel -command [list ns_log Notice "okay"] vwait $channel vwait is expecting a variable *name*. Here, it's waiting for the variable 'sock23' (or whatever) to be set, which never happens. There's a short example on the man page: http://www.tcl.tk/man/tcl8.4/TclCmd/vwait.htm -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] aolserver vs lighthttpd, benchmarks
On 10/3/07, Tom Jackson <[EMAIL PROTECTED]> wrote: > Anyway, I don't really know if this will work as the original, I was able to > fcopy in background a short text file, but larger than 4096 bytes just gets > that amount according to wget. > > In foreground/blocking mode, fcopy returns larger files. > > I've included my test script as well: > > http://rmadilo.com/files/nsbgwrite/nsbgwrite.tcl By using the -command option to fcopy you've enabled background mode, but I don't see a call to vwait. You're not entering the event loop. 4096 happens to be the default buffer size of a Tcl channel. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] aolserver vs lighthttpd, benchmarks
On 10/2/07, Tom Jackson <[EMAIL PROTECTED]> wrote: > ... it would be nice if new code followed AOLserver coding norms. Maybe > you can get naviserver to take out their code. The module I wrote at least > compiled against their server... You're using symbols declared in nsd/nsd.h, which is private to the server and not installed by default. Things in here change regularly. You're asking for trouble... > ...but nsd doesn't run on my 64bit laptop: > > [02/Oct/2007:10:37:45][6858.690331232][-main-] Fatal: NsTclInitObjs: > sizeof(int) < sizeof(long) > Aborted That's *really* old code you're running... ...which is our fault for not making a release. Sorry. You'll have more joy running cvs head and reporting bugs on the mailing list: http://naviserver.sourceforge.net/w/Mailing_Lists -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] configured minthreads, maxthreads doesnt show up with [ns_server threads] command
On 8/4/07, Tom Jackson <[EMAIL PROTECTED]> wrote: > Stephen, > > I agree that you have clearly defined one use of threadpools, essentially > overall limits. But note that threadpools are tied to url patterns. The > concept is that certain urls consume more resources that others, not just > memory, but processor time. Other urls consume few resources, maybe just > static content. Then there could be differences between a development version > of a site and a production version running in the same nsd process. I agree, there are many possible configurations (some more useful than others)... > There is more than one use for threadpools, which > is the original reason for the threadpool maps: prior to threadpools, there > were per-virtual-server settings. This is what was removed and replaced with > the current code. The old way didn't absolve you from setting threads correctly. If you had two virtual servers and you thought the ideal number of threads for an nsd process on a particular machine was 20, you give each v-server 10 (or some other portion). But here's the problem: server1 gets slashdotted, uses up all it's conn threads, and starts rejecting connections. Meanwhile server2 sits idle with spare threads. You could say: well, give each server 20 threads if that will max out the server. Now both servers get slashdotted and response time goes through the roof. Threads are a global resource. > Maybe an > admin section needs to always have available threads if the whole server is > jammed up for some reason. Use ns_pools and ns_limits in combination. You could, for example, create a pool with 20 threads and map both servers to it. Create a limit with maxrun 15 and map both servers to it. Now, when server1 overloads there will still be 5 threads left for the other v-server. Use pools to separate by type: threads with interps vs. without, error pool, etc. Use limits to implement some QOS policy, such as the /admin example. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] configured minthreads, maxthreads doesnt show up with [ns_server threads] command
On 8/3/07, Tom Jackson <[EMAIL PROTECTED]> wrote:
> Stephen,
>
> My concern over this is that one virtual server might have slightly different
> code, for instance development vs. production, and would likely share the
> same threadpool. What exactly is part of the thread in a pool? Is there any
> remaining code or data from one use to another, or is all data in the Tcl
> interp?
Yes, each virtual server gets it's own interp in each thread. There is
no sharing between virtual servers.
> The problem with ns_pools is that without explicit configuration two process
> wide threadpools are created: 'default' and 'error'. Now if any virtual
> server sets up a threadpool named 'default', the new limits apply to the
> process wide 'default' threadpool.
Don't do that? :-)
Thread pools are process-wide because memory (n * interps per thread),
context switch overhead and caching effects, balanced against latency
and paralelism etc. and so on, are properties of the system as a
whole, not 1 of n virtual servers.
To optimise a global resource you need a global overview.
Anyway, here's a snippet from NEWS (same applies to pools):
* New sections for server limits:
ns_section "ns/limits"
ns_param default "Default Limits" ;# Defines a limit.
ns_section "ns/limit/default"
ns_param maxrun 100 ;# Conn threads running for limit.
ns_param maxwait 100 ;# Conn threads waiting for limit.
ns_param maxupload 10240 ;# Max size of file upload in bytes.
ns_param timeout 60;# Total seconds to wait for resources.
ns_section "ns/server/server1/limits"
ns_param default "GET /*" ;# Map default limit to URL.
ns_param default "POST /*"
ns_param default "HEAD /*"
Limits (pools) are explicitly set globally. The only thing to do in
the per-virtual server section (ns/server/*) is to map existing pools
to URLs.
It isn't working for you because you have the definition of the pools
per-virtual server, whereas you should just be mapping servers to
pools. Works for me.
A change of config file syntax is not the end of the world. You could
always fall back to the old config locations if "ns/pools" doesn't
exist, and implement 'ns_runonce -global { ... }' (or whatever) to
prevent toe stepping.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]>
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject:
field of your email blank.
Re: [AOLSERVER] configured minthreads, maxthreads doesnt show up with [ns_server threads] command
Hi,
I think it's a fine idea that conn thread pools are a process-wide
resource. I liked the idea so much, I copied it. (Well, the limits
part, so far...)
But I agree with you; although ns_pools/ns_limits enable some handy
new features, such as dynamic configuration, it sure would be
convenient to be able to set this from the config file.
Here's how that works in NaviServer:
http://naviserver.cvs.sourceforge.net/naviserver/naviserver/tcl/config.tcl?view=markup
'ns_runonce -global { ... }' is used to get around the issue you've
noticed with overwriting. AOLserver has an 'ns_ictl once' command, but
IIRC it's once per-virtual server only, I'm afraid.
Couple of other misconceptions/questions were raised:
Tcl interps belong to the thread which created them. If there's more
than one virtual server configured, there's one interp for each
server, per-thread.
Tcl objects are reference counted. The Tcl documentation hints that
when your C implementation of a command begins, the interp result is
in a pristine state. it's value is "". You might also expect it to be
unshared. But sometimes it's not!
So you can't do the obvious:
Tcl_SetIntValue(Tcl_GetInterpResult(interp), 1);
and instead have to:
Tcl_SetObjResult(interp, Tcl_NewIntValue(1));
It's tempting to do the first because it looks faster. Anywho, it's
not a threading issue.
Hope that helps.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]>
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject:
field of your email blank.
Re: [AOLSERVER] Charset problem
The dump will either contain literal ASCII question marks because Oracle has decided to clip the high bits during export, or it will contain mangled byte sequences which don't correspond to any character which can be displayed on your screen. If Oracle clipped, your out of luck. Otherwise, you can fix this the wrong way and go to bed by using tr, sed, or emacs query-replace. If your data is mainly English with some special characters, such as in resumé, and Microsoft 'smart' quotes etc., it shouldn't be too hard to identify which mangled bytes should be translated into which UTF8 characters. You can get a rough handle on the size of the problem by running something like: LC_LANG=C grep -v -U -c '[[:alnum:]|[:punct:]|[:space:]]' my.dump On 7/19/05, Janine Sisk <[EMAIL PROTECTED]> wrote: > Hi all, > > I've just made a mess for myself and I'm hoping someone will know how > to fix it. It's really more of an Oracle problem and the message below > is a modified version of one i just sent to an Oracle list, but I > thought perhaps someone here would have already struggled with it. > > I took a site that was running under 8.7.1.4 and moved it to 9.2.0.4 > (both on RedHat Linux) using exp/imp. I didn't specify a character set > in either case. The data has accented characters and they have been > working fine in 8.1..7.4. > > Now, it seems that the default setting of NLS_CHARACTERSET in 8.1.7.4 > was US7ASCII and in 9.2.0.4 it's WE8ISO8859P1. Everything I've read > about this conversion says that since it's going from 7 bit to 8 bit > there shouldn't be any data problems. Well, hah! :) We didn't spot > any at first, but now that the client is looking closely he's finding > pages all over the place that have ?? where accented characters should > be. > > The problem was even worse at first; some characters displayed ok > until you edited the page via the web browser, and then they turned > into ?? as well. I was able to fix that, as far as I can tell, by > setting NLS_CHARACTERSET to WE8ISO8859P1 in the environment of the user > running the site. It has not, unfortunately, helped us with the rest > of the mess. > > AOLserver is configured to use iso-8859-1 for it's charset and has been > all along. The only thing that has changed here is the Oracle version > and it's charset. I have this in the ns/parameters section: > > ns_param HackContentType1 > ns_param DefaultCharset iso-8859-1 > ns_param HttpOpenCharsetiso-8859-1 > ns_param OutputCharset iso-8859-1 > ns_param URLCharset iso-8859-1 > > Going back and reimporting the data is a last resort, as we'd either > lose or recreate user data that has been entered since the site was > moved on Sunday night. Is there anything else I can do to fix this? > > In short, heeelp! :) > > thanks, > > janine > > > -- > AOLserver - http://www.aolserver.com/ > > To Remove yourself from this list, simply send an email to <[EMAIL > PROTECTED]> with the > body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: > field of your email blank. > -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Proposed change to "internal redirects" behavior
On Fri, 11 Feb 2005 16:21:49 -0500, Dossy Shiobara <[EMAIL PROTECTED]> wrote: > On 2005.02.11, Stephen Deasey <[EMAIL PROTECTED]> wrote: > > > Why introduce Ns_ConnSetUrl()? Why not change Ns_SetRequestUrl() to do > > > this? > > > > Ns_SetRequestUrl(Ns_Request * request, char *url); > > Ns_ConnSetUrl(Ns_Conn *conn, char *url); > > > > The 'original URL' is a property of the current connection, not a > > Ns_Request structure which is generic and has no back pointer itself > > to an Ns_Conn. I thought was cleaner to add the new Ns_ConnSetUrl > > call, which obviously takes a pointer to the current conn. > > Very cool. I thought that the private Request structure (similar to how > we have a private Conn and a public Ns_Conn structure pair) might have a > pointer back to the Conn/Ns_Conn that the request is being processed by. > I just looked and that's not the case. > > Would it be better to add a pointer to the Conn in the private Request > structure rather than introducing a new (and similar) C API to one that > already exists? I don't think so. Adding a dependency on Ns_Conn to Ns_Request would make the code lex flexible. At the moment Ns_Request can be re-used in other situations, and adding that dependency would complicate that. > I'd be in favor of having the request processor set the originalUrl > attribute of the Conn instead of only changing it when Ns_ConnSetUrl() > is called. So, in the case where no "redirect" has happened, > originalUrl == url will be true. Then, we don't need to introduce this > new Ns_ConnSetUrl() at all? You would have to hope that 404 and 500 errors are a relatively rare event on a production site, so the overhead of copying the URL twice, once for Ns_Request.url and again for Conn.originalUrl, is excessive IMO. In the supplied patch, Conn.origUrl is computed lazily only when Ns_ConnSetUrl() is called. The only way to retrieve it is via Ns_ConnOriginalUrl(), which simply returns Ns_Request.url in the common case where Ns_ConnSetUrl() hasn't been called. If you decide instead to expose originalUrl as a public member of the Ns_Conn struct, then things would be different. But that seems unnecessarily complicated, and the majority of the public members of Ns_Conn have simple wrapper functions anyway. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Proposed change to "internal redirects" behavior
On Fri, 11 Feb 2005 15:20:26 -0500, Dossy Shiobara <[EMAIL PROTECTED]> wrote:
>
> Can you explain this:
>
> - connPtr->responseStatus = status;
> +/* 200 is default. Don't stomp custom redirects. */
> +if (status != 200) {
> +connPtr->responseStatus = status;
> +}
>
> What if the custom redirect isn't found (status = 404) - why should that
> stomp the original (say, connPtr->responseStatus == 500)?
Botched config files is something I didn't explore. In practice what
happens is 3 consecutive error messages are logged:
Error: return: failed to redirect '404': exceeded recursion limit of 3
In which case you should fix your config file :-) But yes, it might
be nice to be more explicit here and make sure the correct reponse
code is returned until the admin can get to it. I believe Apache
handles this.
> > You may also be interested in this related patch which alows a custom
> > redirect handler to know the original URL as sent by the browser and
> > act accordingly:
> >
> > "ns_conn seturl newUrl"
> > http://sourceforge.net/tracker/?func=detail&aid=1013672&group_id=3152&atid=353152
>
> Why introduce Ns_ConnSetUrl()? Why not change Ns_SetRequestUrl() to do
> this?
Ns_SetRequestUrl(Ns_Request * request, char *url);
Ns_ConnSetUrl(Ns_Conn *conn, char *url);
The 'original URL' is a property of the current connection, not a
Ns_Request structure which is generic and has no back pointer itself
to an Ns_Conn. I thought was cleaner to add the new Ns_ConnSetUrl
call, which obviously takes a pointer to the current conn.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]>
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject:
field of your email blank.
Re: [AOLSERVER] Proposed change to "internal redirects" behavior
Hi,
I fixed this bug 6 months ago. There's a patch on SourceForge:
"Invalid response status logged for custom redirects"
http://sourceforge.net/tracker/?func=detail&aid=1013752&group_id=3152&atid=103152
You may also be interested in this related patch which alows a custom
redirect handler to know the original URL as sent by the browser and
act accordingly:
"ns_conn seturl newUrl"
http://sourceforge.net/tracker/?func=detail&aid=1013672&group_id=3152&atid=353152
On Fri, 11 Feb 2005 09:02:42 -0500, Dossy Shiobara <[EMAIL PROTECTED]> wrote:
> Hi,
>
> AOLserver has "internal redirects" that allow overriding of some
> (currently not all, strangely) responses based on the HTTP status code.
> The configuration section looks like this (from the Annotated Config.
> Reference):
>
> #
> # Internal redirects
> #
> ns_section "ns/server/${servername}/redirects"
> ns_param 404 "/notfound.html" ;# Not Found error page
> ns_param 500 "/servererror.html" ;# Server Error page
>
> I have two proposed changes:
>
> 1) Allow overriding of all response codes. This may not actually be a
>good idea, but it just seems odd that only certain response codes
>have their own C API which uses the internal redirect mechanism.
>
> 2) The current internal redirect mechanism changes the URL in the
>request then re-executes it. The downside of the current
>implementation is that a custom error handler for 500 Server Error,
>if it's defined and exists and is served, turns the request into a
>200 OK. The HTTP response becomes a 200 OK instead of the original
>500 Server Error with the custom HTTP response body. nslog also logs
>a 200 OK in the access log, instead of the 500. This just feels
>wrong to me. If you have a custom 404 page, you can't use the access
>log to report on the number of 404's any more since the log will show
>200's for the 404 requests.
>
> I'm still on the fence whether #1 is a good idea or not. In principle
> it's the right thing to do, but implementing it properly will require a
> lot of refactoring of nsd/return.c to make it clean, and even then
> there could be problems.
>
> I'm definitely in favor of making the chnage described in #2, however
> it's a functionality change that will invalidate people's assumptions
> who use internal redirects and I don't know what effect that will have
> on people's applications, especially logfile analyzers, etc. I'm keen
> on implementing this change for 4.1.0, but what I really want to know is
> do people think it's okay to backport this change to 4.0.x? I really
> want to, but if everyone else thinks it's a bad idea ... then I can
> leave it as a 4.1.0-only change.
>
> Thoughts? Problems?
>
> -- Dossy
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]>
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject:
field of your email blank.
Re: [AOLSERVER] AOLserver facelift.
On Tue, 8 Feb 2005 09:06:21 +0100, Zoran Vasiljevic <[EMAIL PROTECTED]> wrote: > This is my wish-list. Do whatever you consider appropriate. > It is *far* from complete, but I guess I have to start with something. > (I have no reason of hiding it, I'm not secret service): Interesting list. Here's what I've been up to and what I'm planning: I've incorporated all the bug fixes and feature request I listed on sourceforge, plus the following additional items: Added a simple cookie API in C and Tcl. Ns_ParseObjv: A routine to ease writing Tcl procs in C by parsing switches, converting data types, writing error/usage messages etc. Created a Tcl callback infrastructure. This makes it easy to create Tcl versions of the many C hooks. Converted some existing code (tclresp.c, tclsched.c etc) to use this. Modified interp allocation to enter interps recursively. This prevents multiple interps per thread being allocated needlesly for the case when you call some command in Tcl which is a hook function, and the hook itself is implemented is Tcl. Virtualised page root handling. You can now register a function (C or Tcl) which returns the page root for the current server. This is *not* the same as url2file as the latter depends on the URL (but see my patches on sf where I've extended this). Create a simple nsmassvhost module which alters the page-root depending on the HTTP Host header. No config file entry needs to be made ahead of time, making it efficient to handle many virtual hosts. Incomplete: At the moment I'm attempting a re-write of Ns_Cache, i'm about 70% done. A much simpler interface is presented: Ns_CacheEval(), inspired by the Tcl module. No explicit locking is required. The Tcl API has been incorporated. The code is much smaller. I've written a new database driver manager, nsdbi. It's functional and aprox. 80% complete, although there are a lot of extra features that I'd like to add. Has native support for bind variables, doesn't use ns_set, gathers statistics, presents a nice Tcl API etc. TODO: Re-introduce the old Ns_ModLog or something similar. *everyone* re-writes this in Tcl. Authentication and authorization API. Important for multi-protocol work. More introspection, for example registered procs/filters (Rob Seeger mentioned this) Config parsing work: better handling of defaults, a command line switch to dump current state, etc. Tom mentioned this. Export more functionality to libnsd. I'm thinking that dummying up a conn request environment will allow easier testing. Plus a whole lot more. There's really a lot of great things that could be done. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] AOLserver facelift.
On Mon, 7 Feb 2005 16:13:53 -0500, Dossy Shiobara <[EMAIL PROTECTED]> wrote: > Okay, this is good. So, how do we define the vision? I'm in favor of > it being a shared vision between all the constituents of the AOLserver > Community, which was my rationale behind the Steering Committee concept. > Individuals from the community speak on behalf of users they represent > and express what their needs are. Someone (most likely me) will collate > all of these needs into a cohesive vision which will guide the direction > of the project. It's obvious what the AOLserver community wants, the lists are here: http://sourceforge.net/tracker/?group_id=3152&atid=353152 and here: http://sourceforge.net/tracker/?group_id=3152&atid=103152 What hasn't been expressed is AOL's intentions. Look, if the patches which have been submitted are not suitable for the AOLserver project, simply tell folks "thanks, but no thanks". Six months ago, Dossy Shiobara PANOPTIC.COM> wrote: > Keep in mind that Jim is *extremely* busy. Most likely, he'll tell me > about changes he's working on and I'll be responsible for messaging out > to the community about it, answering questions about his changes, etc. Can't you just sit down and have lunch with the guy and ask him what's up? You expressed recently on #aolserver that you're aware of some pending changes. Is it top secret? > What is the main reason why you use AOLserver? > > Perhaps it would be useful for everyone who uses AOLserver to write a > single sentence or single paragraph that starts out with "I use > AOLserver because ..." You're having a laugh, right? Can I send this directly to you directly or should this pass through my official representative on the AOLserver Vision Committee? -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] core server design docs
On Sun, 6 Feb 2005 23:05:59 -0800, Tom Jackson <[EMAIL PROTECTED]> wrote: > On Sunday 06 February 2005 12:03, Stephen wrote: > > > AOLserver doesn't do scatter/gather IO. > > > Hmm, well in nssock.c, there is this note: > * SockProc -- > * > * Socket driver callback proc. This driver attempts efficient > * scatter/gatter I/O if requested and only blocks for the > * driver configured time once if no bytes are available. "if requested". The iovec is only ever given one buffer, so there's nothing to scatter or gather. > > Read-ahead refers to the driver thread accepting new connections and > > reading data from the socket before passing it to a connection thread > > to run. This beneficially prevents precious connection threads from > > blocking for data as it dribbles in over a 14k modem. > > > > Read-ahead is more aggressive in 4.0 because it reads the entire > > request, not just the request line and headers as was the case in 3.x. > > That's why the new upload limit of 2MB which caught a lot of people > > I think we are referring to the same thing here. The socket driver uses > scatter/gather, but this is transparent to the application. Two different things. Scatter/gather IO uses multiple separate buffers to send or receive data as a contiguous chunk in a single system call. It's used to prevent data copying and extra memory usage and/or the overhead of crossing the kernel/user-space boundary many times for multiple system calls. Read-ahead is the timing/location of the data read from the client. Non-blocking IO from multiple sockets is efficiently multiplexed in the single driver thread. It would be wasteful to have a conn thread (with it's multi-MB cached Tcl interps) block for IO while data crosses the 'Net. There's context switch overhead, too as each conn thread wakes up to read another packet of data. Well, that's how I read it... -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
The difference between blocking disk I/O and other sources of conn thread 'blocking' is that the I/O problem can be solved with a small number of light weight threads which take the blocking on the chin for you, but there's not much you can do about arbitrary script processing. I don't think the read-the-whole-file-in-the-conn-thread solution scales very well when faced with a directory of CD images or a distribution of Debian packages :-) Dragging this back to the topic at hand, what this highlights to me is that infrastructure can be tricky, so you don't want to be re-implementing it ad-hoc. The Ns_ParseProc interface allows new protocol implementations to take full advantage of AOLserver's cool infrastructure, transparently. Cheers. On Thu, 2004-08-19 at 06:14, Jim Wilcoxson wrote: > I guess I was unclear in my earlier note: > > 1. The HTTP driver thread reads all input for request, w/o blocking. > My understanding is that this is now happening in AS 4 - great! > > 2. A conn service thread creates the output, either by running a TCL > script, executing fastpath to read a file into memory, etc. This > might do stats, file I/O, database I/O, etc., and may block. The > result of this thread is a memory buffer. > > 3. The HTTP driver thread spools the memory buffer back to the client, > w/o blocking. > > I wasn't suggesting that there should be no blocking in step 2, when > the output is being created. That's why multiple connection service > threads are needed. But the problem of a connection service thread > being tied up while it spools a large page/image is eliminated. > > In the application of an image server, it would be nice if there could > be one fastpath thread that did all non-blocking filesystem I/O > (actually just reading is needed), or an asynchronous sendfile, but I > agree with you that there are not good portable ways to do that yet. > > Jim > ge. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
On Wed, 2004-08-18 at 08:43, Robert Seeger wrote:
>
> ...
>
> Constraints:
> - Must be decoupled from the HTTP driver, so that that driver can be
> changed or replaced at any time without impacting any added drivers
I don't think my patch imposes a maintenance burden. It inserts the
following piece of code into the fastpath of the driver thread at the
end of driver.c:SockRead:
if (drvPtr->parseProc == NULL) {
return SockParse(connPtr);
}
i.e. if an alternative driver has not been registered, just carry on as
normal and parse the buffer of data expecting an HTTP request. This is
pretty trivial.
What follows it is a couple of lines of setup code which makes the
just-received buffer available to the Ns_ConnGets, Ns_ConnRead,
Ns_ConnReadLine, Ns_ConnContent etc. APIs.
The parseProc which implements the alternate protocol is then called.
Here's it's signature:
int (Ns_ParseProc) (void *arg, Ns_Conn *conn);
Pretty simple. This is the interface we're committing to support into
the future. I don't think I'm tying any body's hands here.
The first arg is just module specific callback data, the second a
pointer to the widely used Ns_Conn structure. That's not going away
anytime soon.
The return code can be one of: NS_SOCK_READY, NS_SOCK_MORE,
NS_SOCK_ERROR. Even if the internals of the driver thread were to
change radically, I don't think it would be too hard to continue
supporting this. Conversely, I don't think it would be too hard for
module writers to use an alternate API, considering it's small
footprint. Bigger changes have been made in the past.
And that's about it. The rest of the patch is trivial support code.
> Notes:
> - I'd like to see the Handler API be sufficient at the low level to
> allow an HTTP handler to be written that is every bit as efficient and
> speedy as what is being worked on for 4.1
Cut 'n paste the ~100 lines of code from SockParse which already
conforms to the Ns_ParseProc API and you're basically done.
> - I'd like to see the high level API be easy enough to use that one
> could register an "echo server" handler and write the code for it in a
> dozen lines or so
The example POP3 server, which is a simple, single-line oriented
protocol, implements an Ns_ParseProc in 24 line of code.
> - I'd like the progression from high level to low level be gradual, so
> the programmer can address only those issues that pertain to his
> specific protocol
A Tcl API could be wrapped around the Ns_ParseProc interface in the same
manner that ns_register_filter wraps Ns_RegisterFilter(). You would
have to make an interp available to the driver thread which doesn't
normally have one.
The Tcl code which implements your parser could call [ns_conn content]
and use the sls API to maintain state. Or you could wrap the
Ns_ConnReadLine etc. APIs.
Looks like an afternoons work.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of
your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
On Wed, 2004-08-18 at 09:21, Jim Wilcoxson wrote: > ... > > > AOLserver is ideally suited to the majority of server tasks. About the > > > only shortcoming which comes to mind is that conn threads are required > > > to do blocking writes. But fixing that would be of benefit to the HTTP > > > processing side too. > > > > So, we want reader threads and separate writer threads now, too? > > Exactly how many free-range threads do we want roaming the server > > prairie, here? > > You don't need reader threads and writer threads. If input and output > both use event-driven, non-blocking I/O, you can use one thread to do > both, or one thread per CPU, like TUX. You can't do non blocking I/O on a file descriptor backed by disk. Not portably at least. Calls like stat(2) to get the last mod time may also block. That's why I suggested a pool of writer threads. TUX actually does something very similar to handle potentially blocking disk I/O. One of the biggest resource hogs is not the conn threads themselves but the Tcl interps they drag round with them. If you're interested in cutting down on resource usage, you might want to take a look at this RFE: http://sourceforge.net/tracker/index.php?func=detail&aid=1012103&group_id=3152&atid=353152 -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] QOS, SEDA Re: [AOLSERVER] Support for non-HTTP protocols
On Tue, 2004-08-17 at 17:16, Nathan Folkman wrote: > Each addresses different issues. The thread pools are nice in that they > give you a way to isolate (by URL I believe?) certain types of activites > to particular pools of threads, thus preventing any one paritcular type > of activity from utilizing all of the threads. Here's a real world example. > > Imagine a single Web Server that serves both an application that does a > lot of interaction with a database, and also serves up system status > pages and other information. Suppose you have the server configured with > a max of 10 connection threads. Now imagine a user comes along and > decides to do some sort of degenerate query which takes order minutes. > The same user, seeing nothing happening, keeps hitting reload. In fact, > he's so eager to see some results that he hits the reload button 10 > times. You've now got all 10 of your connection threads busy working on > his long running database query. Any user wanting to see the system > status would not be able to get to the page since the server is now > thread maxed. > > Now imagine you've set things up with thread pools. The database tool > gets a max of 5, and the system stats page gets a max of 5. Now our over > anxious database application user can only tie up a maximum of 5 > connection threads working on his database query, leaving up to 5 > connection threads for users wishing to see the system stats page. How do you determine how many threads to add to each pool? Let's say you determine the webserver can process 10 requests concurrently. You could assign 5 threads to each pool. In this situation, one pool may be idle and the server may have spare capacity, and yet still reject requests get assigned to the other pool. Alternatively, you could have one pool of 10 threads and classify connections as they are received. You might then specify that both users and admins can max out at 8 conn threads each. Obviously they can't both have 8 conn threads, so what you're effectively specifying here is that the *other* class of connection gets a minimum of 2 conn threads. The effect is more pronounced the more resource pools you have. Also, nothing to do with the SEDA or thread pool approach but looking at the current implementation, it's a little inflexible in that it assigns requests to pools based on URL. This allows you to effectively partition your order checkout, for example. It doesn't let you handle quality of service situations like favouring logged in users over anonymous users, or favouring fat-cat CEOs over common proles... :-) A flexible callback interface would be nice to have here. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
On Tue, 2004-08-17 at 17:01, Dossy Shiobara wrote: > On 2004.08.17, Andrew Piskorski <[EMAIL PROTECTED]> wrote: > > So far, the only people who've actually claimed they've read the > > patches are the two authors and Zoran. I'm guessing Dossy has read > > them too but I'm not sure. Did you, Don? > > Yes, I've read both patches. I've even discussed Stephen's patch with > Stephen off-list. The last time we spoke you hadn't read the patch so your comments weren't useful. You still seems to have some misunderstandings about how AOLserver processes requests and how my patch may or may not interact. Ask if you have any questions, that's what we're here to discuss. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
On Tue, 2004-08-17 at 16:24, Andrew Piskorski wrote: > I can't seem to checkout the AOLserver head from SourceForge right > now, so I haven't applied it against 4.1 in order to read what it > really does, but according to wc -l, Vlad's patch is a grand total of > 147 lines. Anon cvs has been down since Friday so I haven't been able to rediff against the latest changes. Nothing major has changed though, so probably only some minor patch jiggling required. I'll get to it when I can. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
On Tue, 2004-08-17 at 10:04, Dossy Shiobara wrote: > On 2004.08.17, Stephen Deasey <[EMAIL PROTECTED]> wrote: > > This isn't true. In AOLserver, once a request has completed the conn is > > returned to the driver thread for keepalive. It's monitored in a > > non-blocking fashion until either it times out or the client makes > > another request. In the meantime, the conn thread which just completed > > is free to service other requests. > > Um, what about protocols that requires the server to maintain state with > the connection? If the thread servicing the connection gets returned to > the pool, doesn't the state go along with it? State is stored in session local storage. This is just like conn and thread local storage, but the data lifetime is from socket connect to socket disconnect. The example server illustrates this. > > This makes it possible to have 1000 concurrent connections serviced by a > > pool of just 10 threads -- at any one time most of the connections will > > be momentarily idle. This benefits memory usage and context switch > > overhead. > > This is only desirable for stateless protocols, like HTTP. It's highly desirable for stateful protocols too. The inactivity timeout for POP3 is 10 minutes, so potentially your thread+interp could be kept tied up for that period of time. My email client (IMAP) checks for new messages every 3 minutes -- it doesn't disconnect in between. The request-response-pause pattern is common to many protocols. > > This is one of the major architectural benefits AOLserver has over > > something like Apache. In the Apache model, processes take turns > > listening for connections. When a connection arrives, the process > > switches roles from a listener to a worker and processes the > > connection. The worker processes the connection to completion, > > including idle wait time due to keepalive. > > True, for stateless protocols like HTTP, the AOLserver model is > superior. For stateful protocols, this may not work so well. > > > AOLserver is ideally suited to the majority of server tasks. About the > > only shortcoming which comes to mind is that conn threads are required > > to do blocking writes. But fixing that would be of benefit to the HTTP > > processing side too. > > So, we want reader threads and separate writer threads now, too? > Exactly how many free-range threads do we want roaming the server > prairie, here? > > > > Of course, we're both only speculating here. > > > > Speak for yourself! > > Show me your measurements. And I'm not talking 36D-28-34, either. I think the burden of proof lies with you. A non-HTTP server implemented with the proposed Ns_ParseProc interface is exactly the same architecturally as stock AOLserver. If you think you can rewrite it all in Tcl and make it faster, I'd be interested to see that. I don't think you can do it. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
On Tue, 2004-08-17 at 08:07, Dossy Shiobara wrote: > On 2004.08.17, Stephen Deasey <[EMAIL PROTECTED]> wrote: ... > > Each of your connection threads and all it's resources are committed for > > the life of the connection. It will remain committed until the client > > disconnects. > > This will be true regardless of the implementation (C vs. Tcl) as it's a > requirement of the TiVo beacon protocol -- clients maintain persistent > connections to the server. This isn't true. In AOLserver, once a request has completed the conn is returned to the driver thread for keepalive. It's monitored in a non-blocking fashion until either it times out or the client makes another request. In the meantime, the conn thread which just completed is free to service other requests. This makes it possible to have 1000 concurrent connections serviced by a pool of just 10 threads -- at any one time most of the connections will be momentarily idle. This benefits memory usage and context switch overhead. This is one of the major architectural benefits AOLserver has over something like Apache. In the Apache model, processes take turns listening for connections. When a connection arrives, the process switches roles from a listener to a worker and processes the connection. The worker processes the connection to completion, including idle wait time due to keepalive. > I disagree. SMTP and IMAP are toy servers with regard to complexity, > just as the TiVo server is. They are all very simple protocols. The > challenge is scalability -- the assumption here is that a pure Tcl > solution will not perform well enough to scale compared to the C > solution. It's only an assumption. As you say, parsing is the least difficult part of creating a scalable server. Which is why I added a hook to implement just that, so that you can take advantage of all the AOLserver scalability features. > My hunch is that empirical evidence will show that none of the > "features" of AOLserver will make the C solution perform exceptionally > better than a pure Tcl solution. If you're going to implement in C for > performance, implement something tailored to solving the problem > specifically -- that will be the only way to really get the performance > required. AOLserver is ideally suited to the majority of server tasks. About the only shortcoming which comes to mind is that conn threads are required to do blocking writes. But fixing that would be of benefit to the HTTP processing side too. > Of course, we're both only speculating here. Speak for yourself! -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
On Tue, 2004-08-17 at 06:55, Dossy Shiobara wrote: > > I could very well be wrong. If someone is interested in finding an > off-the-shelf, or implementing from scratch, a load generator for one of > these non-HTTP protocols (say, SMTP) ... I'd like to have a bake-off. > Implemented in pure Tcl, and implemented using one of the two proposed > patches. Then, benchmark the two and see if there's a measurable > difference. Alternatively, you could implement a web server using your tivo server method and use apache bench to test it against stock AOLserver. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
On Tue, 2004-08-17 at 03:28, Bas Scheffers wrote: > Very interesting. After only reading your nutshell descriptions, to me > what would seem like a nice way of handling this is lightweight plumbing > in C and the bulk in Tcl. This is the essence of my proposed solution. You are required to write a parser in C, you can handle the processing in either C or Tcl in the normal fashion by registering a proc. Strictly speaking, you're not even required to parse the request in C. The minimum requirement is that you identify the end of the request (this would be a line end for an echo server request). You can do the actual parsing in Tcl within your registered proc; the content of the entire request is available by calling [ns_conn content]. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
On Tue, 2004-08-17 at 05:46, Dossy Shiobara wrote: > > So far, this has been working really well for me. No modification to > the core involved, and it's implemented in very few LOC overall. Using > the same pattern, I could easily implement an SMTP, IMAP, etc., server > in pure Tcl. I'm not sure what the real performance difference there > would be compared to the two proposals that require C code changes in > the core: as long as the actual "functionality" is implemented in a Tcl > callback that gets invoked from the C code, the performance difference > is probably negligible. The performance difference would certainly be noticeable. In your example code a new thread is spawned for each connection. This can take a non trivial amount of time, especially if a new interp has to be initialized. Each of your connection threads and all it's resources are committed for the life of the connection. It will remain committed until the client disconnects. There are some functional differences too: no access logging; none of the niceties of registered procs or filters; no caching of returned files, etc. etc. None of which matters for a toy like a Tivo server. An SMTP or IMAP server implemented like this wouldn't be worth the bother. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Support for non-HTTP protocols
On Tue, 2004-08-17 at 03:59, Andrew Piskorski wrote: > On Tue, Aug 17, 2004 at 10:32:56AM +0200, Zoran Vasiljevic wrote: > > > Vlad's patch implements entirely new socket-level driver and sticks > > the whole add-in functionality in the driver itself, effectively > > > Stephen's patch integrates into the AS by adding new API call > [...] > > from a connection socket. This way a new protocol connect handling > > looks pretty much like regular http. Hence it utilitzes all of the > > Are there any types of protocols that Vlad's approach can handle while > Stephen's cannot? E.g., protocols that look very different from HTTP? > If so, would it make sense to support BOTH methods in AOLserver? Although the example protocol driver for POP3 generates what looks like a (basic) HTTP request, there is no requirement to do so, it was just convenient. If you were handling some binary protocol, the minimum you would have to do in the parse proc would be identify the end of the message, and set the request line 'MYPROTO /'. You would then register a proc for that method and URL which calls Ns_ConnContent to get access to the payload of the message. The request line becomes a dummy, but it may still be useful. For example you can use it to force processing from one or another connection pool. There is the restriction at the moment that protocols which require pipelining (sending message 2 before receiving the answer to message 1), and by extension protocols which don't need an answer to all requests, aren't supported. I'd like to add that though, it's a requirement of HTTP/1.1 and a handy speed boost. > Also, this is all still only for protocols on top of TCP/IP, right? > No support for UDP or any other sort sockets? Vlad has submitted a patch which adds support for this to the binder: http://sourceforge.net/tracker/index.php?func=detail&aid=463625&group_id=3152&atid=353152 I think the most useful feature here would be unix domain sockets for something like a fastcgi module. This would be transparent to existing socket drivers. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] ad_conn url cuts off after th #
The fragment appears in the URL displayed by your browser, it's not necessarily sent to the server. /fragment.tcl ns_return 200 text/plain "request: [ns_conn request]" telnet localhost 80 GET /fragment.tcl#frag HTTP/1.0 404 Not Found. telnet locahost 80 GET /fragment.tcl?foo=bar#frag request: /fragment.tcl?foo=bar#frag HTTP/1.0 The first example fails because AOLserver is looking for a file named fragment.tcl#frag, whereas in the second the URL is trimmed at the question mark. If all browsers sent the fragment portion of the URL, AOLserver would fail a lot more often. Therefore, it's unlikely they do. On Sat, 2004-08-07 at 01:01, Tracy Adams wrote: > And in my case, the # stays in the URL: > > Here is my URL: > > http://learning.coachville.com/dotlrn/clubs/coachingfundamentalsbacktobasics > /coachingfundamentalsbacktobasics2/file-storage/view/class_3001:_fundamental > _coaching_proficiency_#1:_crafts_collaborative_relationships/Overview?file_i > d=4317 > > > Note the #1. > > Tracy > > -Original Message- > From: Tracy Adams [mailto:[EMAIL PROTECTED] > Sent: Saturday, August 07, 2004 1:59 AM > To: 'AOLserver Discussion' > Subject: RE: [AOLSERVER] ad_conn url cuts off after th # > > Interesting clue. > > When I got to a non-aolserver site, the # remains. For example, go to > http://list.auctions.shopping.yahoo.com/[EMAIL PROTECTED] > amd the # remains. So I don't think the browser just cuts it off. > > But when I go to an AOLServer site, the # does disappear. > > So perhaps AOLServer is doing something - a redirect? > > Tracy > > -Original Message- > From: AOLserver Discussion [mailto:[EMAIL PROTECTED] On Behalf Of > Wojciech Kocjan > Sent: Saturday, August 07, 2004 1:31 AM > To: [EMAIL PROTECTED] > Subject: Re: [AOLSERVER] ad_conn url cuts off after th # > > Tracy Adams wrote: > > Any idea how to reference the COMPLETE URL? > > The # is cut by the browser. > > [EMAIL PROTECTED]:~$ netcat -l -p 8080 195.82.181.6 > GET /test/a HTTP/1.1 > Host: dq.pl:8080 > User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; pl-PL; rv:1.5) > Gecko/20031007 MultiZilla/1.6.3.0d > Accept: > text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q= > 0.8,image/png,image/jpeg,image/gif;q=0.2,*/*;q=0.1 > Accept-Language: pl,en;q=0.5 > Accept-Encoding: gzip,deflate > Accept-Charset: ISO-8859-2,utf-8;q=0.7,*;q=0.7 > Keep-Alive: 300 > Connection: keep-alive > > This is the url I entered in the browser. > http://dq.pl:8080/test/a#b > > -- > WK > > > -- > AOLserver - http://www.aolserver.com/ > > To Remove yourself from this list, simply send an email to > <[EMAIL PROTECTED]> with the > body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: > field of your email blank. > > > -- > AOLserver - http://www.aolserver.com/ > > To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with > the > body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field > of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Security issue in AOLserver (repsonse injection)
On Fri, 2004-07-16 at 13:01, Dossy wrote: > This also raises the issue that AOLserver currently does NOT accept HTTP > requests where header lines are split across multiple lines via > header-continuation. With regard to strict compliance to the spec., > this is a bug and should be addressed separately. I think this is already handled. The comment for Ns_ParseHeader, which is called within driver.c to parse the headers as they are recieved, says: Consume a header line, handling header continuation, placing results in given set. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] AOLserver 4.0.6 has crashing bug leading to DOS.
Line 1110 of driver.c: sockPtr->reqPtr->request->method = "BAD"; request->method is usually a malloced string, but here it's pointing to static storage. After request is finnished Ns_FreeRequest is called on the request pointer, which then tries to ns_free method. The server aborts at this point. On Fri, 2004-07-16 at 13:22, Rob Crittenden wrote: > This causes a core dump? > > rob > > Stephen Deasey wrote: > > > How to reproduce: > > > > > telnet host 80 > > > GET / HTTP/1.1 > > > > > > > > > I sent an email about this early yesterday well before the release, but > > didn't get any response. > > > > > > Thanks. > > > > > > -- > > AOLserver - http://www.aolserver.com/ > > > > To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> > > with the > > body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field > > of your email blank. > > > -- > AOLserver - http://www.aolserver.com/ > > To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with > the > body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field > of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
[AOLSERVER] AOLserver 4.0.6 has crashing bug leading to DOS.
How to reproduce: > telnet host 80 > GET / HTTP/1.1 > I sent an email about this early yesterday well before the release, but didn't get any response. Thanks. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Retiring the SourceForge "Patches" tracker
Only the person who opens a bug (and perhaps admins???) can attach files to a bug report, i.e. a patch. If you remove the patch tacker you'll have to fix all the bugs yourself... :-) On Tue, 2004-07-06 at 09:06, Dossy wrote: > I want to ask if anyone finds any use or value in having a separate > "Patches" tracker at SourceForge. It seems to me the only difference > between the Bugs and Patches trackers is that a "Patch" is a Bug with a > suggested patch to fix it. > > I'm inclined to retire the Patches tracker and move all "patches" into > the Bugs tracker. > > Does anyone have an issue with this change? Otherwise, I'll make it > sometime this week. > > -- Dossy -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Managing request-specific global data
I'd like connection local storage that lasts the lifetime of a client connection to the server, i.e. as long as the socket is open. AOLserver terminology is whacked -- it defines a connection as an HTTP request, and a request as the actual 'GET / HTTP/1.0' line. So I guess what I'd like is 'socket local' storage. Here's the plan: add a hash table pointer to the Ns_Sock structure. Create both C and Tcl APIs for setting/getting simple string values. The hash table will be allocated on first access, so no overhead if not used. The simple string key/values allow interoperability between C and Tcl code, and is required for Tcl anyway as connections can run in different interps so objects can't be passed (just like nsv). Is this the kind of change that is likely to be accepted as a patch against AOLserver, or should I just hack this up for myself? On Thu, 2004-06-10 at 14:03, Mark Page wrote: > This sounds like Connection Local Storage type of need. There is a set > of api's available in aolserver 4.0 which implement this, which you > might find interesting: > > http://www.aolserver.com/40drafts/capi/api/newindex.htm > > Specifically, for example, > -- > > -- Mike > > Mike Schilli > > [EMAIL PROTECTED] so probably there's a better solution out there already? > > Would be great if you could give some insight. > http://www.aolserver.com/40drafts/capi/api/c40-02.htm > > > Mike Schilli wrote on 6/10/2004, 3:47 PM: > > > Hi all, > > > > to manage a global data structure over the course of a single request, > > we were thinking about having a global hash table, indexed by the > > address returned by Ns_GetConn(). > > > > This way, every function could access the request-specific data by > > calling Ns_GetConn(), obtaining a unique "request-id", looking up the > > hash entry and read/write without a need for locking. > > > > The only challange is making the entry go away after the request > > finishes. > > > > Is this a good approach? It seems to me that this kind of requirement is > > pretty common, > > -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] ns_logctl expansion?
What happens if the network or some other part of the infrastructure goes down such that one of the nodes can't communicate with the master? Do you loose log messages? Does the node block? What happens when the master fills it's log disk, do all the clients block? You're going to need a temporary buffer, i.e. a log file, on each node and some error recovery strategy. The easiest thing to do would be to write a scheduled proc which scans the existing log file from the previous seek point up to the current end of file. You could use ns_httppost etc. to send back packets of log data. On Sun, 2004-03-07 at 05:03, Zoran Vasiljevic wrote: > Hi ! > > First, I would not like to start another flame... > > Now, what I would like to ask is: would anybody have something > against to adding additional functionality to the [ns_logctl] > command so it is possible to assign a regular Tcl channel > as the output for the [ns_log] command? > > Like, for example: > >set chan [open /some/file.log w] >ns_logctl pushchan $chan >ns_log notice "This goes to chan $chan" >set chan [ns_logctl popchan] >close $chan > > The push/pop semantic would create a stack of channels > but would take care that first one (the one opened at > the server start) is never closed (i.e. popped back). > > We would need such functionality in order to controllably > redirect (during runtime) the log sink. > > Before I start to code this for our app only, I just > wanted to see wether this might be useful for other > people so we can include it in the standard AS code. > Futhermore, I'm not sure if the proposed push/pop > semantics would be ok. Maybe there are some better > ways of doing this? > > Any thoughts? > > Cheers, > Zoran > > > -- > AOLserver - http://www.aolserver.com/ > > To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with > the > body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field > of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Server doesn't recognize directories starting with +
The patch is against 4.0 but it also applies cleanly against 4.0.2 and 4.1 from CVS. cd aolserver-4.0 patch -p1 -b < ../aolserver-4.0-plus-sign-encoding.patch On Tue, 2004-01-27 at 02:25, Bart Teeuwisse wrote: > Stephen, > > against which version of AOLserver 4.0 did you make the patch? The patch is > not lining up against the tarball of 2003-11-09 as available from sourceforge. > > Neither does it match the current CVS HEAD. > > /Bart > > > -- > AOLserver - http://www.aolserver.com/ > > To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with > the > body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field > of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Server doesn't recognize directories starting with +
It's an AOLserver bug -- it's translating + to space in paths where it
shouldn't. Attached is a patch for 4+.
On Mon, 2004-01-26 at 14:37, Bart Teeuwisse wrote:
> Fastpath.c doesn't recognize directories starting with a plus sign (+). Thus
> _ns_dirlist is never called to return the content of the directory.
>
> This is important to me as I'm trying to setup AOLserver to host GNU Arch
> (http://www.gnu.org/software/gnu-arch/) archives. Arch makes use of
> directories called ++revision-lick and +contents to name a couple examples.
>
> Lacking the necessary C skills, I'd like to ask those who do posses them, is
> it a bug that fastpath.c doesn't recognize directory names starting with a
> plus sign? If not what is needed to fix it?
>
> /Bart
>
>
> --
> AOLserver - http://www.aolserver.com/
>
> To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with
> the
> body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field
> of your email blank.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of
your email blank.
--- aolserver-4.0/include/ns.h.plus-sign-encoding 2004-01-16 04:35:27.0 -0700
+++ aolserver-4.0/include/ns.h 2004-01-16 04:35:45.0 -0700
@@ -1156,6 +1156,15 @@
NS_EXTERN char *Ns_DecodeUrlCharset(Ns_DString *pds, char *string,
char *charset);
+NS_EXTERN char *Ns_EncodeFormWithEncoding(Ns_DString *pds, char *string,
+ Tcl_Encoding encoding);
+NS_EXTERN char *Ns_DecodeFormWithEncoding(Ns_DString *pds, char *string,
+ Tcl_Encoding encoding);
+NS_EXTERN char *Ns_EncodeFormCharset(Ns_DString *pds, char *string,
+char *charset);
+NS_EXTERN char *Ns_DecodeFormCharset(Ns_DString *pds, char *string,
+char *charset);
+
/*
* urlopen.c:
*/
--- aolserver-4.0/nsd/urlencode.c.plus-sign-encoding2004-01-16 04:34:56.0 -0700
+++ aolserver-4.0/nsd/urlencode.c 2004-01-16 04:35:57.0 -0700
@@ -41,6 +41,14 @@
+#define URL_ENCODING 0
+#define FORM_ENCODING 1
+
+static char *EncodeWithEncoding(Ns_DString *dsPtr, char *string,
+ Tcl_Encoding encoding, int scheme);
+static char *DecodeWithEncoding(Ns_DString *dsPtr, char *string,
+ Tcl_Encoding encoding, int scheme);
+
static Tcl_Encoding GetUrlEncoding(char *charset);
/*
@@ -123,10 +131,10 @@
/*
*--
*
- * Ns_EncodeUrlWithEncoding --
+ * Ns_EncodeUrlWithEncoding, Ns_EncodeUrlCharset --
*
* Take a URL and encode any non-alphanumeric characters into
- * %hexcode
+ * %hexcode.
*
* Results:
* A pointer to the encoded string (which is part of the
@@ -141,6 +149,76 @@
char *
Ns_EncodeUrlWithEncoding(Ns_DString *dsPtr, char *string, Tcl_Encoding encoding)
{
+return EncodeWithEncoding(dsPtr, string, encoding, URL_ENCODING);
+}
+
+char *
+Ns_EncodeUrlCharset(Ns_DString *dsPtr, char *string, char *charset)
+{
+Tcl_Encoding encoding = GetUrlEncoding(charset);
+
+return Ns_EncodeUrlWithEncoding(dsPtr, string, encoding);
+
+}
+
+
+/*
+ *--
+ *
+ * Ns_EncodeFormWithEncoding, Ns_EncodeFormCharset --
+ *
+ * Take some form data encode any non-alphanumeric characters into
+ * %hexcode, except spaces which become '+'.
+ *
+ * Results:
+ * A pointer to the encoded string (which is part of the
+ * passed-in DString's memory)
+ *
+ * Side effects:
+ * Encoded form data will be copied to given dstring.
+ *
+ *--
+ */
+
+char *
+Ns_EncodeFormWithEncoding(Ns_DString *dsPtr, char *string, Tcl_Encoding encoding)
+{
+return EncodeWithEncoding(dsPtr, string, encoding, FORM_ENCODING);
+}
+
+char *
+Ns_EncodeFormCharset(Ns_DString *dsPtr, char *string, char *charset)
+{
+Tcl_Encoding encoding = GetUrlEncoding(charset);
+
+return Ns_EncodeUrlWithEncoding(dsPtr, string, encoding);
+
+}
+
+
+/*
+ *--
+ *
+ * EncodeWithEncoding --
+ *
+ * Take a string and encode any non-alphanumeric characters, the
+ * scheme determins the method of encoding:
+ * URL_ENCODE: %hex encoding
+ * FORM_ENCODE: %hex encoding, but spaces become '+'
+ *
+ * Results:
+ * A pointer to the encoded string (which is part of the
+ * passed-in DString's memory)
+ *
+ * Side effects:
+ * Encoded URL will be copied to given dstring.
+ *
+ *--
+ */
+
+static char *
+EncodeW
Re: [AOLSERVER] in-memory database for AOLserver?
MySQL 4 has an in-memory-only table type, and can be linked against as a library (rather than connected to as a server). Better than nsv's (just.. :-) On Fri, 2002-07-26 at 15:20, Andrew Piskorski wrote: > Does and sort of simple, thread-safe, in memory relational database > for AOLserver exist? I don't care about persistence - in fact, I > don't want it.
Re: [AOLSERVER] Digest authorization / WebDAV
SASL: http://www.oreillynet.com/pub/a/network/2002/04/09/sasl.html ...if you're feeling ambitious.
Re: [AOLSERVER] custom replacement for ns_sockopen?
Check out Tcl_RegisterObjType in the manual: http://www.aolserver.com/docs/devel/tcl/tcl-language/tcl8.x/TclLib/ObjectType.htm On Mon, 2001-10-29 at 05:12, Andrew Piskorski wrote: > In AOLserver, the ns_sockopen Tcl commannd is implemented by > NsTclSockOpenCmd which calls Ns_SockConnect (or Ns_SockAsyncConnect, > etc.), which calls SockConnect. Now, all these TCP/IP socket calls > return file handles, which are just integers. > > I have a vendor's proprietary API that works a lot SockConnect, but > it's foo_connect function doesn't return a file handle, it returns a > pointer to a black-box structure, and I have to pass this pointer back > to a all the other foo_* API functions. > > What I want to do, is wrap this stuff inside my own AOLserver > procedures, so I can then call foo_sockopen from Tcl just like I would > call ns_sockopen. So I figure that each time I get a pointer back > from foo_connect, I should call Ns_ServerSpecificAlloc to get a unique > integer ID, and then save my (connectionId, connectionPtr) tuple in > some data structure. That way my Tcl commands can use connectionId, > and never know that connectionPtr even exists. > > My question for you folks is, what is the most appropriate data > structure for me to store my (connectionId, connectionPtr) tuples in? > Is an appropriate structure already defined somewhere, or should I > roll my own? > > I thought I'd use a server-wide NSV, but then I looked in > aolserver/nsd/tclvar.c and realized that there is actually no C API > provided to the NSV faciltiy, just Tcl. Ns_Set is only for strings, > so using it doesn't seem like a good idea. > > (I wasn't sure whether it would be best to store this data structure > either per-thread or server-wide, but I suspect it doesn't really > matter.) > > Also, in the simplest case, I could allow only a single open > connection per thread. Then, all I'd need for my "data structure" is > a single variable per thread. But, how do I even create such a > variable from C? Do I need to use Ns_AllocThreadLocalStorage? If I > just declare a global variable in my-loadable-module.c, what is the > scope of that variable? Per-thread, or server-wide? > > Any advice will be much appreciated... > > -- > Andrew Piskorski <[EMAIL PROTECTED]> > http://www.piskorski.com
