Re: [jabberd2] Removing Cyrus SASL backend proposal
On Mon, Oct 6, 2008 at 6:43 AM, Proskurin Kirill <[EMAIL PROTECTED]> wrote: > Tomasz Sterna wrote: >> >> Cyrus SASL backend is currently abandoned and not maintained anymore. >> It has known bugs with noone wanting to fix them. >> >> The main reason for Cyrus backend inclusion in jabberd2 was GSSAPI >> support. GnuSASL backend has GSSAPI support now, too. >> >> The abandoned SCOD backend was removed lately. >> There are no known bugs in GnuSASL backend implementation. >> Therefore I am for removing Cyrus SASL backend. >> >> Before I do so, I would like to hear what the jabberd2 community thinks >> of the change. > > Agreed. +1 -- To unsubscribe send a mail to [EMAIL PROTECTED]
Re: [jabberd2] Rate limiting functional.
On Wed, Aug 6, 2008 at 6:33 AM, Tomasz Sterna <[EMAIL PROTECTED]> wrote: > Dnia 2008-08-06, śro o godzinie 02:07 -0700, Mark Doliner pisze: >> It still wouldn't be beautifully event-driven, but it should be less >> resource intensive. > > Is it really so resource intensive? > I didn't notice any CPU consumption change on my production system. > (It's still ~5% in peaks ;-) ) Sending this again because some people have told me they couldn't read the first one. I think it happens because my mail client (Google Mail) is sending the body base64 encoded, but the jabberd2 mailing list software is appending a link "To unsubscribe send a mail to" which ISN'T base64 encoded. Which results in a corrupt body. I don't know. We haven't tried it yet, but I'll let you know what happens when we do. Thanks, Mark
Re: [jabberd2] Rate limiting functional.
On Wed, Aug 6, 2008 at 6:33 AM, Tomasz Sterna <[EMAIL PROTECTED]> wrote: > Dnia 2008-08-06, śro o godzinie 02:07 -0700, Mark Doliner pisze: >> It still wouldn't be beautifully event-driven, but it should be less >> resource intensive. > > Is it really so resource intensive? > I didn't notice any CPU consumption change on my production system. > (It's still ~5% in peaks ;-) ) I don't know. We haven't tried it yet, but I'll let you know what happens when we do. Thanks, Mark N�r��zǧu���[h���z�v��칻 �&ޖ+-���F��
Re: [jabberd2] Rate limiting functional.
On Sun, Jul 27, 2008 at 12:58 PM, Tomasz Sterna <[EMAIL PROTECTED]> wrote: > Dnia 2008-07-24, czw o godzinie 12:17 +0200, Tomasz Sterna pisze: >> I will look at the throttling possibilities (instead of >> disconnections) >> during the weekend. >> I have a hunch it is doable. :-) > > It's done. http://jabberd2.xiaoka.com/changeset/639/ > It was actually pretty easy - I hooked into _c2s_time_checks() and added > reading of throttled connections there. > > Please test it if you may. It looks like working OK. But I may not come > up with and tested all edge cases. Neat! On thing we noticed is that c2s calls _c2s_time_checks() more often and therefore iterates through all connections more often (by default every 5 seconds instead of never). One change that would be a huge improvement is to put limited connections into a linked list, and remove them from the list when they're clear. Then you could iterate through the short list of throttled connections instead of the long list of all connections. It would probably be empty most of the time. It still wouldn't be beautifully event-driven, but it should be less resource intensive. -Mark -- To unsubscribe send a mail to [EMAIL PROTECTED]
Re: [jabberd2] Re: Rate limiting non-functional?
Hmm, two people have told me my last email was empty, so here it is again: FYI I just added a "stanza rate limit." Instead of limiting based on the number of bytes it limits based on the combined number of message, presence and iq requests (regardless of their size). It's configurable in c2s.xml -Mark -- To unsubscribe send a mail to [EMAIL PROTECTED]
Re: [jabberd2] Re: Rate limiting non-functional?
On Fri, Jul 18, 2008 at 10:42 AM, Tomasz Sterna <[EMAIL PROTECTED]> wrote: > Dnia 2008-07-18, pią o godzinie 09:31 -0700, Mark Doliner pisze: >> That seems a little odd... I think only data read from the client >> counts toward the limit. > > Disabling rate limiting helped the user that was "flooded out". > Unfortunately I cannot look at this now. FYI I just added a "stanza rate limit." Instead of limiting based on the number of bytes it limits based on the combined number of , and requests (regardless of their size). -Mark N�r��zǧu���[h���z�v��칻 �&ޖ+-���F��
Re: [jabberd2] Remove scod?
On Tue, Jul 8, 2008 at 1:45 AM, Tomasz Sterna <[EMAIL PROTECTED]> wrote: > Dnia 2008-07-07, pon o godzinie 20:10 -0700, Mark Doliner pisze: >> How do people feel about removing the "scod" code from the sx >> directory? > > This is what I did for 2.1 release. > I reincluded it back with introduction of selectable SASL backends in > hope that someone will fix it. > It could be useful for embedded deployments, when you do not need full > SASL library. > > But since nobody seems interested in it and it does not work anyway, I'm > +1 for removing it. Done! -- To unsubscribe send a mail to [EMAIL PROTECTED]
Re: [jabberd2] Re: Rate limiting non-functional?
On Fri, Jul 18, 2008 at 2:23 AM, Tomasz Sterna <[EMAIL PROTECTED]> wrote: > Dnia 2008-07-11, pią o godzinie 13:48 -0700, Mark Doliner pisze: >> 1. Don't disconnect people when they hit the rate limit. Instead, >> stop reading from their socket until the limit has expired. Do we >> have a way to call a function after a certain amount of time has >> passed? We would need one for this. > > There is another problem with this reported to me today. > If user has a lot of messages in offline storage, these get bursted to > the client after connection, hitting the rate limit and effectively > disconnecting the user. :-/ That seems a little odd... I think only data read from the client counts toward the limit. -Mark
Re: [jabberd2] External libraries: libevent and GLib
On Wed, Jul 16, 2008 at 3:17 AM, Tomasz Sterna <[EMAIL PROTECTED]> wrote: > Dnia 2008-07-15, wto o godzinie 15:07 -0700, Mark Doliner pisze: >> Anyway, probably not something we want to dive into now. I mostly >> just wanted to make people aware of the existence of the two >> libraries. > > I wondered many times about introducing GLib into jabberd2. > But as soon I realized that doing this would pretty much mean rewriting > the whole thing, I decided to actually write the thing from scratch and > started Crux http://crux.xiaoka.com/ FYI http://crux.xiaoka.com/ gives me a "forbidden" error: "WIKI_VIEW privileges are required to perform this operation on WikiStart" -Mark -- To unsubscribe send a mail to [EMAIL PROTECTED]
[jabberd2] External libraries: libevent and GLib
I just thought I'd throw this idea out there... has anyone ever thought of changing jabberd to use libevent or GLib? libevent is an event notification library that abstracts the APIs for poll, select, epoll, kqueue and maybe a few other things. I guess we would really only gain support for kqueue (the fast event polling mechanism used on FreeBSD (and other BSDs?)), since jabberd already has implementations for poll, select and epoll. And I guess we would lose support for wasync? It's probably not worth the effort, but if libevent existed when jabberd was first written it seems like someone could have saved themselves a good bit of time. And I'm always in favor of re-using shared libraries. I haven't used libevent, but it seems nice. You can find more information at http://monkey.org/~provos/libevent/ There's also libev, which does basically the same thing but is purportedly a tad bit faster. GLib is a C library with lots of random and useful functions. It provides lots of handy string manipulation functions, a generic linked list structure, a queue, a hash table that grows and shrinks automatically, base64 encoding/decoding (recently), etc. Again, we probably wouldn't gain a whole lot from it immediately, since jabberd has already been written with it's own implementations of a lot of this stuff, but I'm always in favor of re-using shared libraries. I've used GLib a lot and I love it. It's fantastic. You can see the documentation at http://library.gnome.org/devel/glib/stable/ Anyway, probably not something we want to dive into now. I mostly just wanted to make people aware of the existence of the two libraries. -Mark -- To unsubscribe send a mail to [EMAIL PROTECTED]
Re: [jabberd2] Re: Rate limiting non-functional?
On Sun, Jul 13, 2008 at 1:58 AM, Tomasz Sterna <[EMAIL PROTECTED]> wrote: > Dnia 2008-07-11, pią o godzinie 13:48 -0700, Mark Doliner pisze: >> Do we >> have a way to call a function after a certain amount of time has >> passed? We would need one for this. > > We have time checks in main loop, but this is pretty hacky. > Using/implementing more sophisticated event loop is a good idea. Yeah, I agree that that sounds like a good idea. I guess it makes sense to leave the rate limiting in router and it will become more useful when/if someone adds some sort of timeout functionality and then improves the rate limiting. I don't think I'll have time to attempt to do that right now. But that's ok, I'm not sure my implementation would be very good :-) Thanks for the response, Mark
[jabberd2] Re: Rate limiting non-functional?
On Thu, Jul 10, 2008 at 5:51 PM, Mark Doliner <[EMAIL PROTECTED]> wrote: > On Thu, Jul 10, 2008 at 5:38 PM, Mark Doliner <[EMAIL PROTECTED]> wrote: >> On Mon, Jul 7, 2008 at 10:25 AM, Mark Doliner <[EMAIL PROTECTED]> wrote: >>> I've been looking at the rate limiting options in c2s.xml, and looking >>> at the code that intends to enforce those limits. It looks like the >>> connection rate limit would work (but I haven't tested it), but it >>> looks like the byte limit code has a few major problems that would >>> keep it from doing anything useful. Has anyone successfully used byte >>> limiting? >>> >>> For starters there's no call to rate_add(), which means jabberd checks >>> if the rate has been exceeded, but it always checks if 0 > rate_limit, >>> which will never happen. Then there's a call to rate_left(), which I >>> think is intended to have the recv() call only attempt to read the >>> number of bytes that are allowed before you would be rate limited, but >>> the return value from rate_left() is never used. >>> >>> Those two problems are fairly easy to fix. The third problem I've >>> noticed, which is more difficult, is what to do once someone has been >>> rate limited. Ideally jabberd would just stop reading data from that >>> user until the desired number of seconds has elapsed. But that's hard >>> to do because I think our mio code uses readiness change >>> notification/edge-triggered readiness notification instead of >>> level-triggered readiness notification. So we can't just not read >>> from the socket during this function call because >>> _c2s_client_sx_callback() won't be called again. We could disconnect >>> the user pretty easily, but that's a bit clunky. >> >> I noticed that router has similar rate limiting, but there the byte >> rate limit is a little more effective, but I think it has the same >> problem with ceasing to read from the socket once it gets limited. It >> seems weird to me that the router would limit the traffic sent between >> the components and the rate at which new components can connect. I'm >> in favor of removing this limiting from router for the sake of >> simplifying the code and simplifying router.xml. How do other people >> feel? >> >> I noticed another flaw in the rate implementation. rate_t->bad is set >> when rate_add() is called, and rate_t->time is only reset in >> rate_reset(). What ends up happening is that the rate count gradually >> increases and approaches the limit. It eventually calls rate_add() >> which pushes the count over the limit and you get rate limited. >> >> This isn't so bad, except that it happens regardless of the amount of >> time that has elapsed since the first message. > > Oh, and there's another problem which is that the window doesn't > slide. It starts when you increment the count for the first time, > then resets at the end of the timeout. I guess this is a pretty minor > problem when the window is so short, but it's possible for someone to > be limited when they should not have been, and not limited when they > should have been. > > For example, say you have two windows back to back. If someone bursts > a lot of traffic at the end of the first window, and then a lot of > traffic at the beginning of the second window (but not enough to go > over the threshold in either window), they probably SHOULD be limited > because they've sent a lot of bytes within the time span of one > window, but they won't be. Sorry for replying to myself a bunch of times, I've been thinking out loud. I checked in a change that fixes the bigger problems here. There are still two things that could be improved: 1. Don't disconnect people when they hit the rate limit. Instead, stop reading from their socket until the limit has expired. Do we have a way to call a function after a certain amount of time has passed? We would need one for this. 2. The window still doesn't slide, it just starts over. This is negligible if the window is small. If you wanted to limit people to sending no more than 100MB/hour then this might be more of an issue. -Mark -- To unsubscribe send a mail to [EMAIL PROTECTED]
[jabberd2] Re: Rate limiting non-functional?
On Thu, Jul 10, 2008 at 5:38 PM, Mark Doliner <[EMAIL PROTECTED]> wrote: > On Mon, Jul 7, 2008 at 10:25 AM, Mark Doliner <[EMAIL PROTECTED]> wrote: >> I've been looking at the rate limiting options in c2s.xml, and looking >> at the code that intends to enforce those limits. It looks like the >> connection rate limit would work (but I haven't tested it), but it >> looks like the byte limit code has a few major problems that would >> keep it from doing anything useful. Has anyone successfully used byte >> limiting? >> >> For starters there's no call to rate_add(), which means jabberd checks >> if the rate has been exceeded, but it always checks if 0 > rate_limit, >> which will never happen. Then there's a call to rate_left(), which I >> think is intended to have the recv() call only attempt to read the >> number of bytes that are allowed before you would be rate limited, but >> the return value from rate_left() is never used. >> >> Those two problems are fairly easy to fix. The third problem I've >> noticed, which is more difficult, is what to do once someone has been >> rate limited. Ideally jabberd would just stop reading data from that >> user until the desired number of seconds has elapsed. But that's hard >> to do because I think our mio code uses readiness change >> notification/edge-triggered readiness notification instead of >> level-triggered readiness notification. So we can't just not read >> from the socket during this function call because >> _c2s_client_sx_callback() won't be called again. We could disconnect >> the user pretty easily, but that's a bit clunky. > > I noticed that router has similar rate limiting, but there the byte > rate limit is a little more effective, but I think it has the same > problem with ceasing to read from the socket once it gets limited. It > seems weird to me that the router would limit the traffic sent between > the components and the rate at which new components can connect. I'm > in favor of removing this limiting from router for the sake of > simplifying the code and simplifying router.xml. How do other people > feel? > > I noticed another flaw in the rate implementation. rate_t->bad is set > when rate_add() is called, and rate_t->time is only reset in > rate_reset(). What ends up happening is that the rate count gradually > increases and approaches the limit. It eventually calls rate_add() > which pushes the count over the limit and you get rate limited. > > This isn't so bad, except that it happens regardless of the amount of > time that has elapsed since the first message. Oh, and there's another problem which is that the window doesn't slide. It starts when you increment the count for the first time, then resets at the end of the timeout. I guess this is a pretty minor problem when the window is so short, but it's possible for someone to be limited when they should not have been, and not limited when they should have been. For example, say you have two windows back to back. If someone bursts a lot of traffic at the end of the first window, and then a lot of traffic at the beginning of the second window (but not enough to go over the threshold in either window), they probably SHOULD be limited because they've sent a lot of bytes within the time span of one window, but they won't be. -Mark -- To unsubscribe send a mail to [EMAIL PROTECTED]
[jabberd2] Re: Rate limiting non-functional?
On Mon, Jul 7, 2008 at 10:25 AM, Mark Doliner <[EMAIL PROTECTED]> wrote: > I've been looking at the rate limiting options in c2s.xml, and looking > at the code that intends to enforce those limits. It looks like the > connection rate limit would work (but I haven't tested it), but it > looks like the byte limit code has a few major problems that would > keep it from doing anything useful. Has anyone successfully used byte > limiting? > > For starters there's no call to rate_add(), which means jabberd checks > if the rate has been exceeded, but it always checks if 0 > rate_limit, > which will never happen. Then there's a call to rate_left(), which I > think is intended to have the recv() call only attempt to read the > number of bytes that are allowed before you would be rate limited, but > the return value from rate_left() is never used. > > Those two problems are fairly easy to fix. The third problem I've > noticed, which is more difficult, is what to do once someone has been > rate limited. Ideally jabberd would just stop reading data from that > user until the desired number of seconds has elapsed. But that's hard > to do because I think our mio code uses readiness change > notification/edge-triggered readiness notification instead of > level-triggered readiness notification. So we can't just not read > from the socket during this function call because > _c2s_client_sx_callback() won't be called again. We could disconnect > the user pretty easily, but that's a bit clunky. I noticed that router has similar rate limiting, but there the byte rate limit is a little more effective, but I think it has the same problem with ceasing to read from the socket once it gets limited. It seems weird to me that the router would limit the traffic sent between the components and the rate at which new components can connect. I'm in favor of removing this limiting from router for the sake of simplifying the code and simplifying router.xml. How do other people feel? I noticed another flaw in the rate implementation. rate_t->bad is set when rate_add() is called, and rate_t->time is only reset in rate_reset(). What ends up happening is that the rate count gradually increases and approaches the limit. It eventually calls rate_add() which pushes the count over the limit and you get rate limited. This isn't so bad, except that it happens regardless of the amount of time that has elapsed since the first message. -Mark -- To unsubscribe send a mail to [EMAIL PROTECTED]
[jabberd2] Remove scod?
How do people feel about removing the "scod" code from the sx directory? That's the old homegrown sasl implementation, right? I'm always in favor of removing older code in favor of moving forward. And I think using shared libraries is always a step forward. -Mark -- To unsubscribe send a mail to [EMAIL PROTECTED]
[jabberd2] Rate limiting non-functional?
I've been looking at the rate limiting options in c2s.xml, and looking at the code that intends to enforce those limits. It looks like the connection rate limit would work (but I haven't tested it), but it looks like the byte limit code has a few major problems that would keep it from doing anything useful. Has anyone successfully used byte limiting? For starters there's no call to rate_add(), which means jabberd checks if the rate has been exceeded, but it always checks if 0 > rate_limit, which will never happen. Then there's a call to rate_left(), which I think is intended to have the recv() call only attempt to read the number of bytes that are allowed before you would be rate limited, but the return value from rate_left() is never used. Those two problems are fairly easy to fix. The third problem I've noticed, which is more difficult, is what to do once someone has been rate limited. Ideally jabberd would just stop reading data from that user until the desired number of seconds has elapsed. But that's hard to do because I think our mio code uses readiness change notification/edge-triggered readiness notification instead of level-triggered readiness notification. So we can't just not read from the socket during this function call because _c2s_client_sx_callback() won't be called again. We could disconnect the user pretty easily, but that's a bit clunky. -Mark -- To unsubscribe send a mail to [EMAIL PROTECTED]
Re: [jabberd2] s2s fails as resolver not able to resolve DNS
On Wed, 2008-01-23 at 3:41 PM Praveen wrote: > Hi all, > I have 2 jabberd2 installations on my LAN, serving 2 different domains, > domain1 and domain2. However, I'm not able to establish s2s connections > between them. Apparently, the resolver is not able to resolve DNS queries > for domain1 and domain2. I have made entries in /etc/hosts for the same, > and I'm also able to ping one machine from the other. > > I see this on resolver.log of machine serving domain2: > > Wed Jan 23 15:15:43 2008 [notice] [domain1] could not be resolved > > The same thing happens on the other machine too. Any idea? > > Thanks, > Praveen Hey Praveen, I think the resolver does not use the /etc/hosts file when looking up domain names--it only uses DNS. Is it possible for you to add DNS entries for domain1 and domain2? -Mark ___ Jabberd2 mailing list Jabberd2@lists.xiaoka.com http://lists.xiaoka.com/listinfo.cgi/jabberd2-xiaoka.com
Re: [jabberd2] Fwd: Patch to allow C++ modules
I'm in favor of making a change along these lines... the patch just avoids using variables named "private" which are reserved words in C++. It's a pretty unobtrusive change to jabberd2. I think the variable should probably be "private_data" instead of "privateData," though -Mark > -Original Message- > From: [EMAIL PROTECTED] [mailto:jabberd2- > [EMAIL PROTECTED] On Behalf Of umesh sirsiwal > Sent: Monday, December 17, 2007 3:57 PM > To: jabberd2 development > Subject: Re: [jabberd2] Fwd: Patch to allow C++ modules > > > This patch does not any C++ dependency in the code. It just makes it > possible to develop a C++ module for personal use if one desires. > Otherwise, with every new jabber release we need to patch before we can > use it. > > -Umesh > > > On Dec 17, 2007 6:53 PM, Tomasz Sterna <[EMAIL PROTECTED]> wrote: > On Pn, 2007-12-17 at 17:46 -0500, umesh sirsiwal wrote: > > Jabber2d is written in C and because of its choice of data > structure > > variable names, only allows C modules. I want to develop my module > in > > C++ simply because I like doing so. > > Your preferences is up to you. > Even if you had chosen The Dark Side. ;-) > > > I went around changing names of some of the variables so that they > are > > safe for a C++ program. Attached patch accomplishes that. > > I will not accept any C++ dependency into jabberd codebase, so there > is > no need to merge the patch into mainstream. > But it could be of interest for the ones, that like to develop C++ > modules for personal use. ___ Jabberd2 mailing list Jabberd2@lists.xiaoka.com http://lists.xiaoka.com/listinfo.cgi/jabberd2-xiaoka.com
Re: [jabberd2] Storing passwords in DB as MD5
On Nov 4, 2007 at 2:11am Josh wrote: > Is there a way to store c2s passwords in the database as MD5 hashes? More > to the point, is it easy to use an existing database full of usernames and > MD5-hashed passwords? If you want to use an existing database full of usernames and MD5-hashed passwords you can use the pipe authreg module and write a small script that implements the CHECK-PASSWORD functionality. It should accept the username, password and realm as input, md5 the password, and check that it is the same as the version in the database. At stated earlier, this DOES mean you can't use the digest-md5 authentication mechanism--you'll have to use plain. And so it might be wise to require-starttls. -Mark ___ Jabberd2 mailing list Jabberd2@lists.xiaoka.com http://lists.xiaoka.com/listinfo.cgi/jabberd2-xiaoka.com
Re: [jabberd2] asynchronous DNS and resolver component
On Nov 2, 2007 at 2:42 AM Tomasz Sterna wrote: > I had started implementing asynchronous DNS resolving in jabberd, > to really fix problem of resolver timing out requests by accumulation. > > If there are 20 requests to process at the moment, and each takes 5 > seconds to resolve (default libresolv timeout), then with 30 seconds > request timeout (default jabberd timeout), resolver will process only 6 > first requests, with 14 failing with timeout when resolver didn't even > try to resolv it. > I tried to fix it by prolonging timeouts, but it hurts on the user > experience point of view - when a request takes more than 30 seconds to > finish, users tend to cancel it and retry. Answer after 5 mins of > waiting is not an option. > > So, the real solution is to implement asynchronous DNS resolving. > I'm going to use UDNS http://www.corpit.ru/mjt/udns.html library. > Is everyone OK with it? I've never used a library for asynchronous DNS, but this one looks good to me. FYI Pidgin has two homegrown async DNS implementations, one forks child processes and talks to them with pipes, and one spawns threads. Using this library seems like a better idea than implementing either of those methods yourself. > And the second question arises. > With async DNS, we are able to ditch the whole resolver component > completely - s2s will be perfectly able to do the resolution by itself. > Resolver component was only needed to delegate the blocking DNS > resolution requests to another process. > I do see a use for resolver component in big, distributed deployments, > to do central name-resolution for all present s2s components. > But as for now, the resolver does only resolving. The name-2-address > mappings are cached... but in the s2s itself! > So, with the current design, we could move the resolution to s2s and > loose nothing. Or we could move the caching to resolver. > The s2s based resolving could also help in split-horizon DNS cases. > I personally am after the first option. Am I loosing something, or we > could just go with ditching the resolver? Getting rid of resolver seems like a good idea to me. If big, distributed deployments really want to do central name-resolution they could easily run a caching DNS server and have all DNS queries proxied through that. > And then, if we do left resolver behind, does it justify raising the > minor number of jabberd version. It sure is architectural change, and I > think we should release the ADNS branch under 2.2.x line. Raising the minor number sounds like a good idea to me. And it'll help people see that jabberd really is under active development. -Mark ___ Jabberd2 mailing list Jabberd2@lists.xiaoka.com http://lists.xiaoka.com/listinfo.cgi/jabberd2-xiaoka.com
[jabberd2] Use /usr/sbin instead of /usr/bin
Currently jabberd2's binary files are installed to /usr/bin. It seems more customary for server binaries to go into the /usr/sbin directory instead. Examples off servers on my system that are in /usr/sbin: crond, anacron, lighttpd, apache, sshd, ntpd, cupsd, sendmail, smartd How do people feel about changing jabberd to "make install" binaries into /usr/sbin instead of /usr/bin? -Mark ___ Jabberd2 mailing list Jabberd2@lists.xiaoka.com http://lists.xiaoka.com/listinfo.cgi/jabberd2-xiaoka.com
Re: [jabberd2] Location of pid file
On Oct 26, 2007 at 12:34 AM, Tomasz Sterna wrote: > Dnia 25-10-2007, Cz o godzinie 17:20 -0700, Mark Doliner pisze: > > > ./configure --prefix=somedir HAVE to move the whole installation to > > > 'somedir'. > > > So we simply cannot use fixed paths. They HAVE to incorporate > $(prefix). > > > > For binaries and documentation I totally agree, but does it make sense > to do that for files created at run time? > > Two examples, that should give some light to the case: > ./configure --prefix=$HOME/test/jabberd > ./configure --prefix=/opt/oss/jabberd > > User definitely cannot keep any data outside $HOME. True, but in that case they could always edit the config files after building. I think it makes more sense for the default value to be appropriate for a system-wide installation rather than for a user's home directory. > And all data of /opt applications should be kept under /opt hierarchy. I don't think files that change should be put in /opt. The FHS describes a /var/opt directory and says, "Package files that are variable (change in normal operation) must be installed in /var/opt." > > Isn't that MORE of a reason to put the pid files in /var/run? > > It's the decision to be taken during build. > For example, if you're packaging for standard GNU/Linux distribution, > yes it's very reasonable to put pid files in /var/run or /var/run/jabber And that isn't currently possible. Even with a prefix of "/" the pidfile is set to /var/jabberd/pid/component.pid > > If /usr is read-only and jabberd tries to write a pid file to > /usr/var/run it would fail. > > That's why it keeps its state data in @localstatedir@ by default, not > in /usr. But if the prefix is "/usr" then @localstatedir@ is set to "/usr/var" (seems like autoconf and the FHS don't quite agree with each other) -Mark ___ Jabberd2 mailing list Jabberd2@lists.xiaoka.com http://lists.xiaoka.com/listinfo.cgi/jabberd2-xiaoka.com
Re: [jabberd2] Location of pid file
On Oct 25, 2007 at 5:01 PM, Tomasz Sterna wrote: > Dnia 25-10-2007, Cz o godzinie 08:27 -0700, Mark Doliner pisze: > > 2. /var/run/jabberd/component.pid > > > > I realize that not all operating systems follow the FHS, but it seems > > like either of these alternatives would at least be an improvement. > > What do other people think? > > > > Similarly, how do people feel about changing the default log file from > > '@localstatedir@/jabberd/log/component.log' to > > '/var/log/jabberd/component.log'? > > ./configure --prefix=somedir HAVE to move the whole installation to > 'somedir'. > So we simply cannot use fixed paths. They HAVE to incorporate $(prefix). For binaries and documentation I totally agree, but does it make sense to do that for files created at run time? > Ant this is the reason for this 'strange' paths. They are relative to > prefix - by default set to /usr/local, which is a place to software > specific to local installation (remember that in big Unix > installations /usr is NFS read-only mounted from shared storage for easy > system upgrades). Isn't that MORE of a reason to put the pid files in /var/run? If /usr is read-only and jabberd tries to write a pid file to /usr/var/run it would fail. The FHS even says, "/usr is shareable, read-only data. That means that /usr should be shareable between various FHS-compliant hosts and must not be written to. Any information that is host-specific or varies with time is stored elsewhere." (http://www.pathname.com/fhs/pub/fhs-2.3.html#THEUSRHIERARCHY) -Mark ___ Jabberd2 mailing list Jabberd2@lists.xiaoka.com http://lists.xiaoka.com/listinfo.cgi/jabberd2-xiaoka.com
[jabberd2] Location of pid file
By default jabberd 2 uses '@localstatedir@/jabberd/pid/component.pid' as the
pid file for each component. This ends up being
'/usr/local/var/jabberd/pid/c2s.pid' for me when I don't specify a prefix.
This seems weird to me. I don't even have a /usr/var or /usr/local/var on
Ubuntu Linux or CentOS Linux, and I can't find any reference to it in the
Filesystem Hierachy Standard (FHX), so I don't understand why autoconf sets
localstatedir to be '${prefix}/var/' in the first place. The FHS suggests that
pid files be placed under /var/run/
(http://www.pathname.com/fhs/pub/fhs-2.3.html#VARRUNRUNTIMEVARIABLEDATA).
Two possible improvements to this path:
1. @localstatedir@/run/jabberd/component.pid
2. /var/run/jabberd/component.pid
I realize that not all operating systems follow the FHS, but it seems like
either of these alternatives would at least be an improvement. What do other
people think?
Similarly, how do people feel about changing the default log file from
'@localstatedir@/jabberd/log/component.log' to '/var/log/jabberd/component.log'?
-Mark
___
Jabberd2 mailing list
Jabberd2@lists.xiaoka.com
http://lists.xiaoka.com/listinfo.cgi/jabberd2-xiaoka.com
Re: [jabberd2] Distinction between jabberd2 and jabberd14
On Oct 25, 2007 at 12:31 AM Tomasz Sterna wrote: > Dnia 25-10-2007, Cz o godzinie 00:02 -0700, Mark Doliner pisze: > > It took me a long time to get a good feel for the distinction between > > jabberd14 and jabberd2. I think the names are probably confusing to > > anyone not familiar with the projects. It seems like it might be a > > good idea for the jabberd2 project to use "jabberd2" in many places > > where it currently uses "jabberd." > > [...] > > This issue was brought many times and not only in context of jabberd. > Apache HTTPd for example. > > My standpoint is, that major rewrite of the software does not need > renaming the project. Major version bump is enough. > jabberd 1.x is not the same software as jabberd 2.x, like apache 1.x is > not the same software as apache 2.x, like Die Hard 1 is not the same > movie as Die Hard 2. > > The problems you mention are local deployment issues. > There is a problem with many current Linux and Unix distributions, that > they do not handle many different versions of same software at the same > time. But this is a distribution problem (deployment issue). There are > distributions, that cope with this well. > > The site name and list name are legacy of the migration. It was linked > in many places, so I think it is too late to change it. > > > > It just seems like it would be helpful for there to be a greater > > distinction between jabberd14 and jabberd2. > > Next release of jabberd14 will be called xmppd, so there should be no > more confusion. Oh I didn't realize that. That should make things much better. Thanks for the clarification! -Mark ___ Jabberd2 mailing list Jabberd2@lists.xiaoka.com http://lists.xiaoka.com/listinfo.cgi/jabberd2-xiaoka.com
[jabberd2] Distinction between jabberd2 and jabberd14
It took me a long time to get a good feel for the distinction between jabberd14 and jabberd2. I think the names are probably confusing to anyone not familiar with the projects. It seems like it might be a good idea for the jabberd2 project to use "jabberd2" in many places where it currently uses "jabberd." For example, the call to AC_INIT() at the top of configure.ac, which affects where plugins are installed (/usr/lib/jabberd2/ instead of /usr/lib/jabberd/). And install config files to /etc/jabberd2/ instead of /etc/jabberd/. "jabberd/component" is used as the ident when using syslog. Maybe change http://jabberd2.xiaoka.com/ to say "jabberd2" instead of "jabberd," etc. One thing that I'm still confused about is Tomasz said in an email to this list "the name of the project is 'jabberd' not 'jabberd2'"[1] Maybe I still don't really know what's going on. It just seems like it would be helpful for there to be a greater distinction between jabberd14 and jabberd2. Please let me know what you think, I hope I'm not stepping on any toes, Mark [1] http://lists.xiaoka.com/htdig.cgi/jabberd2-xiaoka.com/2007-June/000211.html ___ Jabberd2 mailing list Jabberd2@lists.xiaoka.com http://lists.xiaoka.com/listinfo.cgi/jabberd2-xiaoka.com
Re: [jabberd2] s2s.xml
On Oct 21, 2007 at 7:57 PM, ?? wrote: > What does the in s2s.xml represent? > The comment writes "Our ID on the network". > I think the should be the hostname. As does sm.xml and c2s.xml, I > also changed the default to my hostname. > But It's wrong . sm died. > > Please explpain more details > Thanks The id is just a name that jabberd2 uses internally to refer to the server to server process. Leaving it as the default "s2s" should be fine. Each one of the five jabberd2 processes should have their own unique id--it's what they use to talk to each other. Changing the s2s id to the hostname will break things because the sm id is supposed to be set to the hostname. -Mark ___ Jabberd2 mailing list Jabberd2@lists.xiaoka.com http://lists.xiaoka.com/listinfo.cgi/jabberd2-xiaoka.com
Re: [jabberd2] Server to Server can't connet for DNS reason
On Oct 20, 2007 at 4:20 AM, ?? wrote: > Dear Sir > To test the S2S connection function, I installed two jabberd2 servers in > LAN, that are Server1 and Server2. > Because lack of DNS, I modfied the /ets/hosts in each of Linux as follows: > == > 192.168.82.123 server1 > 192.168.82.122 server2. > == > The first client, its jid [EMAIL PROTECTED] , logon the server1, while the > second one [EMAIL PROTECTED] logon the server2. > Then the first client invited the second, but nothing happened. > In the server1's log write:" server2 can't be resolved". > > I don't think the method have any problem, Why can't the two server > communication? Is it nessary to have a DNS server? jabberd2 currently does not use the /etc/hosts file for resolving domain names--it ONLY uses DNS. There are some notes about this http://jabberd2.xiaoka.com/ticket/63 and http://jabberd2.xiaoka.com/wiki/OldTODO So right now, yes, it is necessary to have a DNS server. -Mark ___ Jabberd2 mailing list Jabberd2@lists.xiaoka.com http://lists.xiaoka.com/listinfo.cgi/jabberd2-xiaoka.com
Re: [jabberd2] Clustering
On 17 October 2007 at 2:20 PM, Tomasz Sterna wrote: > Dnia 17-10-2007, Sr o godzinie 13:52 -0700, Mark Doliner pisze: > > > When the router detects bind conflict. It tells the conflicting > > > components to "go to the lower level". > > > > And there would be a bind conflict when two sm instances try to > > connect to router with the id "jabber.org," right? > > And it would also happen when a single user logs in from two > > locations using two different resources? > > Exactly. > I've described it in "Another step:" part of the example. > > > > > > how does c2s decide which sm instance to forward the login to? > > > > > > C2S needs changes in the SM tracking logic. > > > When it receives advertisements of bare-jids, it starts routing by > > > bare-jids, not domain. Same for full-jids. > > > > When a user logs in they get assigned to one of the sm instances, right? > > Right. > The SM that responded to the session creation request. > We need to record it in the SM. :-) > > (Yes, we do crystallize the idea now. :-) ) > > > > How would c2s decide which sm instance they should be assigned to? > > Would the router make that decision somehow? > > Yes. It's router to decide which instance receives the packet addressed > to the SM (domain) itself. Round-robin comes to mind. Or we could count > sessions hanging on the SM and pick the less busy one. > > Using the routing trees (last paragraph) it's trivial to get needed > information. I've uploaded an initial patch for this to http://jabberd2.xiaoka.com/ticket/170 . It's not finished, but I am at least able to have two sm instances connect to one router, and have two clients log in and exchange messages. I wrote a more detailed description in the ticket. -Mark ___ Jabberd2 mailing list Jabberd2@lists.xiaoka.com http://lists.xiaoka.com/listinfo.cgi/jabberd2-xiaoka.com
Re: [jabberd2] Clustering
> -Original Message- > From: Tomasz Sterna > Sent: Wednesday, October 17, 2007 1:34 PM > To: jabberd2 development > Subject: Re: [jabberd2] Clustering > > Dnia 17-10-2007, Śr o godzinie 13:05 -0700, Mark Doliner pisze: > > Doesn't this make the router a bottleneck? > > Router has almost no logic - it only passes packets through. > I'm worried more about the fact that it is a single point of failure. > > We may introduce router mesh. Set of interconnected routers that > exchange their routing information. > A router receiving bound name advertisement records it in the routing > table. Same for routing method (domain/bare-jid/full-jid) change > requests when connected router detects conflict. > > Example: > sm1 - router1 - router2 - sm2 > > router1 table: > sm1 > router2 > sm2* - advertisement record (presence from component) > > router2 table: > sm2 1234 > router1 2345 > sm1* 2345 - advertisement record (presence from component) > > > > > Creating a clustering component [...] > > I don't like the idea of introducing yet another component in the > puzzle. The clustering functionality should be handled in the router. Hmm, it DOES make sense to have the router do the routing... And the router mesh idea seems cleaner than having a cluster component route packets from one cluster to another using the s2s component... > > Also, when would sm bind to a bare jid? > > When the router detects bind conflict. It tells the conflicting > components to "go to the lower level". And there would be a bind conflict when two sm instances try to connect to router with the id "jabber.org," right? And it would also happen when a single user logs in from two locations using two different resources? > > how does c2s decide which sm instance to forward the login to? > > C2S needs changes in the SM tracking logic. > When it receives advertisements of bare-jids, it starts routing by > bare-jids, not domain. Same for full-jids. When a user logs in they get assigned to one of the sm instances, right? How would c2s decide which sm instance they should be assigned to? Would the router make that decision somehow? -Mark ___ Jabberd2 mailing list Jabberd2@lists.xiaoka.com http://lists.xiaoka.com/listinfo.cgi/jabberd2-xiaoka.com
Re: [jabberd2] Clustering
Doesn't this make the router a bottleneck? Creating a clustering component as specified at http://jabberd2.xiaoka.com/wiki/Clustering appeals to me because it allows for multiple routers in addition to just multiple sm's. It WOULD be nice to be able to take advantage of multiple processors in a single machine, though. Also, when would sm bind to a bare jid? Would it happen when the sm process is started? Or would it happen after the user authenticates? If it happens after authentication, how does c2s decide which sm instance to forward the login to? -Mark > -Original Message- > From: [EMAIL PROTECTED] [mailto:jabberd2- > [EMAIL PROTECTED] On Behalf Of Tomasz Sterna > Sent: Wednesday, October 17, 2007 11:49 AM > To: jabberd2 development > Subject: Re: [jabberd2] Clustering > > Dnia 17-10-2007, Śr o godzinie 10:34 -0700, Mark Doliner pisze: > > I'm interested in adding clustering support to jabberd2. > > Most of us are. :-) > > > We're noticing that sm is definitely a bottleneck in our jabberd2 > installation, > > Yes. SM is definitely a bottleneck for big deployments. > > > I have an idea how to do it a flexible and efficient way. > A way that would fix the SM problem too. > > I call it "dynamic routing tables". > The idea is similar to the way TCP network handles the network > segmentation. > > The router currently holds a list of connected components and its bound > names. It uses this list to make a decision where to put a packet coming > through. > > componentport > -- > c2s 1 > resolver 1 > example.com 1 > test.org 1 > test2.org1 > > So, when it handles packet destined to [EMAIL PROTECTED]/res it puts it > to 1 stream. (or falls back to defaultroute) > > One component is able to bind more than one name. > > But no more than one component can bind a name. Logically - router would > not know where to put the packet in. > > We need to change the way, this bind conflict is handled. > Instead of disallowing new bind of 'example.com' to a component, router > would need to signal to the requesting component and component on port > 1, that they need to switch to bare-jid binding from domain binding. > > Then 1 and 1 would request binds of all it's bare jid's and the > routing table would look: > > component port > --- > c2s 1 > resolver 1 > [EMAIL PROTECTED] 1 > [EMAIL PROTECTED] 1 > [EMAIL PROTECTED] 1 > test.org 1 > test2.org 1 > [EMAIL PROTECTED] 1 > [EMAIL PROTECTED] 1 > > This way, router receiving packet for [EMAIL PROTECTED]/res would put it > to 1 stream and for [EMAIL PROTECTED]/home would go to 1. > Packets to SM itself would go to one of the shared routes - for load > balancing. round-robin? > > Another step: component on port 1 requests bind of [EMAIL PROTECTED] > (user connected with another resource) generating conflict with > component on port 1. Router would then to signal to both components, > that they need to switch to full-jid binding from bare-jid binding. > > After components bind all full JIDs the routing table would look like: > > component port > --- > c2s1 > resolver 1 > [EMAIL PROTECTED]/cl1 > [EMAIL PROTECTED]/res 1 > [EMAIL PROTECTED]/res1 > [EMAIL PROTECTED]/res1 > test.org 1 > test2.org 1 > [EMAIL PROTECTED]/home 1 > [EMAIL PROTECTED]/work 1 > > And router can do a routing for [EMAIL PROTECTED]/res to 1 stream. > On this level we would need to track resource priorities too, but this > is a minor. > > This dynamically expanding router tables would allow us to plug many SM > processes handling same domain even on single host. That would > distribute load to many OS processes. > > Balancer code could be reused to implement more than one default route > S2S processes. Not only for incoming, but outgoing connections too. > (When more than one component requests default bind.) > > > Implementation note: > This routing table would rather be a routing tree with domains on first > level, bare-jids on the second and full jids on the third. > > -- > /\_./o__ Tomasz Sterna > (/^/(_^^' Xiaoka.com > ._.(_.)_ XMPP: [EMAIL PROTECTED] > > ___ > Jabberd2 mailing list > Jabberd2@lists.xiaoka.com > http://lists.xiaoka.com/listinfo.cgi/jabberd2-xiaoka.com ___ Jabberd2 mailing list Jabberd2@lists.xiaoka.com http://lists.xiaoka.com/listinfo.cgi/jabberd2-xiaoka.com
[jabberd2] Clustering
Hi! I'm interested in adding clustering support to jabberd2. We're noticing that sm is definitely a bottleneck in our jabberd2 installation, and we think clustering would eliminate this. I've read through all the clustering pages on the wiki and think I have a pretty good idea of what needs to be done. Has anyone done any work on this? Or have any suggestions or advice? I saw that there's a patch from Bruce Campbell at http://jabberd2.xiaoka.com/ticket/25 for some minor changes to sm. -Mark ___ Jabberd2 mailing list Jabberd2@lists.xiaoka.com http://lists.xiaoka.com/listinfo.cgi/jabberd2-xiaoka.com
