RE: [JDEV] Scaling 1.4.2 with xdb_file... And the patch
> Great, though a diff would be perferable - in case one has > other changes > in the xdb_file.c! It's based on the original 1.4.2 distro so a diff with a stock xdb_file will give you just the changes I've made. > Here's a little one-liner shell script you can use to > re-arrange all the > data files in your spool directory. It should be run from the spool > directory (ie. "cd /var/spool/jabber" or wherever). > > ls *.xml | perl -n -e 'chomp; /^(.)(.)/; mkdir $1 unless -d > $1; mkdir "$1/$1$2\n" unless -d "$1/$1$2"; rename $_, "$1/$1$2/$_";' > > To "undo" you can simply swap the order of the arguments to rename. Thanks for the the script. I'm scaling up from zero users, but I'm sure another file scaling zealot will find it useful. Thanks, Mike ___ jdev mailing list [EMAIL PROTECTED] http://mailman.jabber.org/listinfo/jdev
Re: [JDEV] Scaling 1.4.2 with xdb_file... And the patch
Mike Prince wrote: I've included the patched xdb_file for those stuck in 1.4.2 land and are so inclined. To use the patch just drop in it and compile. It WILL NOT reorg your existing file store (you'll have to write a script to move the files around). The new directory is arranged as ./spool/hostname/m/me/mike.xml where the first fanout directory under hostname is the first character of the username, and the second fanout directory is the first and the last characters of the username. Pretty simple. Great, though a diff would be perferable - in case one has other changes in the xdb_file.c! Here's a little one-liner shell script you can use to re-arrange all the data files in your spool directory. It should be run from the spool directory (ie. "cd /var/spool/jabber" or wherever). ls *.xml | perl -n -e 'chomp; /^(.)(.)/; mkdir $1 unless -d $1; mkdir "$1/$1$2\n" unless -d "$1/$1$2"; rename $_, "$1/$1$2/$_";' To "undo" you can simply swap the order of the arguments to rename. -R ___ jdev mailing list [EMAIL PROTECTED] http://mailman.jabber.org/listinfo/jdev
RE: [JDEV] Scaling 1.4.2 with xdb_file... And the patch
I solved my own problem. I added support for directory fanout below the spool directory (isn't open source wonderful). I've included the patched xdb_file for those stuck in 1.4.2 land and are so inclined. To use the patch just drop in it and compile. It WILL NOT reorg your existing file store (you'll have to write a script to move the files around). The new directory is arranged as ./spool/hostname/m/me/mike.xml where the first fanout directory under hostname is the first character of the username, and the second fanout directory is the first and the last characters of the username. Pretty simple. Thanks, Mike > -Original Message- > From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On > Behalf Of Mike Prince > Sent: Thursday, February 27, 2003 10:11 AM > To: [EMAIL PROTECTED] > Subject: [JDEV] Scaling 1.4.2 with xdb_file > > > I'm scaling up my service (running on JabberD 1.4.2 of course > ;) and am running into the expected xdb scaling problems. > Further complicating the issue is that I'm using extended > namespaces in my XMPP. I'd like to prove my service can > handle 250K accounts. > > The desired solution should be quick to implement (hours to > days vs. days to weeks). Note that I fully expect to move > over to an SQL solution in the long term. > > After looking at both xdb_file and xdb_sql, I believe a quick > hack at xdb_file is the answer. Currently xdb_file puts all > the user XML files in a single directory. I'd increase the > fan out of the directory, changing ./spool/localhost/mike.xml > to ./spool/localhost/m/mi/mike.xml > > Has anyone already done this? Is there a better path? > > Thanks, > > Mike > > > ___ > jdev mailing list > [EMAIL PROTECTED] > http://mailman.jabber.org/listinfo/jdev > xdb_file.c Description: Binary data
[JDEV] Scaling 1.4.2 with xdb_file
I'm scaling up my service (running on JabberD 1.4.2 of course ;) and am running into the expected xdb scaling problems. Further complicating the issue is that I'm using extended namespaces in my XMPP. I'd like to prove my service can handle 250K accounts. The desired solution should be quick to implement (hours to days vs. days to weeks). Note that I fully expect to move over to an SQL solution in the long term. After looking at both xdb_file and xdb_sql, I believe a quick hack at xdb_file is the answer. Currently xdb_file puts all the user XML files in a single directory. I'd increase the fan out of the directory, changing ./spool/localhost/mike.xml to ./spool/localhost/m/mi/mike.xml Has anyone already done this? Is there a better path? Thanks, Mike ___ jdev mailing list [EMAIL PROTECTED] http://mailman.jabber.org/listinfo/jdev
RE: [JDEV] Re: Open Source?
I agree with Ulrich,
There are allot of open source companies like: mySQL, Zend, etc... Those
provide open source licenses and also commercial services and products,
maybe you guys at Rhymbox should do it that way too.
By the way I am a big fan of your client Rhymbox, it is beautiful and
very easy to use, I wish you guys allot of success in your endeavors,
and hope you make a client that will top the only rival for me outside
jabber client's the Trilian client, I really think that inside the
jabber client's you have no rival.
My wish list: A bit more extensibility thru the use of plug-ins.
The ability to see the stream in plain text like other
client's do is something that I wish for.
OOB made easy, drag and drop exchange of files, in out
of band connections.
PS: I know this is not Rhymbox list, but I think this licensing issue is
something that all developers will have to face sooner or later. And
having a reference on the list might help, maybe a licensing FAQ, on
jabber.org..
Just my 2cents...
Best Regards,
Daniel MD
-Mensagem original-
De: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Em nome de
[EMAIL PROTECTED]
Enviada: quinta-feira, 27 de Fevereiro de 2003 11:35
Para: [EMAIL PROTECTED]
Assunto: Re: [JDEV] Re: Open Source?
Hi,
I guess it depends on the interpretation of open source. As long as
rhymbox
is not GPL'd, the term open source is more or less meaningless, for me
as a
developer all my source code is open source :)
I do not to stab sebastiaan, but i'd choose a proper license model... it
helps a lot. several companys wanted to obtain and buy/whatever
sourcecode from
me, too, and with a proper licence model everything gets sorted quite
fast.
hope this helps...
ulrich
> Ok, time for some clarification.
>
> I'm one of the two RhymBox developers. Shalom contacted me a few days
> ago with this very same question and I answered his questions, as I do
> with every request. I did not receive any response from him.
Whatever.
>
>
> > Shalom Levytam wrote:
> >
> > Thanks, this helps a bit.
> >
> > I guess Rhymbox is closed source then...
>
> Take a look in the subdirectory "src" where you installed RhymBox. It
> contains all scripts, HTML and CSS that you need to modify the GUI.
> The source code for the .exe framework is not distributed because
pretty
> much the only reason you would need that is to create commercial
> application based on RhymBox. And we don't want that right now.
> We do however give out that source code to friends, people we trust,
> people who pay, etc. Contact me directly for more about this. JDEV
is
> not a mailing list about one client.
>
>
> > Tijl Houtbeckers wrote:
> > "
> > Free services
> > RhymBox client and Jabber server
> > We offer a next-generation Jabber client. RhymBox is open source and
> > free of charge for personal use. "
> >
> > It doesn't state what licence though, but "free of charge for
personal
> > use" sort of implies not free for commercial use Perhaps a more
clear
> > licence is provided in the download (since it's a DHTML application
I
> > assume source is bundled with the download (or maybe there's a way
to
> > compile DHTML these days).
>
> There are ways to conceil DHTML code but they are flawed, hardly worth
> the effort and the exact opposite of what "open-source" means to the
> RhymBox project. The license states something like "you can modify
the
> source code as long as you don't make a million bucks selling it
behind
> our backs". Not in those words though. ;-)
> The license is the first screen of the installer, and is also placed
in
> the installation directory ("License.txt") for reviewing.
> We want people to be able to tweak the program interface, to learn
about
> the Jabber protocol, to have an example of programming for the
different
> libraries and languages that we use. Nothing more.
>
>
> --
> Sebastiaan
>
> ___
> jdev mailing list
> [EMAIL PROTECTED]
> http://mailman.jabber.org/listinfo/jdev
>
--
Ulrich Staudinger http://www.die-horde.de http://www.igpp.de
Product Manager @ http://complat.sourceforge.net/jnlp/
JID: [EMAIL PROTECTED]
___
jdev mailing list
[EMAIL PROTECTED]
http://mailman.jabber.org/listinfo/jdev
___
jdev mailing list
[EMAIL PROTECTED]
http://mailman.jabber.org/listinfo/jdev
Re: [JDEV] Re: Open Source?
I would suggest you read the Open Source Definition (at
http://www.opensource.org/docs/definition.php) before calling your
software Open-Source. In particular, your licensing does not meet
sections 1 and 6.
-David Waite
Sebastiaan Deckers wrote:
Ok, time for some clarification.
I'm one of the two RhymBox developers. Shalom contacted me a few days
ago with this very same question and I answered his questions, as I do
with every request. I did not receive any response from him. Whatever.
Shalom Levytam wrote:
Thanks, this helps a bit.
I guess Rhymbox is closed source then...
Take a look in the subdirectory "src" where you installed RhymBox. It
contains all scripts, HTML and CSS that you need to modify the GUI.
The source code for the .exe framework is not distributed because
pretty much the only reason you would need that is to create
commercial application based on RhymBox. And we don't want that right
now.
We do however give out that source code to friends, people we trust,
people who pay, etc. Contact me directly for more about this. JDEV
is not a mailing list about one client.
> Tijl Houtbeckers wrote:
"
Free services
RhymBox client and Jabber server
We offer a next-generation Jabber client. RhymBox is open source and
free of charge for personal use. "
It doesn't state what licence though, but "free of charge for
personal use" sort of implies not free for commercial use Perhaps a
more clear licence is provided in the download (since it's a DHTML
application I assume source is bundled with the download (or maybe
there's a way to compile DHTML these days).
There are ways to conceil DHTML code but they are flawed, hardly worth
the effort and the exact opposite of what "open-source" means to the
RhymBox project. The license states something like "you can modify
the source code as long as you don't make a million bucks selling it
behind our backs". Not in those words though. ;-)
The license is the first screen of the installer, and is also placed
in the installation directory ("License.txt") for reviewing.
We want people to be able to tweak the program interface, to learn
about the Jabber protocol, to have an example of programming for the
different libraries and languages that we use. Nothing more.
___
jdev mailing list
[EMAIL PROTECTED]
http://mailman.jabber.org/listinfo/jdev
Re: [JDEV] jabberd patch
> If you want to put a hack in your client that's fine by me but it > doesn't mean other clients are obligated to. IMHO it's the protocol and > the server that are misbehaving since they kick the client without > giving any reason. Yes the server does not give the specific reason but it does tell you something big has happened and clients should not be ignoring that and just trying to reconnect IMO, until the error codes are implemented in those cases human intervention should be needed to get the client to reconnect, in most cases such errors will happen very infrequently unless something is badly wrong at the server end or the client end (e.g. sending invalid xml) so I cant really see what you problem is. > So you can choose between a few options here: > 1. accept the protocol is broken and not implement reconnect, thus > compromising on the functionality of your client (some people will > switch to clients with a different client if they can) 2. Introduce a > bad hack to solve this bug in some cases. Compromises functionality in > some other cases. 3. Leave things as they, accept the *protocol* is > flawed and should be fixed. The more the problem will occur the faster > it will be fixed properly and the sooner serveradmins will upgrade. > Don't compromise on functionality because the error is in the protocol > and servers. > > As you can see I prefer 3. You can choose 1 or 2 if you want, but you > can't acuse developers who choose 3 of misbehaving. It is not bad IMO and helps stop it in most cases and will help promote updates for reasons I will detail later in this email. Also the protocol is already being fixed/enhanced with error codes. > >Ah but hopefully we can get the stream error codes into jabberd2 > >before it goes final so they can be used to reliably determine the > >reason for disconnection. > > I hope so too.. there has been discussion on the XMPPWG but I haven't > read the outcome, if any (do the mailinglist archives have a search > function yet?) IMO it seems to be reaching concensus. > I really still don't think you should solve problems by introducing > hacks into the client. If we fix the protocol, and then upgrade the > clients *properly* this will only be a good stimulation for those > thousands of servers to upgrade. If we introduce a hack servers are > less likely to upgrade and clients less likely to implement a proper > fix because it already works "properly" in a large part of all cases. > This way the hack will become "semi-offical" and we all know what kind > of problems that brings along. I dont consider it a hack and fixing the protocol is already in progress. Also since just using the stream:error as Exodus does will stop auto-reconnecting in the cases you mention were it might still want to try auto-reconnecting I would argue that it would be a source of stimulation for the server devs to update their servers to support error codes, and it would be stimulation for client devs too. > This is exactly the kind of thinking I'm opposing here. Just because a > (very) large part of the servers supports your hack that's a reason to > go ahead with it? This is what incompatabilities are made of! It goes > directly agains the thinking behind open standards because it corrupts > them. BTW it is not my "hack", it had already been thought of by someone else (the Exodus devs), it in no way introduces incompatibilities since having stream:error's is a standard part of the protocol AFAIK, so it is not corrupting anything it is following the protocol and I resent the implication. > Example: > Let's say that jabberd1.4.2 for win32 and linux do not send a > when shutting down. According to you I'm not allowed to > reconnect if it sends a stream:error, but it doesn't so I reconnect. > Now I'm writing a new jabber server SuperExtraXMPPJabberD. When it > shuts down to restart it's very polite and sends a The > server is restarting (or if you're still into matching > CDATA it sends "Disconnected" instead). However users start complaining > that their clients won't autoreconnect like with jabberd. I "explain" > to them this lenghty discussion, but some don't care and install a > different server. Now my marketshare is even smaller :( Then what you do is use the error codes which are not far off being decided from what I can gather. > I have some free time.. and there's a new standard with errorcodes. I > implement it.. however, the clients don't implement it as fast as they > could cause they already solved the problem. Well solved.. they used > some hack you suggested. But hey, it works on all the servers with a > significant deployment eh? Well it wont look bad for you since if the clients dont follow the new error codes and dont try auto-reconnecting when they can do it looks bad for the client, all you do it recommend a client to them that follows the newer standard which im sure most (if not all) of the good clients will follow, thats the great thing about open protocols with
Re: [JDEV] jabberd patch
"Matthew A. Miller" <[EMAIL PROTECTED]> wrote on 26-2- 2003 2:42:45: > >The more I read this thread, the more I agree with both sides... > >Is there middle ground to be had? What if XMPP were defined in such a >way that if a connection/session attempted to auth as an already online >one, it was up to the server. If the server decided it was allowed, it >sent the appropriate iq-error to the original connection (302?) along >with the disconnect. If the server decided it wasn't, it sent a 409 or >405 iq-error to the new connection along with the diconnect. > >Does the above sound reasonable? If so, I'll make a point of >approaching the XMPP I-D authors on it. In the current XMPP spec (draft-ietf-xmpp-core-04) we can find this: " The following codes are defined for stream-level errors: 302 - Redirect 400 - Bad XML 404 - Unknown Host 410 - Gone 500 - Internal Server Error 505 - Version Not Supported " I'd like to see 409 included here as well. 409 is already including amongst the standard error, so it's can be used for denying authentication: " 409 (Conflict) - Code 409 is returned when a request cannot be fulfilled because of an inherent conflict (e.g., because a client attempts to authorize a resouce name that is already in use). " The description would have to be changed a little too since it's not request-specific anymore. -- Tijl Houtbeckers Software Engineer @ Splendo The Netherlands ___ jdev mailing list [EMAIL PROTECTED] http://mailman.jabber.org/listinfo/jdev
Re: [JDEV] karma settings
Justin Georgeson wrote: I read the notes in jabber.xml about karma, but none of e the formulas are explained. Do I have to read the code to find out how the 'low' karma settings translate to the actual badnwidth given? AFAIK the explanation is given only in jabber.xml, with some examples. -- Fabio Forno - PhD Student Politecnico di Torino - Dip. Automatica ed Informatica C.so Duca degli Abruzzi 24 - 10129 Torino (Italy) Phone: +39 011 564 7137 - JabberId: [EMAIL PROTECTED] ___ jdev mailing list [EMAIL PROTECTED] http://mailman.jabber.org/listinfo/jdev
Re: [JDEV] Re: Open Source?
Hi,
I guess it depends on the interpretation of open source. As long as rhymbox
is not GPL'd, the term open source is more or less meaningless, for me as a
developer all my source code is open source :)
I do not to stab sebastiaan, but i'd choose a proper license model... it
helps a lot. several companys wanted to obtain and buy/whatever sourcecode from
me, too, and with a proper licence model everything gets sorted quite fast.
hope this helps...
ulrich
> Ok, time for some clarification.
>
> I'm one of the two RhymBox developers. Shalom contacted me a few days
> ago with this very same question and I answered his questions, as I do
> with every request. I did not receive any response from him. Whatever.
>
>
> > Shalom Levytam wrote:
> >
> > Thanks, this helps a bit.
> >
> > I guess Rhymbox is closed source then...
>
> Take a look in the subdirectory "src" where you installed RhymBox. It
> contains all scripts, HTML and CSS that you need to modify the GUI.
> The source code for the .exe framework is not distributed because pretty
> much the only reason you would need that is to create commercial
> application based on RhymBox. And we don't want that right now.
> We do however give out that source code to friends, people we trust,
> people who pay, etc. Contact me directly for more about this. JDEV is
> not a mailing list about one client.
>
>
> > Tijl Houtbeckers wrote:
> > "
> > Free services
> > RhymBox client and Jabber server
> > We offer a next-generation Jabber client. RhymBox is open source and
> > free of charge for personal use. "
> >
> > It doesn't state what licence though, but "free of charge for personal
> > use" sort of implies not free for commercial use Perhaps a more clear
> > licence is provided in the download (since it's a DHTML application I
> > assume source is bundled with the download (or maybe there's a way to
> > compile DHTML these days).
>
> There are ways to conceil DHTML code but they are flawed, hardly worth
> the effort and the exact opposite of what "open-source" means to the
> RhymBox project. The license states something like "you can modify the
> source code as long as you don't make a million bucks selling it behind
> our backs". Not in those words though. ;-)
> The license is the first screen of the installer, and is also placed in
> the installation directory ("License.txt") for reviewing.
> We want people to be able to tweak the program interface, to learn about
> the Jabber protocol, to have an example of programming for the different
> libraries and languages that we use. Nothing more.
>
>
> --
> Sebastiaan
>
> ___
> jdev mailing list
> [EMAIL PROTECTED]
> http://mailman.jabber.org/listinfo/jdev
>
--
Ulrich Staudinger http://www.die-horde.de http://www.igpp.de
Product Manager @ http://complat.sourceforge.net/jnlp/
JID: [EMAIL PROTECTED]
___
jdev mailing list
[EMAIL PROTECTED]
http://mailman.jabber.org/listinfo/jdev
Re: [JDEV] Re: Open Source?
Ok, time for some clarification.
I'm one of the two RhymBox developers. Shalom contacted me a few days
ago with this very same question and I answered his questions, as I do
with every request. I did not receive any response from him. Whatever.
Shalom Levytam wrote:
Thanks, this helps a bit.
I guess Rhymbox is closed source then...
Take a look in the subdirectory "src" where you installed RhymBox. It
contains all scripts, HTML and CSS that you need to modify the GUI.
The source code for the .exe framework is not distributed because pretty
much the only reason you would need that is to create commercial
application based on RhymBox. And we don't want that right now.
We do however give out that source code to friends, people we trust,
people who pay, etc. Contact me directly for more about this. JDEV is
not a mailing list about one client.
> Tijl Houtbeckers wrote:
"
Free services
RhymBox client and Jabber server
We offer a next-generation Jabber client. RhymBox is open source and
free of charge for personal use. "
It doesn't state what licence though, but "free of charge for personal
use" sort of implies not free for commercial use Perhaps a more clear
licence is provided in the download (since it's a DHTML application I
assume source is bundled with the download (or maybe there's a way to
compile DHTML these days).
There are ways to conceil DHTML code but they are flawed, hardly worth
the effort and the exact opposite of what "open-source" means to the
RhymBox project. The license states something like "you can modify the
source code as long as you don't make a million bucks selling it behind
our backs". Not in those words though. ;-)
The license is the first screen of the installer, and is also placed in
the installation directory ("License.txt") for reviewing.
We want people to be able to tweak the program interface, to learn about
the Jabber protocol, to have an example of programming for the different
libraries and languages that we use. Nothing more.
--
Sebastiaan
___
jdev mailing list
[EMAIL PROTECTED]
http://mailman.jabber.org/listinfo/jdev
