Hi James!
James Bunton schrieb:
> Ok, current SVN trunk now hashes with MD5s.
I think it would be nice, if py-transports would hash the same way as
wpjabber and jabberd14 are hashing the spools. I think this would make
it more clear to the user, where to find files.
The way wpjabber introduced hashing is the following: calculcate the CRC
over the filename (format "username.xdb" in case of wpjabber and
jabberd14). Use the second and third character as the first directory,
and the 5th and 6th character as the second directory name.
E.g. the file "mawis.xml" is placed in the directory 77/35/mawis.xml and
"james.xml" would be placed in the directory 77/5B/james.xml.
This is the format I reused when implementing hashed directory names in
jabberd14.
Tot kijk
Matthias
From [EMAIL PROTECTED] Fri Jul 14 16:39:20 2006
From: [EMAIL PROTECTED] (Matthias Wimmer)
Date: Fri Jul 14 16:39:30 2006
Subject: [py-transports] Problems with current directory hashing method
In-Reply-To: <[EMAIL PROTECTED]>
References: <[EMAIL PROTECTED]> <[EMAIL PROTECTED]> <[EMAIL PROTECTED]>
<[EMAIL PROTECTED]> <[EMAIL PROTECTED]>
<[EMAIL PROTECTED]>
Message-ID: <[EMAIL PROTECTED]>
Matthias Wimmer schrieb:
>> Ok, current SVN trunk now hashes with MD5s.
> The way wpjabber introduced hashing is the following: calculcate the CRC
> over the filename (format "username.xdb" in case of wpjabber and
> jabberd14). Use the second and third character as the first directory,
> and the 5th and 6th character as the second directory name.
What I forgot: it's a CRC32 calculation and the result is represented in
upper-case hexadecimal digits.
I guess the reason why CRC32 has been choosen in favour over MD5 is,
that calculating a CRC value is less CPU expensive. - The reason why the
2nd, 3rd, 5th and 6th character is taken for the filenames is (I think
so), that this allowed the implementor of wpjabber (which is written in
C) to reuse already allocated memory for the string and to just change
the 1st dn 4th character to '/' and cut the string after that.
Matthias
From [EMAIL PROTECTED] Fri Jul 14 18:02:58 2006
From: [EMAIL PROTECTED] (Chad Jannusch)
Date: Fri Jul 14 18:03:36 2006
Subject: [py-transports] Problems with current directory hashing method
In-Reply-To: <[EMAIL PROTECTED]>
References: <[EMAIL PROTECTED]>
<[EMAIL PROTECTED]>
<[EMAIL PROTECTED]>
<[EMAIL PROTECTED]>
<[EMAIL PROTECTED]>
Message-ID: <[EMAIL PROTECTED]>
On 7/14/06, James Bunton <[EMAIL PROTECTED]> wrote:
>
>
> Ok, current SVN trunk now hashes with MD5s.
>
> If you run svn up and restart the transport then your spool dir will
> be automatically migrated.
>
> Also, the avatar cache directory will be moved to the root of
> <spooldir/>. That is, alongside the msn.host.com folders so that if
> anybody has multiple PyMSNt processes running on different domains
> they can share the avatars cache.
>
> I've tested this upgrade procedure a couple of times, and the worst
> it can do is move your files into bad places. I'd appreciate people
> trying it out though.
>
> Thanks :)
>
> ---
>
> James
>
> _______________________________________________
> py-transports mailing list
> [email protected]
> http://www.modevia.com/cgi-bin/mailman/listinfo/py-transports
>
Hello,
I upgraded to the latest SVN and restarted the PyMSNt process. In my
original config.xml the spooldir wasn't defined so PyMSNt just defaulted to
the directory it was lanched from. Since the spooldir wasn't defined I
recieved the following error:
Moving the avatar directory from msn.host.com up to the spool directory.
An error occurred during one of the automatic data update routines. Please
report this bug.
Traceback (most recent call last):
File "/opt/PyMSNt/PyMSNt.py", line 12, in ?
main.main()
File "/opt/PyMSNt/src/main.py", line 391, in main
app = App()
File "/opt/PyMSNt/src/main.py", line 331, in __init__
housekeep.init()
File "/opt/PyMSNt/src/housekeep.py", line 26, in init
noteListF[noteList.index(note)]()
File "/opt/PyMSNt/src/housekeep.py", line 136, in doMD5HashDirUpgrade
shutil.rmtree(config.spooldir + X + "avatars")
File "/usr/lib/python2.4/shutil.py", line 155, in rmtree
onerror(os.listdir, path, sys.exc_info())
File "/usr/lib/python2.4/shutil.py", line 153, in rmtree
names = os.listdir(path)
OSError: [Errno 2] No such file or directory: '/avatars'
After defining the spooldir in the config.xml file the upgrade worked fine,
I was still told that it couldn't move my avatars directory and that I
should just delete it but the spool directory was processed correctly.
Just figured I'd pass it on,
Chad Jannusch
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://modevia.com/pipermail/py-transports/attachments/20060714/a1966a82/attachment.htm
From [EMAIL PROTECTED] Fri Jul 14 21:05:26 2006
From: [EMAIL PROTECTED] (John Huang)
Date: Fri Jul 14 21:05:33 2006
Subject: [py-transports] twisted problem
Message-ID: <[EMAIL PROTECTED]>
I am running Windows 2003. I configured pymsnt-0.11.1 according to
instructions I found in the forum and websites.
When I try to use python pysmnt.py I receive the follwing error:
Unable to install any good reactors (kqueue, epoll, poll).
We fell back to using select. You may have scalability problems.
This reactor will not support more than 1024 connections at a time.
Could not find the XML DOM. If you're using Twisted 2.x make sure you
have twist
ed.words installed.
Traceback (most recent call last):
File "PyMSNt.py", line 11, in ?
import main
File "C:\pymsnt-0.11.1\src\main.py", line 36, in ?
import xmlconfig
File "C:\pymsnt-0.11.1\src\xmlconfig.py", line 4, in ?
from tlib import xmlw
File "C:\pymsnt-0.11.1\src\tlib\xmlw.py", line 80, in ?
from twisted.xish.domish import SuxElementStream, Element,
unescapeFromXml
ImportError: No module named xish.domish
Initially I had python 2.4 installed with Twisted 2.4, then I downgraded
to python 2.3 and installed 2.4 Twisted for 2.3 Python but still receive
the same error. I see a Twisted folder in my Python23\Lib\site-packages\
folder along with words folder in there, so I think Python and Twisted
are installed correctly. Is there something I am missing?
Thanks,
John
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://modevia.com/pipermail/py-transports/attachments/20060714/231a549d/attachment-0001.html
From [EMAIL PROTECTED] Sat Jul 15 01:32:09 2006
From: [EMAIL PROTECTED] (James Bunton)
Date: Sat Jul 15 01:32:17 2006
Subject: [py-transports] Problems with current directory hashing method
In-Reply-To: <[EMAIL PROTECTED]>
References: <[EMAIL PROTECTED]>
<[EMAIL PROTECTED]>
<[EMAIL PROTECTED]>
<[EMAIL PROTECTED]>
<[EMAIL PROTECTED]>
<[EMAIL PROTECTED]>
Message-ID: <[EMAIL PROTECTED]>
On 15/07/2006, at 4:02 AM, Chad Jannusch wrote:
> On 7/14/06, James Bunton <[EMAIL PROTECTED]> wrote:
>>
>>
>> Ok, current SVN trunk now hashes with MD5s.
>>
>> If you run svn up and restart the transport then your spool dir will
>> be automatically migrated.
>>
>> Also, the avatar cache directory will be moved to the root of
>> <spooldir/>. That is, alongside the msn.host.com folders so that if
>> anybody has multiple PyMSNt processes running on different domains
>> they can share the avatars cache.
>>
>> I've tested this upgrade procedure a couple of times, and the worst
>> it can do is move your files into bad places. I'd appreciate people
>> trying it out though.
>>
>> Thanks :)
>>
>> ---
>>
>> James
>>
>> _______________________________________________
>> py-transports mailing list
>> [email protected]
>> http://www.modevia.com/cgi-bin/mailman/listinfo/py-transports
>>
>
>
> Hello,
> I upgraded to the latest SVN and restarted the PyMSNt process. In my
> original config.xml the spooldir wasn't defined so PyMSNt just
> defaulted to
> the directory it was lanched from. Since the spooldir wasn't defined I
> recieved the following error:
>
> Moving the avatar directory from msn.host.com up to the spool
> directory.
> An error occurred during one of the automatic data update routines.
> Please
> report this bug.
> Traceback (most recent call last):
> File "/opt/PyMSNt/PyMSNt.py", line 12, in ?
> main.main()
> File "/opt/PyMSNt/src/main.py", line 391, in main
> app = App()
> File "/opt/PyMSNt/src/main.py", line 331, in __init__
> housekeep.init()
> File "/opt/PyMSNt/src/housekeep.py", line 26, in init
> noteListF[noteList.index(note)]()
> File "/opt/PyMSNt/src/housekeep.py", line 136, in doMD5HashDirUpgrade
> shutil.rmtree(config.spooldir + X + "avatars")
> File "/usr/lib/python2.4/shutil.py", line 155, in rmtree
> onerror(os.listdir, path, sys.exc_info())
> File "/usr/lib/python2.4/shutil.py", line 153, in rmtree
> names = os.listdir(path)
> OSError: [Errno 2] No such file or directory: '/avatars'
>
> After defining the spooldir in the config.xml file the upgrade
> worked fine,
> I was still told that it couldn't move my avatars directory and that I
> should just delete it but the spool directory was processed correctly.
>
> Just figured I'd pass it on,
> Chad Jannusch
Thanks. That's fixed in r199. Don't suppose you have a backup so you
can run it again to confirm?
---
James
