Re: [Twisted-Python] Python3: should paths be bytes or str?
On 01:26 am, wolfgang@rohdewald.de wrote: The porting guide says No byte paths in sys.path. What porting guide is that? doc for FilePath says On both Python 2 and Python 3, paths can only be bytes. I stumbled upon this while trying to find out how much work it might be to make bin/trial run with python3 admin/run-python3-tests already passes for all twisted.spread related tests but I still need to clean up a lot. after adding an assert to FilePath.__init__, python3 bin/trial ... gives File "/home/wr/ssdsrc/Twisted/twisted/scripts/trial.py", line 601, in run config.parseOptions() File "/home/wr/ssdsrc/Twisted/twisted/python/usage.py", line 277, in parseOptions self.postOptions() File "/home/wr/ssdsrc/Twisted/twisted/scripts/trial.py", line 472, in postOptions _BasicOptions.postOptions(self) File "/home/wr/ssdsrc/Twisted/twisted/scripts/trial.py", line 382, in postOptions self['reporter'] = self._loadReporterByName(self['reporter']) File "/home/wr/ssdsrc/Twisted/twisted/scripts/trial.py", line 369, in _loadReporterByName for p in plugin.getPlugins(itrial.IReporter): File "/home/wr/ssdsrc/Twisted/twisted/plugin.py", line 209, in getPlugins allDropins = getCache(package) File "/home/wr/ssdsrc/Twisted/twisted/plugin.py", line 134, in getCache mod = getModule(module.__name__) File "/home/wr/ssdsrc/Twisted/twisted/python/modules.py", line 781, in getModule return theSystemPath[moduleName] File "/home/wr/ssdsrc/Twisted/twisted/python/modules.py", line 702, in __getitem__ self._findEntryPathString(moduleObject)), File "/home/wr/ssdsrc/Twisted/twisted/python/modules.py", line 627, in _findEntryPathString if _isPackagePath(FilePath(topPackageObj.__file__)): File "/home/wr/ssdsrc/Twisted/twisted/python/filepath.py", line 664, in __init__ assert isinstance(path, bytes), 'path must be bytes: %r' % (path,) AssertionError: path must be bytes: '/home/wr/ssdsrc/Twisted/twisted/__init__.py' If paths are being represented using unicode somewhere and you want to use them with FilePath then you have to encode them (or you have to add unicode path support to FilePath and let FilePath encode them). Unfortunately it's not entirely obvious how to make FilePath support unicode paths since not all platforms Twisted supports represent filesystem paths using unicode. The choice python-dev made to bridge this gap was the creation of the "surrogateescape" error handler for the UTC-8 codec. This lets you pretend that any time you need to convert between bytes and unicode the correct codec is UTF-8 (with this special error handler). It's not clear this was a good choice (since the result is unicode strings that may contain garbage which will confuse other software) but it's also not clear it's possible for Twisted to try to make any other choice (at some point Twisted has to interoperate with the path-related APIs in Python itself - `sys.path`, for example). Not sure if that helps you at all. Maybe it outlines the problem a little more clearly, at least. Jean-Paul ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] Python3: should paths be bytes or str?
The porting guide says
No byte paths in sys.path.
I am not sure what this means, I would assume that
file paths should always be native strings. Or does it mean
that sys.path must only contain ascii bytes?
doc for FilePath says
On both Python 2 and Python 3, paths can only be bytes.
and svn commit 35410 by itamarst
changed the doc for some path functions in python/filepath.py
from str to bytes
but not all of them:
fgrep 'type path:' filepath.py
@type path: L{str}
@type path: L{str}
@type path: L{bytes}
@type path: L{bytes}
@type path: L{bytes}
I stumbled upon this while trying to find out how much work it might be
to make bin/trial run with python3
admin/run-python3-tests already passes for all twisted.spread related
tests but I still need to clean up a lot.
after adding an assert to FilePath.__init__, python3 bin/trial ... gives
File "/home/wr/ssdsrc/Twisted/twisted/scripts/trial.py", line 601, in run
config.parseOptions()
File "/home/wr/ssdsrc/Twisted/twisted/python/usage.py", line 277, in
parseOptions
self.postOptions()
File "/home/wr/ssdsrc/Twisted/twisted/scripts/trial.py", line 472, in
postOptions
_BasicOptions.postOptions(self)
File "/home/wr/ssdsrc/Twisted/twisted/scripts/trial.py", line 382, in
postOptions
self['reporter'] = self._loadReporterByName(self['reporter'])
File "/home/wr/ssdsrc/Twisted/twisted/scripts/trial.py", line 369, in
_loadReporterByName
for p in plugin.getPlugins(itrial.IReporter):
File "/home/wr/ssdsrc/Twisted/twisted/plugin.py", line 209, in getPlugins
allDropins = getCache(package)
File "/home/wr/ssdsrc/Twisted/twisted/plugin.py", line 134, in getCache
mod = getModule(module.__name__)
File "/home/wr/ssdsrc/Twisted/twisted/python/modules.py", line 781, in
getModule
return theSystemPath[moduleName]
File "/home/wr/ssdsrc/Twisted/twisted/python/modules.py", line 702, in
__getitem__
self._findEntryPathString(moduleObject)),
File "/home/wr/ssdsrc/Twisted/twisted/python/modules.py", line 627, in
_findEntryPathString
if _isPackagePath(FilePath(topPackageObj.__file__)):
File "/home/wr/ssdsrc/Twisted/twisted/python/filepath.py", line 664, in
__init__
assert isinstance(path, bytes), 'path must be bytes: %r' % (path,)
AssertionError: path must be bytes:
'/home/wr/ssdsrc/Twisted/twisted/__init__.py'
--
Wolfgang
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] pb.Copyable knowledge barrier
On Sep 7, 2014, at 12:51 AM, Kevin Mcintyre wrote: > Jelly performance factors below expectations. I am not clear on the significance of this statement. > Can we say Copyable is the lowest order jelly? The notion that a copy holder > can't ask "is my copy good anymore?" makes it so. Essentially root says, > I'd prefer not to repeat unit of work nor keep track of the resulting copies, > here have the original or resulting copy. Asking whether your copy is good any more is a PB-level task. Jelly itself is a separate layer which is about getting the right data to the right place, not keeping it updated. > My main issue is a copy-holder calling for a copy to determine is the copy is > good anymore. I know, see cacheable but it's problematic. "problematic" is passive voice :-). What are the problems? > At this point I'm unsure of what jelly actually does well. Malheureusement, Jelly's main claim to fame is "it's better than pickle". But with all this renewed interest in PB perhaps we'll get pre-deserialization schema enforcement and type checking, and then it will have some real advantages :). > who's the JellyCon2014 keynote speaker? One day we will have a conference - Tx/Rx, of course - and there will be a talk on this. But first we need several people in the community who wants their contribution to Twisted to be organization and community oriented rather than code ;-). -glyph___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] pb.Copyable knowledge barrier
Jelly performance factors below expectations. Can we say Copyable is the lowest order jelly? The notion that a copy holder can't ask "is my copy good anymore?" makes it so. Essentially root says, I'd prefer not to repeat unit of work nor keep track of the resulting copies, here have the original or resulting copy. My main issue is a copy-holder calling for a copy to determine is the copy is good anymore. I know, see cacheable but it's problematic. At this point I'm unsure of what jelly actually does well. who's the JellyCon2014 keynote speaker? On Sat, Aug 9, 2014 at 6:58 PM, Glyph wrote: > On Aug 8, 2014, at 9:17 PM, Kevin Mcintyre wrote: > > correction. I was stupidly reprocessing list on remote copy side. Seeing > much better results now ~10K per second. > > > Whew. I don't know exactly how well I would expect this perform, but > those other results looked off by a couple orders of magnitude :-). > > -glyph > > > ___ > Twisted-Python mailing list > Twisted-Python@twistedmatrix.com > http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python > > ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
